Just push in 4 magnets and you're good to go

As the bits touch the magnet, they will not fall out even when upside-down.

Easy to pull out the bits with limited force