The hole you were chaseing is on the top of the tube. Whatever dirt or shavings (if any) shurely fell inside.. Maybe not enough to worry about..
Anything that did go inside will be attracted to the magnet in the bottom, and even if the shavings got to the gears, they're softer than the gears and bearings. So no worries there.
I cleaned the outside 1st and the shavings were very very minimal. I get more shavings with I sharpen my knifes. I dipped the tap in gear oil to help stick the shaving to the tap and I backed out after each cut. Hardly any and it was more like dust then shavings.
I use wheel bearing grease on the tap to stick as much metal as possible but the likelyhood of any damage from those shavings is not that great.