Increasing the drill size is splitting the horn, because the flutes are not cutting, and thus the whole drill becomes a wedge. Drill side cutters also get dull, but usually don't get sharpened. JMO
When I made my first horn I remembered kindergarten (yah I can remember that far back!!!) Gloves were hooked together via a ribbon across the back and down the arms of a coat. I tether all my pegs with a hunk of leather shoe lace. Works for me.
I first make/buy a plug then put a slight taper on it. I then drill a hole a slight bit smaller than my plug. I then take a medium size file without a handle on it and put that handle end in the horn hole I just drilled and rotate it to make a tapered hole in the horn. I do this until I get the plug to fit snugly but not too tight.