If it was mine, I would drill the hole thru the stock out so it was about 1/16" larger than it now is.
The additional 1/32" clearance on both sides of the screw will allow the barrel to be pushed rearward to eliminate the gap and IMO, the 1/16" larger hole will not create any real weaknesses in the stock.
It's much better to have the barrel seated against the stock than it is to have the tang screw absorb the recoil and transfer that jolt to the screw hole like it will if it is left like it is.
As for shims, the wood grain is running from butt to muzzle. The likelihood of making a 1/32" thick shim with the grain running in the same direction is very poor. If the grain on the shim was running perpendicular to the bore, it's end grain will be pointed up for all to see and it will never visually blend in with the adjacent wood.