I have done this experiment and usually found that the POI drops as you increase the powder charge, to a point anyway. This seems counterintuitive but, it was explained to me by an old timer this way:
The 3F charge will produce slightly more velocity than your 2F charge of the same weight. This will result in the ball leaving the barrel sooner ”“ duh right? On ignition, the barrel immediately begins to rise ever so slightly and continues to do so until the energy of the recoil has dissipated. With a higher velocity charge, the ball will leave the barrel sooner, therefore be less affected by the rising barrel and strike lower than the lower velocity ball would at the same point of aim. Now I am not talking about much here and in some cases, it may not even be noticible but, in theory the higher velocity charge will hit lower.
Someone else may have another idea on this but; this explanation seemed to me very plausible.