I have 2 older .50 caliber CVA Hawken rifles... and it sounds to me like your rifle may have a "tight" bore. However, here are my 4 suggestions to "cure" your problem.
I'd do 4 things:
1) I'd use a thinner patch... maybe a .010 to .012 inch cotton or cotton denim patch thickness and lube it with plenty of lube as recommended by other Members.
2) I'd also use a thin vegetable fiber wad between the powder load and the patched ball. This will eliminate any "burn-through" of the relatively thin patching material. However, check your fired patches and if there is no "burn-though", don't bother using the thin vegetable fiber wads.
3) I'd use a different patch-lube. My black powder shooting buddy uses water-less hand soap and I tried it... it works well and it SEEMS to not only lube the ball/barrel, but tends to clean out the "crud" build-up to a point as well. I've since switched to a semi-solid mixture of bee's wax and liquid Crisco. It works well, too. However, when I run out of the bee's wax, I'm going back to using the water-less hand soap because I really didn't find anything wrong with it and it works well, too.
4) As another member recommended... I'd buy a solid brass range-rod with a "T" handle. It gives one more "pushing power" with very little "bend" when loading. If you "dry ball" and don't have a CO-2 "gun" to clear the "dry" ball, you can use that "T" handle wedged between a fork in a tree or something else you can "wedge" that "T" handle in and then, holding the rifle, "walk" the dry ball out using the screw device. By having that "T" handle, "walking" the ball out of the rifle suddenly becomes very easy. You can also "shoot" the dry ball out by putting a little FFFg or FFFFg black powder in the flash channel by removing the screw from right side of the end of the flash channel, adding a very small amount of powder, then replacing the screw, add a new cap to the nipple and fire it in a SAFE direction.
It's amazing how much velocity is created by only a very small amount of the black powder.
Strength and Honor...
Ron T.