When using bore brushes in Muzzle loaders, it really is important to buy the kind that has good twisted brass wire that hold the bronze brushes.
The cheaper bore brushes that use twisted aluminum wire and especially the ones where the ends are snipped off, will often come apart in a ML barrel. Since that probably will require taking the breech off the barrel to get a busted bore brush out, it is best to stay away from them. This is definitely a time when less expensive or cheaper is NOT good at all.
It has been surprising to me how some "gun" oils will harden and solidify over time, even when they normally aren't advertised for long term storage protection like that in your rifle. There have been quite a few times I ran into this and only "lacquer thinner" purchased from the Big Box Hardware stores will really cut the hardened/solidified oil or protective coating. I buy it in one quart cans, but a pint is more than enough to do this kind of job. It won't hurt the bore as long as after you clean the barrel, you oil it afterwards.
Actually, I would use a G.I. "Patch Loop" and patches wet with Lacquer Thinner to "wet" the bore even before you use a bore brush. Lacquer thinner will take a lot of the solidified oil out just on the patches alone. When I could no longer get much of the solidified oil out with the lacquer thinner wet patches, then and only then would I switch to a bronze bore brush.
The .54 caliber brush you bought is and has to be larger in diameter than the bore so the brushes are forced into the rifling grooves and will really scrub well. However, with the solidified oil/preservative, the bore diameter is now smaller than .54 caliber.
What would be great to use to clean that stuff out of your bore would be a G.I. Bronze .45 cal. Pistol Brush along with some lacquer thinner. The G.I. Bore brush actually is about .50 caliber and these things are VERY strong and won't break/come apart in the barrel. These are often sold by dealers at gun shows who sell "surplus" stuff and sometimes in old fashioned Military Surplus Stores. If you can't find one, then I would recommend using a .50 caliber bronze bore brush and again, especially with the twisted brass wire on the inside.
Again, after you use Lacquer thinner and many other solvents, you need to first dry the bore as much as possible and then "wet" the bore with a good gun oil. Then use patches on a patch loop to remove excess oil.
Gus