I recently used CO2 to blow what turned out to be an old load out of a CVA barrel. The load had been there for an unknown time, probably years, maybe decades. The bore was pretty clean (just a little light surface rust) down to where the load was. There appeared to be some heavier rust down where the powder had been. I'm not sure what kind of powder it was, though the few grains I was able to salvage after blasting the bore clean with CO2 looked very much like FFG black powder. I got the rust cleaned out pretty well, I believe, but it was noticeably more rusty where the powder had been.
You figure that sulfur is going to oxidize and form acidic compounds over time. I'd avoid leaving the gun loaded for that reason alone.
And, of course, as mentioned before, any time you leave a gun loaded, you are leaving a trap for the many people out there who don't know TO test, or HOW TO test, a muzzleloader to see if it's loaded. You could easily be leaving a lethal and attractive trap for your kids' kids to chance upon 30 years from now, if you have a heart attack, misplace the gun, etc., etc.