Necchi:
The corrosive element in early primers was not mercury, but potassium chlorate, which leaves a residue of potassium chloride, a hygroscopic salt. It is the salt residue which causes rusting by absorbing moisture from the atmosphere.
In fact, many early non-corrosive primers eliminated the chlorate, but retained the mercury - unfortunately, the mercury attacked the brass of the cartridge case, rendering it unfit for reloading.
Most modern priming compositions are based on lead styphnate, with frictionators and fuels added - none of these leave a hygroscopic salt residue, and none of the modern cleaning solutions are water-based, as water is not needed as a solvent for the non-salt residues.
The most one can say is that the residue cannot prevent rust, but any of the modern solvents should serve perfectly well for maintaining a primer-powered firearm.
On the other hand, water will probably work, too, if soap is added...
mhb - Mike