Did you ever think that though the players treated it as a perk, it was never meant to function that way?
No, because it's what actual Medieval priests did as well. Also, priests used to be able to get robbed, then it was changed so they wouldn't get robbed (because they were always getting robbed). I never knew it had been changed back.
While true, it is something that I, personally, always considered a loophole. It should not be possible to courier huge amounts of gold around without risk.
So priests should be able to do something to manage the risk; simply reducing the usefulness of the class is not a fix.