When a realm takes control of a region, do you expect to be able to take over, police until core, build your rcs, then walk away and never come back? Or do you expect to have to come back and do some courts and police every now and then to keep it in to condition?
For some reason, people have gotten the idea into their heads, despite all evidence to the contrary, that once a region is converted and shrines/temples built, that they should never, ever have to go back and touch that region again. So the behavior that we expect and account for with region control, is completely ignored when it comes to religion.
If you convert a region in the middle of nowhere, then walk away and never come back, how could you possibly expect to have it stay yours forever? Doubly true when other religions are strong in the area, and even preaching their own faith.
So, yeah, if you convert the region and build stuff, come back every now and then. Not only will you keep more followers, you'll make more gold in contributions for your temples.