If your loyalties don't lie with your Church, then you can leave it.
If your loyalties don't lie with your realm, you can leave it.
See? Works both ways. Conflicts of loyalty are intentional, and intentionally do not have an obvious "correct" resolution.