Another option would be to create situations where militia get bored after a while and are more likely to kill some peasants if stationed in the region, this in turn will lower moral in the city. On the other hand, hiring and firing militia is already a very costly thing, so not 100% certain about this one. You can of course not send your militia elsewhere for instance.
I do agree that a nerf somewhere is required, but it's a delicate thing.
Increased cost over time, or a greater food consumption would be 2 more easily to be implemented suggestions I think. Or simply higher cost alltogether as they're not ruled and kept in check by nobles, they must be done so by gold.