They need to be nerfed and that's it. They should be worse against both humans and rogues. Being worse against rogues will help both BT and Dwi to shed some realms.
Given how rogue prone those continents are, not having effective militias would mean everyone would have to squat their capitals just in case a huge horde comes in. I don't think that's an improvement.