I agree. After all, the game is called Battle Master. If you are ordered to drop your trade/diplomatic efforts/bureaucratic work and pick up a unit to fight, well, you pick up a unit of your choice and fight or face the consequences of not following orders.
Don't want a unit? There is always the Priest class.