I literally spend the first three months with a unit of 15 men. When I finally scraped together some allies I got it up to 20.
It sucked hard, 300 CS was like my dream unit. Eventually I shanked the king in a coup, but still, 10% of a poor rural region is a raw deal, clearing 22-25 gold a tax day and actually worried my men might go up in training because then I couldn't afford to fix damage.
There really should be a code of "don't be a douchebag" for all lords.