Well the biggest annoyance is how your unit gets set to mercenary which drains your money and stops you from training up your Trading skill. Second, people don't really sell food or if they do it's at a high price so you can't trade anything for a profit.
