What a commoner "wants" to do is generally of little importance. A commoner should do what his lord tells him to do. If the lord says "Travel to Asylon and battle monsters there", then it's not the commoner's place to object or question his liege's orders.
Can you defy your liege's orders, and go live in the realm next door? Sure. Should you? No. It's part of playing the role of a commoner.
Meh, any noble with a unit can arrest any adventurer anywhere. So if it comes down to the adventurer really being an unreliable pain, then go find him and arrest him and execute him. Especially if they left their homelands. It's funny.
