Considering how the average adventurer is super rich compared to the average peasant, crazy trained in swordfighting, and decked out in armour, this doesn't really fly with me
You wear armour while you sleep in the cabin? Even a highly trained sword fighter isn't going to beat down an entire crew. The whole point is that the advy IS super rich, and a peasant, therefore his disappearance is unlikely to be noticed by anyone important.