Nobles never travel completely alone. They always have at least some retainers with them—even if they may leave them at their camp (or inn, or whatever) when they go a-murdering.
When you're attacking someone, it's not when he's out alone, it's when he's sleeping, among his own people.