I agree with the logic of peasants not caring about which realm they are in. They would have been more concerned about their lords than the whole realm. Maybe get rid of loyalty for now and once things get cleaned up, add loyalty for a specific noble?
