It would be cool if leaders could invite people from lands they are at war with so there was no worry of arrest. Just set a flag that a ruler can turn off or on.
Or just let them get arrested, then they can change realm while in Prison and be released, simple.
