Mixture of both. They'd squabble and fight but they wouldn't do that until they were mostly safely in one realm rather than just quitting en masse because they couldn't possible see themselves at home in Swordfell.
I seem to recall that we Devs though something similar with the western migration. Forecasting player reaction is a fraught issue. This is partly why I am more in favour of things that progress in game and provide mechanisms for players to feel relevant. In general I would suggest you don't target certain realms using logic regarding their culture and player base. Geographical arguments are I feel a different matter, it ends up targeting realms sure, but not because of the actions, play styles or other elements of the player interaction.
