They move from regions that have a high percentage of max population to a region with a low percentage of max pop. So you should never seen them going both A>B and B>A at the same time.
The type of regions involved is not relevant. (I think...) I have seen them go from rurals to mountains, and from townslands to cities.
This should really help repopulate a depleted region or group of regions.