I would love this possibility. Your realm got devastated by a large starvation, daimons, or you want to take long-rogue nations? Get that production back up by taking pop from the neighbor!
It would basically turn population into a forcibly exchangeable commodity. You can already loot neighbors for food or gold, this would add population to the list. An extra potential incentive to wage war on others, as well as a potential to cause friction between realms. I like the direct and meta possibilities of such a mechanic.