If it's a single region that lost 3k peasants, then it will recover quickly because all of your adjacent regions will be pouring in theirs, assuming that the local region lords know what they are doing.
If many of your regions are down 3k peasants, then yes, you're in trouble, but the enemy had to take a lot of time and effort to achieve that.
Hell, if it's on the border, your enemy's regions will also deplete themselves to populate it!
