The numbers in a region at any one time are likely not the issue. The speed with which they spawn and the waves they hit in, occasionally with larger numbers could wear down any attempt to hold. Since they wore down existing realms with decent infrastructure it is safe to assume the number in any one region are not the full picture.
Except the realms in the area didn't so much get worn down as they just abandoned the area because we were basically told it would be impossible to resist.
