Because then they would not be the hub of the Empire. That's the whole idea behind keeping them not just alive, but giving them a prime role. It's the ceremonial and bureaucratic capital.
But how much fun is it going to be for them playing in a realm of just three people?
