This was shot down before. If recovery is too easy, then being beat down is without consequence, making wars pointless.
Furthermore, attacking realms can manage the damage they do. If they decide to turn a realm rogue before annexing it, then they should live with the consequences of their decision. Nothing's forcing them to devastate what they wish to colonize.