So, I've thought of something odd.
Society only goes up if you're active, right? It stops if you go inactive?
But... does population growth stop if you go inactive?
If not, then there are major advantages to periods of inactivity, as it seems like low-pop colonies reach their society-level downside before they get big in terms of population. You could do the equivalent of "mind control" to society level, but at lower cost and better population effects.
Thoughts?