I thank you for that stimulating answer. But it says nothing about how it is possible the centres decay when they are barely used.
If you aren't using a building, it is fair to assume that no one is up keeping it either. Similar to decrepit unused buildings in almost any city in the world. The game is assuming that the production of the region is being put to other uses, and as no one is doing building maintenance, the building is decaying. Pretty simple.
