I think that there should be only two discrete slots for buildings per estate, one to aid production and one for administration. So production buildings can be built in the production slot and vice versa. Ah, for RP and fun purposes there can be some miscellaneous buildings which can be built in any slot.
That would compel lords to create more (effective) estates and to maintain good relationship with his knights.