sure, but he's not going to skip going to war over it. if stats lower and he needs more gold, he'll just raise taxes. Long-term, he can deal with it properly.
Yes, it is realistic to say that many nobles spent a lot of time administrating their fiefs. but you are wandering off the real topic here - do ALL of your nobles need to spend so much time managing the Duke's regions for food production that they can't possibly send a few men off to fight some monsters? We're not playing Farmville.