It seems to me that a few days in the academy with a personal instructor would teach you much more about dueling than an hour or two on the battlefield. Getting to be a top-notch fighter would probably take much longer if all you did was fight battles than if you spent all your time with a private instructor.
In this case, I think that realism might have to take a backseat to a more balanced game. Also, you could assume that nobles would train with their men quite often, so it wouldn't just be "an hour or two". Adding a slight skill gain for training a unit would be realistic, but it would also encourage peacetime skillwhoring.
But having trained swordfighting for one character to the limits of the academy, I have to say that it takes LONG time, and a LOT of money. The number of people actually have the time and money to do that is probably pretty limited.
The fact that it takes so much time and money is the root of the problem.
