Regarding doing vs training, I can not say. On one hand, I agree that participating in a few wars is not all it takes to create a master swordsman. I agree fully. Should you take a few untrained peasants, give them spears and lead them to war it is unlikely that any who manage to survive a handful of battles would become expert warriors of legendary proportions. I was never suggesting that.
I am suggesting that without practical application, all the training in the world will only take you so far. Just as sending untrained people into battle is unlikely to them very far. What I am talking about is a class of people (knights) who trained from childhood on to become proficient warriors and how battle would influence these people. Should these highly trained warriors survive dozens of battles, I can not believe that they would not show real development in the fighting arts.
All that is really quite minor to me though. What is important to me is the ability for characters to increase their skills by doing their job. Some classes/skills allow this easily. Swordfighting/Jousting/Leadership, however, can only really see significant gain via academies. Whether this is historically accurate and realistic or not is not all that important to me.
What is important is that it would, in my opinion, be so much cooler if my character could become more skilled at swordfighting/jousting/leadership by doing his job and fighting in battles. We want people to wage war and all that as this game is BATTLEmaster. Every incentive given to encourage this behavior is a good thing in my opinion. Right now war is fun but it does have it's downsides.
I had a character in Cathay a while ago that had assumed the position of a Duke's Champion. One of his jobs was to partake in duels on the Dukes behalf. I though this was a pretty damn cool idea and it gave me motivation during the time of peace my character found himself in. Then war came and as much as I loved it I couldn't help but feel that this war was holding my character back a bit. After all, despite fighting in battle after battle the character was just not developing his swordsmanship at a rate that could even be considered worthwhile. This change would mean that going to war had one less possible negative aspect associated with it.
As far as the literal mechanics, I can only propose what I think would be good. The devs, should they take a liking to the idea, would certainly be in a better position to judge what would best work for the game as I doubt many know the workings of BM better.
Regarding my suggestion, I doubt that it would make Academy training obsolete. I would even suggest that you could STILL progress at the academy faster, and certainly with more precession, than on the battle field. TO know this for sure would require data though.
* How often, on average, does a noble engage in battle per week/month/year or whatever.
* How often, on average, does a noble participate in a battle in which his unit does not engage any other units?
* What is the average life expectancy of characters (from creation date to date of death/retirement)?
* Using the averages above, how skilled could a character become via battle alone?
* How skilled could a character become using the academy using the only the academy over the course of an average life expectancy?
Answering these questions would allow us to tune the percentages. If 90% chance of skill growth is to rapid in comparison to academies, then drop the percentage. Or make battlefield skill growth progressively more difficult as skill increases like at the academy (as long as the number of battles over a given span of time would result in similar levels of skill growth as training at the academy over that same time period would).
