Which was my whole point about using large cheap units in the first attack. By increasing the number of men attacking, you increase the amount of time spent attacking the walls, doing more damage to the walls (the main point of the first attack). Its only point is to reduce the walls to the point where your quality infantry can face theirs on more level terms (fighting an enemy on level 4 walls is much, much easier than fighting one on level 7 walls) and thus win the battle.
The best way would be to have two armies, one that whittled the enemy defences with large, cheap units, while the other goes in once the walls have been reduced. This, of course, is not always feasable (especially if your realms are equal in strength). Thus, you would have to hit with large, cheap units, then refit and follow up with quality units.