Think upon a battlefield in real life. Would cavalry charge across their own lines to be able to attack a "worthy" unit. Would they spend the time to retire to the back lines, move around their own army to relocate in a charge position, when doing so may well take so long that the tactical situation has changed? Better to charge that little unit, route them and knock them completely out to the battle as soon as possible and then turn you attention to other targets. Otherwise perhaps cavalry can just sit out the entire battle waiting for a decent target to show up within the part of the line they can reasonably target.
Sure, you can't really move a charge much once it's started, but you should be able to better aim the charge in general than a 3-man unit, even at a distance. 'cause after all, if you won't be moving your cavalry around constantly for a better angle, they won't be redeploying their units all the time either.
