Well, there would be different scales of risk corresponding to different stealth levels, and as a consequence, different amounts of damage. For instance, low level risk might be sneaking into the homes of sleeping peasants, killing the family, moving on to maybe a few others. That might get only a dozen or so, and the effect is negligible (Which is why one probably wouldn't use it). But for the higher risk stuff, well...you know how infiltrators destroy recruitment centers and fortifications? Have him mingle around when the peasants are gathered in a field for harvest, or all celebrating in the square during a festival, or something like that, and then the casualties go up.
It doesn't have to be based solely on swordfighting. I'd expect that being an infiltrator, the sword should be among the last weapons used.
