No, you're most likely a ruler who pissed someone off. More likely several some-ones. No one hates an infil that much. They bounty the rulers who seceded to create new realms, or who switched sides in a war. Or the judge that tortured them or all their realm-mates. Not the infil that just did what infils do. Besides, the really good infils generally don't get found out.
And even if they do, their swordfighting skill makes putting a bounty on them unlikely to achieve much.