This is a very complex subject.
What I think is necessary:
1. Anonymity, for both plaintiffs and the Titans themselves. If a rating system is to be implemented, I suggest granting the Titans special aliases for their Titan roles. The true identities of the Titans should be known only to Tom, but the community could conceivably come to know and recognize certain prominent Titans by their aliases. Or they could remain totally homogeneous, known only collectively, and ratings could be assigned behind the scenes instead.
2. A system of recording precedent. Be this resource known only to the Titans or available to the community at large, precedent is a valuable tool for decision making. If available to the community, it could have the beneficial effect of reducing unintentional infractions. Granted, there is always the chance of rules lawyering, but everyone should have their day in court so to speak. Like real-life judicial systems, the rules (or rather, how they are interpreted) would be organic. They would grow and change with the community. People *should* have the opportunity to disagree with decisions and make their arguments known, and the Titans *should* have the opportunity to reassess precedent and deviate from it, provided they publish the reasoning behind their decisions.
2a. A system of appeals. This could be solely Tom, who would function as a sort of Supreme Court, or the case could go to an entirely different group of Titans. If the two groups disagree, Tom could again be the final arbiter.
3. A diverse pool of Titans. This could be based on the medal system, but I would hesitate to base such an important function on such an arbitrary rating system. Then again, I have trouble thinking of a better way to do it, so perhaps that would acceptable, with a decision-rating system to compensate.
Just thoughts, anyway.