I currently have 4 fame points, 2 of them prestige related and rest 2 are for Leading an Army and Unique Item. I got the Lead an Army fame when my Marshal character took control of the army in a minor battle against monsters. Unique Item fame was received when he bought one from an adventurer.