Yeah, this appeared to happen when he stepped down as king during a rebellion, which I'm assuming you lose prestige from. Seeing as he's done other prestige-costing actions since then, such as requesting family gold and moving to a new island, just resetting the prestige to the new player value would probably be generous.
