Well if they want you out of your dukeship, they have three options: exile you until step down, convince you to step down through words/bribery, or have infiltrators stab you until you are wounded so long that you lose dukeship due to inactivity.
No words or bribery will work, my character actually has several infils on his payroll, and exile wouldn't work. Sev would hang around just to cause damage.
