The exploit here is a band of griefers destroying the duke's prestige by swapping in and out of the duchy. This can even be plotted to appear in such a way as to be completely IC.
Given that dukes cannot control who goes in and out of their duchy, and that prestige can be so difficult to gain, I am 100% against this feature.
