Rightful owners are the ones with power. D'Hara lost paisly D'hara didn't recover paisly for a month or more. Its not yours chenier stop acting like youre entitled to it just because you lost your duchy and have your panties in a wad
There's nothing stopping the former owner of a territory from continuing to press its claim. In fact, that's how a lot of wars happen.