All of that is already possible. Infils are not bound to enemies of the realm. You can already have different 'clans' in the realm, which the ruler must juggle and if things get to worse you can have a civil war by one (or more) duke(s) secceeding. We've seen examples of these things on multiple occasions. Oligarch - Sirion is the most recent, although rather poorly executed example of this, but one nonetheless.
Considering that Oligarch is slowly turning the war in their favor... wouldn't say it's poorly executed. Especially when Oligarch has around 20 nobles, many of them ex-pats of Sirion.
