Please be aware of the Forum Rules of Conduct.
Why not just consolidate in the south away from the people who dislike you? Ivory Vale's former territories look awfully cozy.