You were there before the realm's units, right? That makes you the defender, and as others pointed out, "you cannot attack if you have already been declared a defender" move to another region if you want to engage them/continue looting. 
I moved there while my units were set to murderous from my own territory. Sorry, but no, your scenario is not what happened. So please don't give me a patronising explanation. I've been around here for a long time, I know what causes defender status, thank you.
This sounds like a top secret mission 
For the choicest meatsacks to send to our torture farms, of course.
