There is an elevated chance of being captured, and the border guards would try to capture you. Success is not automatic, but it there is a relatively high chance of it happening.
If you get this message, that means that if the Treaty System were yet live, you would have been captured. It should give you an idea of the frequency.
However, adventurers are
not supposed to be captured just from unopened borders. It's supposed to require closed borders. So this is a bug, and one I thought I'd fixed, too.