Ok, I asked several times and always got the same answer, but I want to have it on a record here and coming from a dev:
If the same thing was done using noble characters instead of adventurer characters - would it be ok?
Yes, for a variety of reasons.
Three of the top reasons are that nobles were always intended to be the ones using scrolls, and nobles either have to bring a unit (and face battle) or risk capture just from moving through the region, and, of course, that they can't pop into a region, read a scroll, and pop right back out with it being literally
impossible for anyone in the region to do anything about it.