Sorry, but that's just not true. If you were to head east, you'd find plenty of them very quickly. You're just used to thinking of the Lighthouse as being a dead end.
Just like the Palm Sea, the Desert of Silhouettes probably attracts rogues all the time.