I kind of agree with JV here. I believe stepping up the ladder is one of the great appeals of BM and the ladder is pretty big for that purpose. Someone at the bottom could of course shout to someone on top "Hey, let's duel!", but he should honestly expect response somewhere along the line of "Sure, come up here where I am, I'll wait."
I've seen like 4 duel challanges, out of which 3 became duels. Personally I barely ever got to employ some of my sick burns to turn down duel challanges, and I have lots of those (burns, not challanges).