The only case I'd consider honor cost being justified is if someone accepts a duel challenge, but then moves away, because 1) it's cowardly and 2) he accepted the challenge himself, it was not imposed upon him by the game or another.
Otherwise, as for honor loss for not accepting, that just opens the door for a lot of griefing.