Perhaps I have confused its usage with what you would have in military, ie. a captain could call a sergeant just that, "sergeant Jackson" and it would be quite fine. But if a private did that, there might be a hell to pay, so the private says "Sir" instead.
The sergeant would give the private latrine duty and the night watch for calling him "sir"...
Now since your argument isn't one of correctness of title, but rather one of proper etiquette, it is in fact proper for a knight to use Earl when talking to a lord of a region. if he were talking to a duke or higher, there might be a reason to be miffed.
