While some outside the religion might not care, others would. A pagan lord would probably pay more attention to a bishop or archbishop's mail than a common lay priest.
That's why there are elder priests and normal priests, the former of which have more options and can be clearly differentiated.