There are certain accents which I associate with "being thick". I assume this has come from my few interactions with people with those accents, and those people not appearing as intelligent as myself. An arrogant statement, you may say, but this "appearing less intelligent" is from my experience of learning the same material along side them, and they being slower to grasp the concept, and becoming frustrated by this. There are only so many times you can hear "I just don't get it" in an accent before it starts to grate.
Additionally, during my time in education, those who had had elocution lessons or came from well spoken families tended to perform better in testing, and I had what I thought were better conversations with them (though this could be though a difference in interests, of the less well spoken not wanting to appear like one of the posh snobs).
As you can see from my profile info (<<<), I am from Manchester. There is a popular and cultural dislike of people from Liverpool, the other large industrial town in NW England. Scousers, being there, with their stupid accents and stupid faces - why don't they get a work ethic, the petty criminals?
When it comes to phone conversations, I think everyone will dispair when a particular accent comes through, as you know it is nothing else but a call centre.