I think autism intensifies this - I cannot spend much time with someone whose voice is high and nasal, piercing, full of hisses and clicks*, or has a lot of "vocal fry" (voice sounds crunchy. This seems to be a popular affectation lately). NTs don't even seem to notice any of this. I actually find myself turning the radio off at times because of Hiss-Click-Crunchy-Voice Overload (I do call it that). And yes, I found meetings excruciating at times when I was working, because of this.
I like contralto voices and bass voices, or smooth voices in the higher registers - unfortunately, it seems as though people pay very little attention to their own voices in the US; I was a lot happier when I lived overseas, as there were so many more people I could listen to for extended periods, quite happily.
This is a basic sensory aversion. Some people can't stand to wear wool next to their skin (I'm one of those, too, though I love it as a top layer). This is exactly like that, only with audio.
*Not like click languages - Xhosa and others. I love to hear those. More like someone who's right on top of a mic with bad, bad filters.
_________________
"I believe you find life such a problem because you think there are the good people and the bad people," said the man. "You're wrong, of course. There are, always and only, the bad people, but some of them are on opposite sides."
-- Terry Pratchett, Guards! Guards!