I've never been annoyed by a voice before, but other human sounds set me off. On the weekend, my boyfriend was over playing an old Genesis game which he had played often as a kid, and he was whistling along to the repetitive MIDI music constantly. I was trying my very best to handle it and just suck it up, because I was fully aware that he was doing nothing wrong, but it was really severely getting to me. I had to get away and get my headphones because I just couldn't handle it anymore. It wasn't even a matter of being annoying, I couldn't sit there anymore. Oddly enough, whistling by itself doesn't really bother me, just whistling along to something it seems.
Singing also sometimes sets me off, but only with certain people, mainly female.