Speaking voice, yes I've always disliked it. Singing voice, I didn't like it at all when I first heard it on a tape recording, but singing became my special interest, and after years of singing and listening back to recordings, I got to almost like it, though I can still feel very critical of any imperfections I think I can hear.
I think we get bombarded by the sound of people who have learned to project their voices and really get the intonation clear - comedians, commentators, teachers, newsreaders, actors, ads, etc. - so it's like feeling bad about photos of ourselves because of all the pretty photos of celebrities.
I'm always surprised and dismayed by how much I seem to mumble on spoken word recordings. When we speak, the sound of our voices travels a different path to our ears than it travels to other people's ears, so it will sound different. I guess it's hard for me to know how mumbly I'm being because I already know what I'm saying, so my ability to discern my own words isn't really tested until I listen back after more or less forgetting what I said.
I guess the cure would be to self-train for clear diction, or to find somebody who will teach it.