I feel like I kind of belong here.
I was mute for 2 years during high school - I would literally speak only to 3 particular people, and never while others were around. Luckily one of them knew me well enough that he could read my mind most of the time and speak for me whenever necessary.
Nowadays, I do speak but I go through times, periods or situation where it is impossible for me to do so. When I am embarrassed, I cannot speak (not that it matters, because when I am embarrassed I just run away anyway). Or sometimes if I go through a extremely bad period, I cannot speak then either. Sometimes when I start to get overstimulated or am around lots of strangers, I am unable to speak then also.
I tend think kinesthetically / visually / patterns, which in some regards makes it much more difficult for me to communicate with words. I struggle greatly with accurately expressing myself in written format, but it is much, much worse verbally. What also doesn't help is that I mix up words, letters, or my word order occasionally, along with not being able to spell - naturally people tend to mock me for these things even though it is not something I can control, it's not lack of education or me not paying attention, my brain is just wired a bit differently and sometimes gets signals mixed up in those areas.
I have given up defending myself verbally, it's just one of the times when I cannot speak. You're right in saying that all it does is make you look guilty and while it does annoy me, these days I don't waste my time correcting other peoples assumptions and judgements, it's their own problem to deal with.