Always.
I was at a party one night at three am, and some girl walks up to me, and says, "Where the **** did you just come from?" I was so quiet that nobody noticed me. My friend walks into the room and the girl says, "Is your friend special?" He says leave him the **** alone. That's Doug. I tried making conversation with the girl, and told her I was a Libra, and Libras tend to be a little off, and she looked like she saw a ghost. She backed into the kitchen and left.
My first girlfriend in CA dumped me the day after I found out I had Aspergers. I was on my way to my friend's house, and through the window heard he was on the phone with her, and she said, "I feel like I'm dating a child." I listened to the conversation before knocking, and heard a lot of stuff.
I often catch a conversation that wasn't supposed to be for my ears.
The most common thing I get, is that I'm immature, and creepy.