I suspect a more likely thing is that you're giving off the wrong signals unconsciously.
For example, when I was introduced to my partner's friends, several of them thought I really didn't like them, or hated them. No one could point out what things did it exactly, but some combination of silence, closed off posture, unconscious scowling, or who knows what convinced these people that disliked them and wanted to be away from them immediately.
So, I wouldn't assume your friends and family are talking behind your back. Does your family know about your condition/diagnosis? If so, and even if not, though you can be more open if so, you could just ask, "Soandso seemed kind of put off when we met. Did I do something wrong? Am I imagining it?"
There's a phrase I picked up somewhere that I love. Never attribute to malice what can be more easily explained by stupidity. But for aspies, a corollary might be "Never attribute to malice what can be more easily explained by miscommunication or incorrect assumptions."