i think i know what you're talking about. i struggle with it too. people are always accusing me of frowning at them, etc.--a lot of times i'm not even aware of it. when i'm in social settings with lots of people, most assume i don't like them. sometimes i think it's just a safety mechanism--a way of keeping people at a safe distance. at other times, it may be an expression of genuine dislike.
and in fact, when i am in large group settings, i am pretty unhappy. everything looks like a "clique" to me---whether it really is one or not. at least, this is true in large groups of people off-the-spectrum. i feel clumsy, most of the conversation makes little sense to me. small talk is something i can't do, and at times even agitates me. social politics---the "who-can-be-the-most-charming"--that only wears me out. i start feeling resentful. i rarely (if ever) go to NT gatherings any more.
i do live in a very gentle town, though. maybe that sounds kind of cheesy, but it's true. people smile (genuinely) and wave quite a bit. even though i'm still slow to socialize here, this seems to have been helpful. i find myself frowning a lot less. i don't have a sense of "social politics" here.