As for me, I have less trouble with unspoken social rules. Most of the unspoken rules, I find, have actually been extensively chronicled on the Internet. One might think this would make life easier, and indeed, I'm now at a point where I can predict what I need to do in most any interaction. The challenge for me, and many others I know, is in the constant, draining banter. I know how to feign enthusiasm when I'm supposed to, but it's still such a draining activity. A day of interacting with American NTs is soul-deadening. Bumping into someone is always excitement, then enthusiasm, then a complicated dance of small-talk until we finally part ways.
Tell me if I'm wrong here; I'm basing the following conversations off a pretty small sample-size of Swedes I know.
A phone call I'd have with an American acquaintance would always be enthusiastic. "Hello? Oh, hey, Steve! Good! Wow, it's been so long! How the hell have you been? Uh huh! That's so great."
Swedes I know are much more reserved. "Ah? Hej, Staffan. Good to hear from you. It's been awhile. What's going on?"
I realize not everyone is the same, but the difference is stark, to the extent where I can scare off the anxious Swede with the sort of conversation that we'd consider reserved, over here. I find it difficult to match the constant false-engagement we're obligated to show the outside world; it must be a bad fit when people feel you don't care because you don't seem excited to see them, or when groups feel you do not truly wish to belong when you don't fuss over simple, silly things.
And tangential, perhaps, but I also feel that Germany and much of Scandinavia, especially, have a better track record of resolving the conflicts that inevitably come up with AS. Proper society in the USA prefers to ignore social problems rather than deal with them. There is an unspoken rule not to rock the boat or say anything negative or critical; Germans and Swedes I've worked with have few qualms about telling me their feelings verbally. It's infinitely easier to smooth over social blunders with someone who's inclined to work it out rather than someone who dances around the issue, acting quite upset while also saying that nothing is wrong. The direct approach to conflict can feel severe at first, but in a way it's also refreshing.
Though that's a lot of detail about what's really just one in a sea of reasons. I wonder if my perceptions here are true or not.