I realize that us "Aspies" are not inherently toxic folks, like say a narcissist, or control freak, or a histrionic or Type-A personality or what-have-you... but if anyone put that label on us, I'd say it was more due to misunderstandings OR based on secondary manifestations of ASD/HFA, such as being openly pessimistic or getting inappropriately upset (that is, based on depressive or anxiety conditions) - our primary manifestations just seem to be regarded as mental illness or "not all there" or "spaced out". Which is not inherently toxic, but just something that's shunned and stigmatized.
So about the misinterpretation: case in point, when I was 25 had a short term girlfriend (this was 2 years before my diagnosis so I had not idea what was "wrong" with me), and I would say the wrong things by not picking up hints that she wanted to do spend time with me going out to places and I had preferences to work out or watch a certain show at a certain time - then when she got upset and I acted genuinely bewildered, she lashed out at me with "Come on Jayo, I'm not one of your little b***ches that you can treat this way!!" - this made no sense at all to me, but then running it by one of my more enlightened NT friends, he surmised that she must've thought of me as the stereotypical alpha male "stud" who could just treat women as he pleased due to having options, i.e. he picked up the hints but he just didn't give a damn.
While I was somewhat flattered by this - after all, I had things going in the looks dept, worked out, had a good IT job, and was intelligent and witty/funny... it just didn't feel right. Basically, she regarded me as TOXIC. Because, in her mind and life experience (she too was 25), this sort of behaviour is much more common in the 20-something male population than "clinical cluelessness". So she was just using heuristics, you might say.
We were only together for four months, then I was the one who cut her loose - just saying we were very different people (duh!!) and she accepted of course. I just couldn't stand her drama anymore.
I brought up this anecdote with the therapist who diagnosed me two years later, and then IT ALL MADE SENSE...and I also connected the dots with others who'd complained (either to third parties that I got word from, or to me) that I had some passive-aggressive personality disorder. Which, of course, is a very toxic type of person (I had a former housemate who was totally PAPD and a covert narcissist, he was toxic incarnate! )