I am a psych major but I still have a very long way to go and I am on the spectrum myself. Basically, your symptoms meet the criteria on the DSM but the doctor who told you this maybe because he doesn't think that you have Asperger's Syndrome. I know another young man who was diagnosed with ADHD but his symptoms are too close to Asperger's Syndrome to me. His doctors tested him for it but they claim that there wasn't a trace of AS in him. However, he has been known to have limited interests social with drawl.
As for your diagnosis, I would check with different experts who just might be able to find a link connected to AS.