Sometimes.
With more severe autism, it can be very obvious, though there are some things that look like autism but are actually cerebral palsy or something like that.
Then moderate autism, a difference is, 99% of the time, obvious, but it may just be MR, or something else. I know a very outgoing, moderately autistic adolescent boy who has moderate mental retardation. He is social and greets people, but I heard that when he was a baby the doctors told his parents that he'd never speak.
Asperger's... I can speculate that someone may have it or some characteristics of it, yet I cannot immediately come to the conclusion, "_____ most definitely has Asperger's Syndrome! I should ask him/her about it."
I hope that answers. :P