I definitely feel more comfortable around kids on the spectrum (doesn't matter if HFA or LFA).
As for teens and adults, it depends on the person. One genderqueer aspie teen I know I get along with well (but we've drifted apart because he's busy with a boyfriend and doesn't get along with some other people we hang out with), one aspie teen boy and one adult aspie woman I feel a bit awkward around because they're so much less socially aware than me, one aspie teen guy I mostly get along with but I've never brought up his diagnosis because his mom told me and I don't know if he knows she told me or even knows his diagnosis, so that makes it a little awkward.
As for LFA adults and teens, I've met very few, but I relate similarly to them as to younger LFA kids. None of them used much AAC, so our communication was mostly nonverbal, which I think is a lot easier for me.