In a way, he's right. But, in a way, he isn't.
I think you like him for sincere reasons. You just might like him for his intelligence. I also think you might be one who likes to "take care" of people. A relationship between an Aspie who doesn't mind being taken care of, and somebody who likes to take care of people, might not be so bad.
Right now, my 60-year-old stepmother (4 years older than me) has to take care of my 85-year-old father to a certain extent. She doesn't feel frustrated by it, though, because she loves him.
When I was 25, I used to get the "hots" for women who were your age. I still find there are many attractive women in their 50's. To me, it's all in the "attitude" rather than mere physical looks.