I have never been to a small town but from what I read from books, people living in small town seem to know each other a lot, this might be good for some people but for those who enjoy privacy, having people that know you well and might just come to your place for no apparent reason could be a nightmare.
I live in a city, the bad thing is pollution and it's really crowded, good thing is that there's some sort of anonymity, like you can have lots of privacy, and it's super convenient for grocery and stuff. It can be noisy but you can opt for apartments on a higher floor (I've once lived on the 41st floor).
I personally think a big city would be better for an aspie due to the privacy big cities can offer, but then again it depends on the level of light pollution and air pollution and noise pollution you can stand.