Yeah, you just can't win.
I had to wean myself off the whole not looking at people to now I stare at people too much now apparently. I don't seem to notice most of the time, but when I do I will usually try to avert my eyes for a few moments then look back. It seems to help make me look more...NT?
Even my last long term relationship of seven years got creeped out because she stated I was always staring at her. All my other Aspie tendencies drove her nuts as well. The routines, the OCD, not liking lots of light, not liking the TV (or other sounds on) when I was trying to sleep. The list goes on.
Meh, I give up.
Damned if you do and damned if you don't.