I don't know for sure, but I think I've always lacked eye contact. According to the best friend of my grandmother, I was "extremely shy" when she first met me when I was 2. I don't even remember what my mother's eyes look like and I last saw her about 2 weeks ago! I never accumulated that image. In fact, the only eyes I know for sure are my own, and that of a relative who I had a mock job interview with last week because he told me to look at them and I made an effort to remember what they look like.
Looking into people's eyes is mentally draining to me; it makes it difficult for me to think clearly. It's intimidating.
During school, in order to look like I was paying attention, I'd usually look at mouths. But a habit I had never broken up until college, was to generally stare down at my desk or my notebook. Once I broke that habit and started trying to look like I was paying attention, my ability to focus and recall details dropped along with my grades, but it is only a minor factor in the drop of my grades IMO.
I'm always had problems with speaking; I've never overcome the speech impediments I've always had. From grades 1-4 and from grades 9-10, I was in speech therapy classes. Both therapists always commented on my lack of eye contact, monotone voice, stiff posture, and from grades 1-4, my apparant failure to interact.