I've had a difficult time maintaining eye contact as a child. My teachers would always ask me to make contact when talking to me one-on-one: even after they asked me I felt very uncomfortable doing so that I could not even make out what they were saying after I did. I don't make eye contact with my parents, I don't think they make much contact with me either. In fact, it was sometime in the 5th grade that I noticed this because my mom asked me to make eye contact with her and it made me frightened.
By that time I did have some understanding of what autism was, and that maybe I also was. I made friends at school, but never retained them or had them over at home. When I heard that the other students were regularly visiting each other, I felt left out. Nearly every student made fun of how I walked (I have no physical disability), and I tried to study how they would walk. Maybe if I could exaggerate how they walked, I would be able to walk the same as them. The point is, by the time I noticed other people made eye contact with each other when conversing, I started to notice my autistic tendencies.
Mostly what I have done since is assume a gaze in the general area between their neck, mouth, and hairline, while periodically making short eye contact. I have practiced eye contact, but still find it hard. I'll try to make it whenever the conversation is about something particularly serious.
I went out with a guy a few times, and whenever we were talking he said I was always looking in another direction. He likely felt offended about that too. Even after I tried making contact after knowing that he was not fond of my non-contact, it still did not feel right. It hurt me inside somehow.
Whenever I need to compute something I have the particular need to look away from the person I'm talking to. I cannot even maintain a gaze at that point. I roll my eyes up to clear any contact, and then I can make my calculations (or remember something, recite a definition, etc). Particularly true where I work because people are always asking me mathematical questions ("How much does a $2 exacta box with three horses cost?"). Since there's a terminal in front of me, I tend to look at the screen whenever asked a question that requires me to figure something.