I don't feel uncomfortable with eye contact at all. Well, not that much, anyway. I used to love staring into a beautiful pair of eyes. But I'm sort of an artist, and eyes have always been one of my favorite subjects.
So it's not that I don't like looking at them. It's just that they don't draw my attention like they're apparently supposed to. When I'm talking to someone, I don't feel a need to look at their eyes (unless maybe if I feel a particular affection toward them), and I feel awkward staring in the same place for so long. It's just a position I can't really hold. My eyes want to go down to their hands, or the table, or whatever else is around. I do my best to keep looking up at their face occasionally so as not to insult them. Some people take it badly if you don't look right at them the whole time. My friends have come to understand this and don't worry about it, but to others I come off as disinterested or aloof. I suppose that's partly true, but it's never in distaste.
There's also the issue of only being able to focus clearly on one thing at a time. I can look at someone, or I can listen to what they're saying, but I can't always do both.
_________________
"If knowledge can create problems, it is not through ignorance that we can solve them." - Isaac Asimov