Most people with Aspergers have issues with eye contact - it's one of the most common symptoms. Good news, it will get easier over time. Especially if you practice. It's also easier if you're comfortable with whoever you're talking to. Also, WHAT you're talking about makes a huge impact. Small talk vs. emotionally-charged topics, sensitive issues, etc. Do the best you can - if you don't force yourself to do it at least a little, it will take a lot longer to improve, not to mention that it's really satisfying when you do manage to do it! - but if it's too uncomfortable, than don't. If you want, you can explain that you just have difficulty maintaining eye contact, or just use tricks like looking at a person's mouth instead of their eyes.