People tend to fear or feel threatened by what they consider out of the norm. Different responses can come from that. Making up stories, avoidance, or by challenging you.
I've had some issues with this kind of thing. I've been told/thought to be I'm intimidating, a snob, lazy, manipulative, a liar etc. While I've just sat there wondering what I did to make them hate me. It's very hurtful to be accused of and treated poorly when you only ment to get along and do the right thing. So I definitively feel for you.
A method I adopted when I was younger was to just to try to be invisible. Not stand out and don't do anything that you might not see most others doing. I don't especially think that's good advice to implement though. A piece to take from it though would be to maybe take note of how relaxed looking guys conduct themselves and maybe try to emulate that. If you adopted some passive body behaviors it might take some of the edge off your presence.
When passing by people, entering a room, etc giving a nod, wave, hello, quick to anyone you may have an eye connect with could make you seem more friendly. If you don't do that other people sometimes will think you're a jerk and certain characters will get in your face about it; particularly because you're male.
Other stuff... I don't know, I still can't figure out many of these things that I run into. 
_________________
"Inside the heart of each and every one of us there is a longing to be understood by someone who really cares. When a person is understood, he or she can put up with almost anything in the world."