I took to hiding in my room early on, to avoid conflict. My family did not get me on any level and my parents were determined they were going to force me to become a normal boy. Needless to say, they failed repeatedly, but it was me who was the failure.
In school, I kept to myself and read or drew all the time. A few people knew who I was, but I was never on anyone's social radar, except for a handful of other geeks like me.
When I started working, I was fortunate enough to land in a career that allowed me to spend most of my time in a closed room, playing music and essentially talking to myself (there was an audience on the other end of the microphone, but I couldn't see them and only had to deal with them on the phone when I felt like it). I got along with most of my peers pretty well, but most were creative, funny people, so we had a good time working together. A few were vain, obnoxious egomaniacs, but I just avoided and tried to ignore them.
My problems came in dealing with management types, who feel they have to control everything, even when they don't fully understand what's being done. Those people hated me and would go out of their way to find ways to torture me.