I understand how you feel, JayCat. When I was first diagnosed with AS, my first thought was, "does this mean I'm mentally ret*d?" I became very self-conscious of everything I did, because I was painfully aware that most of my behavior is a result of having AS. However, over time, I gradually came to accept myself. I don't make any apologies about who I am, what I do, or what my special interests are. Eventually you will reach this point in your life as well. Don't despair.
Anyway, to answer your question - I have lost many special interests before, mostly through naturally getting bored with them. However, I did force myself out of a special interest once due to peer pressure even though I wasn't ready to stop being in love with it. That was when I was a member of another forum where a lot of the members were really cruel and becoming popular meant you had to act a certain way (in other words, much like high school).
Anyway, my special interest at the time was a popular anime called Naruto, and that's when I noticed that a lot of the other anime geeks were getting bullied for enjoying it. They were treated much like Twilight fans are treated today. Up to that point, I had kept my love for Naruto a secret, and I decided to force myself to end my obsession with it before anyone else found out about it so I could keep being a member of the forum.
Looking back, that was really foolish of me. Instead of forcing myself to stop enjoying something I really loved, I should have just left that forum and joined a different one where my interests would have been more acceptable. But in my defense, I hadn't yet been diagnosed with AS at that time and I had no idea that Wrong Planet existed.