If I'm understanding correctly, you were trying to make a joke, correct?
If I'm understanding right, than I definitely know the feeling. It's almost like a double standard. When people make a joke about me, or involving me, that hurts me or bothers me, I'm told that I need to have a sense of humor about myself, that I shouldn't take things so seriously.
But when I try to be humorous or crackwise, suddenly I'm the bad guy, and I'm tactless and thoughtless and I hurt other people.
It's gotten to where I've just quit trying to be funny, because it'll inevitably go wrong, and another person will wind up thinking I'm a jerk. I abhor the thought of being disliked, because I'm so desperate to be liked, so instead, I opt to just not do anything that could possibly provoke any dislike. I'm just plain neutral.