Yes, subtle bullying is one of the cloudiest issues I've ever seen. Some people have this thing where they'll look for all the world like they're being offensive and denigratory when it's actually meant (and may be taken) as being very friendly. Other times people will say apparently friendly things but the intention is quite the opposite.
I think it's often impossible to know the difference immediately. With a bit of luck, over time it can become clearer which it is. It's rare that anybody will admit to subtle, unprovoked aggression, so all we have is our instincts, diligent observation and reasoning.
I used to yank people's chains a lot when I was younger. I was never aware I was doing it until they lost their tempers, and even then I was often quite bewildered and couldn't see how what I'd said could have that effect. But when I looked back a few years later I could see it more clearly, and so I became more careful, more courteous. I still hand out the occasional dig, but mostly I reserve it for people I'm pretty sure won't take offense. Sometimes I use it on people who have dished it out. I think I'm fairly safe to do that because if they were just trying to be friendly then they'll probably like it, while if their motives are hostile, giving them a taste of their own medicine might teach them a bit of respect.