My guess is that empathy is subjective. That is to say, that empathy is your perception of someone else's emotion. Therefore, when they say you lack empathy, what they are really saying is that they can't tell what you're feeling (or, they think you aren't feeling what they expect you should). Likewise, if they say you have empathy, it just means they think you feel what they expect (because of the outward signs). Empathy, therefore, is horrendously subjective. And when people don't realize this, they start condemning people for "lacking empathy" not because that person doesn't feel, but because they don't show it "correctly" (which is to say, as expected).