This happens all the time. And it makes me so anxious when it happens.
The problem is, unless they're so freaking annoyed to the point of screaming, they won't make it obvious enough. So once you don't get their subtle cue, you continue to annoy them and then they're even more annoyed that you don't stop with whatever you were doing that annoys them. Their anger grows until they do reach the point of screaming at you and you finally realize there's something going on for quite some time. Then you feel like crap and you ask what's going on.
Then they think you're crazy for not seeing something so obvious (I mean, all normal people would be annoyed by it right from the start, right?)
And it often ends up being something stupid, like not greeting them properly, or talking about your special interest for three minutes even though they're bored and you didn't realize. (And I do think three minutes and not three hours). You'd think anybody can take three minutes of conversation about the stuff they're not interested in; I know I can. But no. If you're not talking about socially acceptable subjects, even ten seconds is more than they can take.