Yeah, I feel very invaded by such stuff. It messes up my sense of 'inside' and 'outside,' if that makes any sense: people intruding to a private, internal level where they shouldn't be. It makes me feel inside-out (or something). It's less that when I was a kid, tho.
Luckily, nowadays), I just tend to freeze and stare, and being tall, large and male (and middle-aged) makes that seem intimidating, I think. When I was younger I think I looked nervous (or something...) a lot, and I got a lot more crap from random strangers, then.
You probably look like a decent person which is why those jerks called you out. And, it always seems like bullies/predators have a finely tuned instinct of who to target. I recall reading somewhere that people with PTSD tend to have a higher rate of re-victimization. I'm not sure if I properly understood what that meant, but it seems to imply that predators can tell who has and hasn't been victimized before and target those who have. Isn't human Nature wonderful? 