This used to happen to me all the time, especially when I was in my basement, wrapped up in something on the computer and my parents would shout to get my attention, which annoyed the living hell out of me. At one point, they did it so much that I started hallucinating them shouting just out of the blue, and when I went upstairs and asked them what they wanted, they would just say that they didn't call for me.
Similarly, back when I was in high school, in one of my classes the as*hole kids made a habit of shooting rubber bands and paper wasps at me, as well as throwing other random stuff, and this really pissed me off. Again, it started happening so often that I would hallucinate these kids throwing stuff at me, and I would get pissed off at them just out of the blue.
If there's one thing I loathe about being on the autism spectrum, it's my sensory issues. Sometimes I just wish I could be an insensitive f**k like everyone else, though when I consider that my sensory issues have a bit of a silver lining, what with my somewhat-heightened senses, I also wonder how people could get by without "feeling" things as much as I do.