I have this too where with certain people, games, places and the like I keep a mental image of the place when something bad happened.
It makes it hard to enjoy certain games in that respect when I like them, but when something terrible happened before, during or after I was playing them (with Assassin's Creed 3 I developed serotonin syndrome the night before, so I'll always associate that game with that, which sucks, because I love the game)