Assuming I remember correctly and understand the neuroscience behind it (admittedly a broad assumption, but bear with me), your perception of time very much changes as you get older.
If you'll forgive a gross overgeneralization, basically, your brain doesn't bother to form much in the way of detailed long-term memories for familiar activities and experiences. So as you age and gain more experience, you don't form as many detailed memories, thus... kind of montaging your own life. It feels like things are going by faster because they don't register on you as strongly.
There's also the fact that 20 years, for me, is less than half my life (I'm 43 now). You change a lot less between 23 and 43 than you do between 3 and 23, or even 14 and 34. So 20 years ago just doesn't feel that long ago to me, despite knowing objectively that it's kinda been a while.