Yes, there is soooo many things to perceive and analyze in social interactions... that,s probably why it's such a common dread for people... the body want to protect itself.
But, as a full hermit, I have a few ideas before you go down a path that might make it worst in the long run...
It might be the person... if it's a "energy vampire" always talking about negative things and complaining about her bad love life... it will exhaust you more than someone who's trying to share a good time, and not just dump her s**t on you, steal your soul, and walk away.
It might be the activity... if you spend the evening playing a shooter game full volume with loud heavy metal music when you usually read a book in silence... play a board game instead. It will be way less over stimulating. The Averages (normal people) smoke pot like a chimney to try to cool their system down from the madness... at the same time... instead to lower the sound (I can play videogames on mute with soft music!) so... some fun things are not good for anyone.
Maybe you can make it last less long, and take some bathroom pauses to breathe deeply... not ideal to do XD But at work it used to help me... even when some strange culture shat beans that smelled like death XD It was still better than too many things moving around...
But I have severe adrenal fatigue, so if to heal you need to isolate yourself utterly... been there, done that, still doing it~
Enjoy~