For me, the price of being around people at all is an increase in overload. I have come to automatically note them as something I have to expend energy perceiving, because otherwise bad things can happen unexpectedly around them. This leads to a constant drain on energy unless I can make myself stop singling them out for my attention, and then when I'm not paying attention they often intrude themselves into the center of my attention, or try to. They also do this when they think I'm not paying attention but when in reality I'm just not sending out the proper signals.
The "bad things can happen unexpectedly around them" comes from a long time of finding that if I am not paying direct attention to them, or if I even just look to them as if I'm not paying direct attention to them, then they start doing things like waving their hands in my face, hitting or kicking my body, shaking my body, screaming in my ear, and other things that are highly aversive (some of them worse than all that, such as the kids who used to jump up and down on my hands and arms in an attempt to get a reaction out of me since I was reacting to nothing they could tell anyway). Then in mental institutions, shutdown was very dangerous because it ended up with me getting in trouble at best and being beaten and tied up at worst.
All of this meaning that I try to pay attention to people for safety reasons even if I am not socializing around them. I have found that it is dangerous to not notice them so I notice them. And noticing them takes effort, and if I have to notice them for long enough, it becomes impossible to notice them and also impossible to understand a whole lot of things: they take away all my energy I use for understanding things or being able to do things.
Socializing with them is hard on a whole new level because it requires a certain level of interaction. I find it rewarding to socialize with some people, but still draining with all but one or two of them. (One of them I have known and trusted for so long that her company is almost no more aversive to me than the company of an object. I can get exhausted by her, but it's much harder because my brain processes her as so thoroughly familiar that she's not tiring unless we talk to each other too long or something in which case it's the words, not the company.)
So despite enjoying socializing and needing some degree of it, the price can be very steep, and it is basically exhaustion which leaves no room for standard processing of anything or response to my environment, in extreme forms it means that I freeze more often and for longer and am able to do fewer things. So I am very careful about socializing.
My favorite people to socialize with are cats, and I have a volunteer job doing exactly that. To sit in a room full of cats, I can understand the social context far more effortlessly than a room full of humans, and I interact with them on a level that is more natural to me. It is less tiring and draining even though I am severely allergic to them.
_________________
"In my world it's a place of patterns and feel. In my world it's a haven for what is real. It's my world, nobody can steal it, but people like me, we live in the shadows." -Donna Williams