I'm not on the spectrum.
You'd need to share the context of when people say you are "pushing people away".
It could be that it's said at the social level, how people perceive you're different to an NT who is generally attracted to company. Perhaps at work, you dont do small talk, and then people think you "are pushing them away". NTs will perceive that as negative instead of neutral. NTs are very needy - we need stroking and comforting to remember we are liked.
Or are you "pushing" close people away? NTs who have reason to think you should have a close tie to them, like family and relationships?
Does it feel to them that you are disinterested in their interests and their lives? That might feel like you are pushing people away. Or perhaps you do things, that look to an NT, like you don't care about them, when in fact you do.