I think people are confused here. This doesn't have anything to do with size. I have been a weaker "bigger guy" overweight for a long time, and there are plenty of skinny guys who are very strong and coordinated.
This is about strength and coordination. I played baseball and basketball when I was a child. My coordination in throwing the ball accurately and with power was not the same as the other kids. I practiced, and I practiced, but my muscles just didn't develop as they did for the other boys.
I could throw a baseball accurately with moderate power on one throw, and then on the next throw completely miss wildly. The time when my arm and shoulder would cock back and then release the ball, my hand would screw it up and let go of it at the wrong time. Other times I would get it right, but it never stuck.
I gave up on sports in middle school when I didn't make the basketball team. I was really good at basketball sometimes, and then really mediocre at other times and never consistent.
When I was a soph. in HS, I took a weight training class. I was verbally bullied a little a bit by some POS, but it never lasted long, and I mostly ignored him. I was just an easy target because I hardly talked to anyone. I never ever remember saying anything to provoke him or anyone else, some people just hated me.
I had never lifted weights before, but apparently I was way weaker by default the other guys who never lifted weights either. My starting weight for benching, was the bar plus either 10 or 15 on each side. I couldn't do 25s. It took me months before I could get up to 45s, and then maybe + another 5 or 10max. And then I could only do maybe 3.
I can do 15 military perfect push ups, and maybe 25 regular ones.
I have been kinda weak my whole life obviously, and even worse that I have been mentally weak as well. I don't think they have to be tied together completely, but in my case and for guys in general, it is obvious see how being weaker and uncoordinated physically can be detrimental for developing self-confidence. Don't need to be S. Holmes to see it.