Sounds like Theory of Mind as I understand it. If you can't see how something makes sense, it's hard to believe anyone else can.
I was like this when younger too. My son, who is now twelve has the same problem, and I find it very hard not to get caught in the viscous circle it can cause. My wife's brother, also an Aspie, used to be very much the same way. He's 28(?) now, and starting to question his paradigms, but still has quite a lot of difficulty with it.
I think the toughest thing for an Aspie to learn, if they are like this, is to just let it go. Learning that being right isn't as important as getting along with others was one of the hardest lessons I had to learn.
Your meeting up with your brother reminds me of something that happened with my father years ago. I hadn't lived with him for about ten years, then moved in with him and his new wife (new at the time). It's amazing how fast we can slip right back into old patterns. It was like we picked up right where we had left off ten years before. During the ten year period living on my own, we got along perfectly fine. Within weeks of moving back in with him, we were both acting just like we did ten years before. It drove my step mother nuts, and she wouldn't put up with it. She ended up telling us both to grow up or get out. We both felt pretty sheepish after that, and got along much better after that. Every time I wanted to say something contradictory, I would just NOT say anything. Life was so much better. We all get long really well now.
_________________
I'm not likely to be around much longer. As before when I first signed up here years ago, I'm finding that after a long hiatus, and after only a few days back on here, I'm spending way too much time here again already. So I'm requesting my account be locked, banned or whatever. It's just time. Until then, well, I dunno...