As you said, they're small, but it's possible. The odds are much more than 100 thousand to one and there's probably no logical reason to think that we are but it's possible.
If the Neandertals had larger brains does that mean smarter? Hypothetical: then, if so what if they already built space worthy civilizations erased evidence of the supiority and advanced civilization here then colonized another planet because they were too humane to oust a greater population of intelligent beings, but in their haste some had to stay behind and merged in very small numbers with the cro-mags and all these alien sightings are their attempt to improve the human race and make them more advanced like them?
That seems much more implausible than just saying that a species evolved on one of the billion plus planets each in one of the 100 plus galaxies. It actually seems somewhat like a science fiction to be honest (no offense, taken, I hope) . Anyway, a larger brain doesn’t signify intelligence. Einstein’s brain was smaller than that of the average human male’s and the brain of an elephant is larger than ours. There’s no doubt in my mind that elephants are intelligent but this would signify that elephants are at least 2/4s more intelligent than us.