Some of it is born talent, and some of it is luck/right place at the right time/being favored my media/sponsors etc.
I will say that sports that rely on judge decision, can be biased. In figure skating, and in gymnastics; There are many people who feel the person that won bronze, should have won gold in the event, but because the judges were being biased and jerks, and showing favoritism to certain athletes, and giving other athletes them deem unfavorable, poor scores etc.
In basketball, a team can make a Hail Mary shot from half court, and score, and that was pure luck. They won the championship. In baseball, bottom of the ninth, batter hits a home run/grand slam, and wins the world series. In American football, a player in the last few seconds of clock time/game time, is lucky enough to not get tacked make it to the end zone and score a winning touchdown. So that's where it comes to luck.
But you have to have talent and skill in some shape or form, but yeah. It's flawed in some cases, and I do get exactly what you mean OP.