Why do we say "sex assigned at birth"?
But biological sex IS determined by the X and Y chromosomes (or by the lack of the Y chromosome); it's as simple as that.
Gender (and non-gender) is determined either by society, or by an individual person. Of course, there are cultures where there are 3 (or more) genders.
If somebody wants me to call them "they," or says they're "nonbinary," I accept the person's word on that. I'm not going to carry a sign saying that people should either be a man or a woman.
While that is what I was also taught decades ago in public school, recent science actually contradicts the popular view of XX/XY as definitive in the way I was taught. Here’s another article, this one from the editors of Scientific American in 2017:
In the cultural realm, this shift in perspective has already received a wide embrace. “Nonbinary” definitions of gender—transfeminine, genderqueer, hijra—have entered the vernacular. Less visible perhaps are the changes taking place in the biological sciences. The emerging picture that denotes “girlness” or “boyness” reveals the involvement of complex gene networks—and the entire process appears to extend far beyond a specific moment six weeks after gestation when the gonads begin to form.
To varying extents, many of us are biological hybrids on a male-female continuum. Researchers have found XY cells in a 94-year-old woman, and surgeons discovered a womb in a 70-year-old man, a father of four. New evidence suggests that the brain consists of a “mosaic” of cell types, some more yin, others further along the yang scale.
These findings have far-reaching implications beyond just updating the biology textbooks. They have particular bearing on issues of personal identity, health and the economic well-being of women. That is because arguments about innate biological differences between the sexes have persisted long past the time they should have been put to rest.
Here’s another article, this one from Stanford Medicine SCOPE in 2015.
Furthermore, scientists now understand that everyone's body is made up of a patchwork of genetically distinct cells, some of which may have a different sex than the rest. This "mosaicism" can have effects ranging from undetectable to extraordinary, such as "identical" twins of different sexes. An extremely common instance of mosaicism comes from cells passing over the placental barrier during pregnancy. Men often carry female cells from their mothers, and women carry male cells from their sons. Research has shown that these cells remain present for decades, but what effects they have on disease and behavior is an essentially unstudied question.
This is an uneasy way to think about bodies in a social world where sex is still defined in binary terms. Legal frameworks rely on being able to classify someone as male or female, and social status is often determined by the sex on a birth certificate. Parents and doctors of intersex infants face thorny ethical questions about potential surgeries, therapies, and how to raise the child. The implications of better understanding and socially recognizing DSDs are huge.
And, since the Standford article references it, the 2015 feature article from Nature:
When genetics is taken into consideration, the boundary between the sexes becomes even blurrier. Scientists have identified many of the genes involved in the main forms of DSD, and have uncovered variations in these genes that have subtle effects on a person's anatomical or physiological sex. What's more, new technologies in DNA sequencing and cell biology are revealing that almost everyone is, to varying degrees, a patchwork of genetically distinct cells, some with a sex that might not match that of the rest of their body. Some studies even suggest that the sex of each cell drives its behaviour, through a complicated network of molecular interactions. “I think there's much greater diversity within male or female, and there is certainly an area of overlap where some people can't easily define themselves within the binary structure,” says John Achermann, who studies sex development and endocrinology at University College London's Institute of Child Health.
I understand, of course, that there are exceptions. I get it. There are "exceptions" to most things.
I have known some very feminine people who identify as being a man, and vice versa.
Like I said, if a person, say, wants to be known as a woman, but appears to be a man, I have to accept that.