Bias in Self Questionaries

Post Reply New Topic

So here is something I came to realize over the last few months: self-questionnaires are biased against people on the autism spectrum, in the statistical sense of the word. I learned this from a therapist I was talking with who worked with many people both on and off the spectrum. He noticed that the typical diagnostic tests for depression, bipolar, and anxiety were completely useless for people on the spectrum as they didn't answer them in the way most people answer them. Usually this is done in the fact people on the spectrum answer very low on all the tests, but from conversation with the person and observing them it's obvious they suffer from depression or anxiety (not sure about the bipolar test though ). One conclusion he came to for why this happens is that people take the questions too literally, when many of the things in these self-evaluations aren't quite literal, and the arbitrary numbering system doesn't help. Well, this also made me realize many of the tests regarding empathy and imagination are certainly biased, as those tests are chock full of non-literal language, as that's the only real way to ask about those things. I noticed myself after I took one spontaneous imagination test I was taking the questions way too literally, and thus I got a lower result than I really should of. One major problem is that I don't think the medical community at large has been doing enough to acknowledge this bias and than find ways to work around it.

Yeah, I've definitely noticed this. These sort of things tend to give me odd answers that are clearly wrong or misleading and I and those I've worked with on these things have to acknowledge and work around it.

I'm not sure how to solve this, but it is pretty frustrating.

I think the term your are looking for is confirmation bias. People who self report ASD tend to score higher on the tests then people who are unaware of autism. This is a common problem psychological testing.

No, I'm not talking about confirmation bias at all. I'm not talking about tests for autism, I'm talking that the whole format is biased against people with autism, meaning their scores do not accurately reflect reality.

This is why I keep nagging people to get an official diagnosis from an appropriately-trained and licensed mental-health professional. On-line tests seem to have been written from the presumptive point of view that any response you give could indicate that you have an ASD.

I've taken the same on-line tests over and over again, giving answers that should indicate neurotoxicity behavior, and still scored highly as having an ASD.

So, if it's not confirmation bias on the part of the person taking the test, then it's prejudicial weighting on the part of the test's writer.

Again, I'm not talking about tests that are made to test for ASD, I'm not talking about confirmation bias (though that does apply to everyone so I agree you should get professionally diagnosed), I'm talking against bias against autistic people with the general self reporting format. Really tests for ASD are the LEAST likely to exhibit this type of bias as they were specifically made with people on the spectrum in mind and statistically adjusted for such.

My husband and I tested this with the RDOS test. He accused me of trying to get an ASD result. So he took the test FOR me, and tried to make the answers exceptionally NT (without lying). I 'scored' higher. He then took it for HIM for a baseline, and then tried to make it say he had ASD (but real answers), and he scored MORE NT.

To him, it showed that there was a clear difference in the behavior between NT and ND persons, and that one would not be able to 'fake' a higher score if generally answering honestly about behaviors.


Generally speaking, however, I have this problem when it comes to basic psychological and career assessment tests (official and not). I can clearly see what the result of the answers would score someone, as well as what they are actually asking about. (I.e. asking if you ever thought about killing yourself would relate back to suicidal ideation or suicidal tendencies depending on how strongly you answer the question). I have a hard time believing that there are people that are actually 'fooled' into revealing a previously undetected psychiatric condition. (However, the psychologist who assessed me for these things and ultimately diagnosed me with ASD mentioned that due to a relatively superior intellect, that view was not surprising).

I guess I am the only person actually read what you were saying! Lol!

I get that you're talking about tests for issues not directly related to ASD (like for depression, or for OCD) being taken by people who have ASD's, and how being ASD might skew the results of the tests which are designed for NTs.

The reason being that ASD folks take the questions on the tests too literally. I can imagine that being true, and can sorta imagine how that would work.

But I am wondering if you could give us a few examples of questions like that- that an excessively literal minded person might answer differently than most folks.

One specific question I remembered was asking if you had thoughts racing in your head faster than you could speak.
With something regarding spontaneous imagination, there was something about not controlling what you imagined and interpreted it as being something akin to a hallucination, when I realized it just means that you imagine things without an initial plan in mind, which I do.

Other than straight literal mindedness, the other problem is a lack of a reference point for many of these, and addition to making it harder to understand slight figures of speech, it makes it harder to reference things on a numerical scale.

I did. I just replied to someone else first, and then to the question, but only as far as to what I had experienced. I self identify as being autistic and/or having asperger's, but didn't think that it was something I should have reiterated.

I have this problem. Take the question you give as an example.

Yes. I have more thoughts than I can effectively articulate. However, I have many things which I need to worry about (which I detailed, and then erased, but yes, a substantial list of things to consider daily). So considering that I have a plausible reason to have multiple thoughts, and it isn't resultant of something like ADD or ADHD or a manic episode or drug use, I would mark 'No'. Because when things are peaceful, and nothing is going on, then no, I do not have racing thoughts.

Every question of that type requires a similar period of reasoning, logic, and consideration.

I don't think this is related to autism specifically, but yes, I also have this problem. At that point all you really have is your confirmation bias, and it's frustrating.

Yes. It also makes these tests and assessments more onerous and difficult than they need to be. If they could be taken with a proctor that could just explain what they are asking, or if they could just ASK if, then it would be much simpler.

Another example is when they ask two separate questions in one, where the answer to one may be yes, and the answer to the other no. My husband (and NT friends) have no difficulty answering these questions, while I can argue and rationalize my way in and out of a question thoroughly enough that they no longer understand the meaning of the question, and then no one can answer it.

To which I get told that I'm making it harder than it needs to be.

ETA: As a sidenote, another reason why I have taken to disclosing my diagnosis. When people look askance at my deemed difficult thinking, that seems to explain it.

its hard enough to be self aware but also unbiased to a point where a self evaluation can even tell you anything you don't already know.
and then the questions! tbh my problem is when they're too literal or specific and it doesn't really apply to me, or when they expect a simple agree/disagree scale type of answer for a complex question.

I see mention of career assessments. Part of the reason I want to get some professional career counseling is because any self-reporting tests I do give me results that are frustratingly off-base. The most common thing is they'll suggest engineering-related careers; something I do not believe I have particular aptitude for and I know I have zero interest in doing. I suspect a skilled professional would be able to assess me properly and come to very different conclusions.

Personality tests tend to give me weird and inconsistent results, too.

Me, and yes. I agree. My same exact struggle. What I AM learning is that I may be suited to the field in some ways.