Adult Asperger's Assessment (Why it's inaccurate)

Post Reply New Topic

Apparently, there is an assessment for adults with AS, and I have been given the opportunity to take a look at it and instantly spotted numerous weaknesses which I'd like to bring to the attention of the mental health community.

This test has a bias because it was written most likely by an NT with NT assumptions.

The answers which can be given for each question are...
Definitely Agree, slightly agree, slightly disagree, and definitely agree.

I have issue with many questions, however I will illustrate the questions I have the most issue with as I feel they are most likely to produce inaccurate results. I will state the question, the assumption I think is being made or my perceived purpose of the question, and then I will state why I think the question is poorly conceived.

"I usually notice car number plates or similar strings of information."

Purpose: To identify level of special detail oriented fixation.
Weakness: A person with AS/autism will likely interpret this question literally. So while the question likely strives to determine if the person has a fixation with any form of minute detail, those who an affinity to non-numerical details will answer negatively to this question if they do not consider letters, or patterns, or unusualness of rocks a "similar string of information".

"Other people frequently tell me that what I’ve said is impolite, even though I think it is polite"

Purpose: To determine the person's ability to determine the appropriateness of their comments in a social situation.
Weakness: The question starts with "Other people frequently tell me". It relies on the presence of other people and the willingness of other people to tell the individual something. It has been my observation that many with AS are only seldom in a situation where another person may be so blunt with them. Most people will not inform an individual of a rude comment they have made, under the pretense of being polite.

"I am fascinated by dates."
Purpose: To identify unusual points of fixation.
Weakness: It can only identify the very specific fixation of dates. Many people with AS have have some form of dyscalculia and are not only quite bad with numbers, but with concept of time, and dates as well. I believe most people with AS have non-numerical fixations.


"I would rather go to a library than a party."
Assumption: Parties are loud, socially active places that someone with AS would not want to be.
Weakness: The weakness is in the assumption. "Party" is not defined at all and a person with AS may or may not have social phobia. The question should state what type of party.

"I enjoy social chit-chat."
Assumption: People with AS would not enjoy social chit-chat because of social deficits.
Weakness: The weakness is in assumption. Many people with AS are quite social and will indeed chit-chat, however it is usually one sided chit-chat.

"When I talk, it isn’t always easy for others to get a word in edgeways."
I suppose this is to counteract the weakness in the question above. So fair enough.

"I am fascinated by numbers."
Again, why the fixation of the testers on numbers? What about rocks? What about letters? What about rail cars? What about beads? I think this numbers stuff is overstated.

"When I’m reading a story, I find it difficult to work out the characters’ intentions."
Assumption: A person unaware of the character's intentions would realize it.
Weakness: Many people with AS do not get the sense they are blind to the intentions of others. This question cannot reliably detect the social deficit in intends to because many with AS think they understand the gist of the situation fine and in reality are mis-interpreting.

"I would rather go to the theatre than a museum"
This is like the party question. What type of theater? What type of museum? Is there an assumption by the tester that the theater is more of a social place? Because it is not. You are far more likely to encounter a social situation at a museum.

"I find it easy to “read between the lines” when someone is talking to me."
Again, like the "when I'm reading a story" question, the person may think they can read between the lines fine and actually cannot.

"I find it easy to work out what someone is thinking or feeling just by looking at their face"
Again, see above.

"I am good at social chit-chat."
Again, see above.

"People often tell me that I keep going on and on about the same thing."
This is like the "other people frequently tell me" question. Someone has to TELL you, and most people would be too polite to do so.

"I find it difficult to imagine what it would be like to be someone else."
Assumption: A no answer means the person understand others do not see the world as they do.
Weakness: A person who thinks everyone sees the world they way they do would still answer "no" to this.

"I enjoy social occasions."
Same weakness as the "chit-chat" one.


"I find it difficult to work out people’s intentions"
Same weakness as the other reading people ones.


"I enjoy meeting new people."
Same weakness as the "chit chat" one.

I also have issue with the questions about imagination.

"I can easily tell if someone else wants to enter a conversation."
Weakness: Same as the "chit-chat" ones. There are people with AS who are sufficiently oblivious to their obliviousness.


"People often tell me that I went too far in driving my point home in a discussion."
Again, the weakness is that someone needs to be around and willing to TELL the person.


"I often find it difficult to judge if something is rude or polite"
Weakness: Assumes prior knowledge of rude and polite. Most people with AS do not consider what they say as rude even if others do.


"I can pick up quickly if someone says one thing but means another."
Again, a person with AS may think they do this just fine, when they do not.


"I find it easy to put myself in somebody else's shoes."
This is the same as the "I find it difficult to imagine..." question. Also, the figure of speech may be taken literally.


"I am good at predicting how someone will feel."
Again, the person may think they are and really aren't.


"Other people tell me I am good at understanding how they are feeling and what they are thinking."
Again, this question relies on the presence and willingness of other people when other people may not be around to do this, or, if they are, may not express this explicitly.
This may also be taken literally prompting the person to answer "No" when they actually are good at understanding how others feel. Another weakness with this is understanding how one feels is different from empathizing how they feel.

"Friends usually talk to me about their problems as they say that I am very understanding."
What if friends talk to one about their problems and do not say they they are very understanding? What if they say something else? Again, a person with AS may take this question literally and answer in a way which does not fulfill the purpose of the question.

And I could go on, but I won't because the weakness are repetitive. The questions are poorly phrased for those who may take them too literally, rely on the person's ability to perceive themselves, undefined assumptions and the contribution of others who may not exist or not contribute.

Is that test Simon Baron-Cohen's creation?

But yeah, I've seen that before and all the "people tell me that..." questions are a major problem. It depends on things people what other people might or might not say. And what about those who are nearly mute or otherwise don't talk to people much? It seems as if the test was designed with a very stereotypical ASC person in mind: "active, but odd" (talks to people a lot, but is oblivious to mistakes and so keeps going), math/number oriented, "techy" not "artsy," uninterested in people. I'm surprised there wasn't a question about trains.

Also, questions like:

"I find it easy to put myself in somebody else's shoes."
"I am good at predicting how someone will feel."
"I am good at social chit-chat."

require a certain amount of self-insight. If someone is a bit oblivious they might think they're very good at those, even if they really aren't (and especially if no one ever tells them).

I haven't read the research establishing the scale you mention above as a measurement tool. However, I can tell you that the wording on the questions, although important, is not of the utmost concern at this point. The point is, that NT tend to score, on average, about 16 points differently on the scale than those known to have AS. This is a wide enough difference to indicate statistical significance, and would indicate that further investigation might be warranted with people who score on the higher end. This test is not, in and of itself, diagnostic.

Again, I haven't read the research, but the standard way of developing these psychometric instruments is to come up with a ton of questions that a panel of experts might think NTs and AS folks would answer differently. The next step would be to administer the list of 100 questions to both populations (with at least 100 responders from each population). Statistical differences between the responses of both populations would be used to narrow the questions down to about 20 that seem to provide an indication of AS characteristics.

In addition, there would usually be something done called factor analysis, which takes the list of "good" questions and sorts them into common categories, this can be observed by looking at what questions end up grouped together. I'm familiar enough with the instrument to see that desire to be alone, attention to detail and specific interests are probably examples of such question categories. When putting a scale like this together it is important that there are at least 3 questions in each category--this stabilizes the results. Therefore, if one question is read one way by a subject (as you point out), there are still two other good questions on that topic that will balance out any spurious results from the interpretation from the specific wording on a single question. This is why some of the questions tend to seem almost repetitive, because they are grouped around key factors.

The last step, is to include some irrelevant questions. These are actually not scored, and can be included for several different reasons. This first is to limit the ability of a subject from guessing just what is being tested--they are designed to make it difficult for someone to fake the responses of an Aspie or an NT should they be tempted to try. Sometimes, these types of questions are designed to see if the subject is reading carefully, or are stated in the negative to eliminate folks who answer "strongly agree" to every question.

What I'm trying to point out to you, is that if this scale has been constructed using proper psychometric techniques, the wording issues you point out do not eliminate this test as a measurement tool to indicate those who may be AS versus those who may be NT. This is because the instrument as it stands now delivered statistically significant differences in the responses of those two populations. NOW...where the wording becomes more key and the results more questionable...are among UNdiagnosed Aspies. This tool is not enough to base a diagnosis on, and a high score MAY indicate that further investigation could be warranted. It is a much more difficult task to develop a tool that would be effective in finding Aspies who do not know they are Aspies--because during development, you would have to give the questions to 100 Aspies who don't know they are (prior to diagnosis, then later diagnose them). That's not such an easy experiment to conduct.

Not sure if that helps or if it just got really confusing. Point is, the way people interpret one question or another, here or there should not be enough to change a score from 16 to 32. Yet, if someone scores a 30, there is no guarantee they are or aren't Aspie. Same thing if they score a 35. This is not a diagnostic tool. However scores of around 32 (meaning falling within the range that most Aspies scores do and out of the range most NTs do), might indicate a need for further investigation. To my knowledge, that's the only way this scale is used. Hope that helps. Sorry to ramble...

Even with all issues presented here satisfactorily addressed, I believe it would still be (or suspect it still can be quite) difficult to accurately assess an adult with AS. I spent an appreciable sum of money talking with a well-seasoned therapist for several sessions a little less than a year ago, and at first he had no problem asking my wife (during our second hour) some questions about her dealings "with his Aspergers Syndrome". Personally, I was not completely certain at that time, but the more we talked the more sure I became and the more the therapist went the other way. I could easily see his skewed view as caused by so many things I have learned to act out in order to survive and "cope" a bit, but he could not discern the obvious from that din. So, and after since being turned away from another place until some doctor might give me a referral, I doubt I will ever be able to even get a professional assessment of the inaccurate kind.

Due, in part (perhaps) to learning how to adapt in order to "survive," as you put it.

The test under discussion (yes, it's the Baron-Cohen one) is one of the self-assessments I used. My score: 37.

Chronos, you have some valid points re: insight.

pandorazmtbox, your description of how these kinds of tests are designed is very accurate--tested on a statistically significant number of people, "balancing" questions to account for self-skewing, etc.

It's true that such a test might be most useful as a starting point/impetus to get tested professionally. However (and I've talked about this before) for me, taking it and stopping there was enough. At my age (31), even getting professionally diagnosed might be extremely difficult, as leejosepho said in the above quote.

So--my self-diagnosis is part "tested," part instinctual, and part...just a decent "label" for my "weirdness."

Another scale that might help with indicating AS as a possible explanation (I hate to even type self-diagnosis, but at $4,400 a pop for a legit consultation, I need to get over my scientist's brain), is the empathy scale. This one is harder to find, but there is one online. Similar to the one mentioned above, Aspies tend to score significantly lower than NTs. It's just one more thing to take to your doctor or therapist when they try to make the assessment of an adult. Coping mechanisms should make it hard to detect as an adult, that's how we get through life.

Do you mean the "empathy test" with only 15 questions? Someone posted that in the "Members Only" forum.

I think that it's way too short to accurately assess anything.

Yes, and I just hate it whenever some skeptical, ill-informed or outrightly-arrogant body tries to toss "convenient" in there. I have no idea how many different labels people might have for other different people, but this one is the *only* one I have ever heard that actually fits my life-long endless quacking and struggled waddling better than the glove at OJ's trial.

VERY well-said!

Short does not disprove the science behind a scale. A great factor can be accurately tested with three good questions.

AS is far too complicated an issue to diagnose from either the AQ or the empathy scale, but if you test out with 80% of the known AS population on both, it's a lot harder for therapists to automatically turn you away as NOT AS. These scales are part of standard assessment as I understand it, but actual diagnosis requires a much more thorough investigation than simply responding to a couple objective questionnaires.

Testing only for a presence of empathy (while assumingly expecting to find apathy) or wrongly looking for actual displays of same?

The test is a statistical instrument designed to generate a score that correlates as highly as possible with a clinical assessment of a diagnosis of Asperger's syndrome, and it is pretty good. For instance, Baron-Cohen S, Wheelwright S, Robinson J, Woodbury-Smith M. The Adult Asperger Assessment (AAA): a diagnostic method. J Autism Dev Disord. 2006 Nov; 36(8.):1141-2. - "Abstract. At the present time there are a large number of adults who have suspected Asperger syndrome (AS). In this paper we describe a new instrument, the Adult Asperger Assessment (AAA), developed in our clinic for adults with AS. The need for a new instrument relevant to the diagnosis of AS in adulthood arises because existing instruments are designed for use with children. Properties of the AAA include (1) being electronic, data-based, and computer-scorable; (2) linking with two screening instruments [the Autism Spectrum Quotient (AQ) and the Empathy Quotient (EQ)]; and (3) employing a more stringent set of diagnostic criteria than DSM-IV, in order to avoid false positives. The AAA is described, and its use with a series of n = 42 clinic-patients is reported. Thirty-seven of these (88%) met DSM-IV criteria, but only 34 of these (80%) met AAA criteria. The AAA is therefore more conservative than DSM-IV."

Every individual will pick questions they don't like, but that does not invalidate the questionnaire at a population level. As a screening test (and one that has become freely available) it serves a useful purpose.

You need to shop around - other threads have given diagnosis costs in the range of $400 to $1,200 and mostly towards the higher end of the range - or completely free if in education or some other circumstances (e.g. receiving state-funded psychiatric treatment).

The questions are poorly phrased and make poor assumptions.

this is a good example - i struggled to answer many questions on relevant "aspie quizzes" and such for similar reasons you describe.

here is the problem as i see it (in addition to your assessment that NT writes quiz question - really meant for observer of afflicted person, and furthermore one with awareness of how suspected aspie is received by people who don't know him, not aspie him/herself): aspies are too analytical for the questions. they will outsmart the questions, answering them "wrong" sometimes in an urgent attempt to answer them accurately / literally. there is discussion of this in the book Women From Another Planet. for example, one woman takes issue with a question of preferring to go places alone, knowing that her accurate and honest answer (that she prefers to go grocery shopping with an aide) does not indicate what it is supposed to.

also i wonder about extroverted aspies (there certainly must be a few): do they score as NTs in these types of tests? so many of the questions pertain to social avoidance / anxieties that i imagine they must.

although i was concerned with the empathizing / systemizing quotient test pair that some of the questions outside of my areas of interest would skew the test - i know full well i have a systemizing thought process, but i also knew questions re: interest in the stock market and memorization of historical dates (which leave me cold) were meant to indicate that. still, i scored as an "extreme systemizer." i think that is a good test, since it it is actually two tests and the relevant factor is the ratio of the two scores.

an adult assessment should consist of many interviews with the person in addition to any such testing, and should rely on the person's self-reporting of their experience along with exterior observation.

great topic, chronos.

There was a mother of a child with AS who had posted on here not too long ago who was tested and thought she did very well on reading facial expressions only to learn she actually did very poorly.

Personally I never got the sense I was somewhat oblivious in social situations and probably would not have realized on my own. Now I can only wonder what it is I'm missing and second guess myself. I am not entirely oblivious and I do pick up on social cues enough to conduct most of my daily interactions but I found it quite interesting that I was usually the "odd one out" when it came time to share interpretations of the stories and the characters we read about in a writing class I took. I'm not saying I was wrong all the time, in fact I think sometimes I'm more correct in my interpretations, but I I tended to interpret the stories and characters from a slightly different angle than the rest, which I suppose was more analytical and I accepted things as factual more frequently. If the writer wrote "Jimmy liked the red balloon," then I accepted it as is, while I discovered the others in the course read some deeper meaning into this, or the instructor would want to know why Jimmy likes the red balloon. Can't Jimmy just like red balloons?

And then there was this test in which one must arrange the cards with drawings on it into a story...a particular story apparently.

I'm not sure if I scored low on that because of my physical speed, which is mildly impaired, or I was just plain wrong in my sequence of things. I thought I had done well on it.