Another Perspective on the “Autism Epidemic”
October 26th, 2007
Over the years, we have heard a lot about the “autism epidemic” and about how it will bankrupt the nation and lead to a “tsunami” of adults needing long-term residential care. You only need to look at a graph of the autism prevalence from 1993 to 2006 (USDE data) to see the terrifying news (Fig. 1):
[click on the graph to see it larger; click the "back" button on your browser to return to the post]
Figure 1.
But, there’s more to the story than meets the eye (especially if the eye is only looking at the above graph). What about a context for these numbers? What happens if we look at autism in the context of all IDEA disability categories? The graph below shows autism, mental retardation and all disabilities (all thirteen IDEA categories combined) in children ages 6 - 17 years, from 1993 to 2006 (Fig. 2):
Figure 2.
Autism is the red line ‘way down at the bottom of the graph. From this perspective, the “autism tsunami” looks more like a ripple. Another interesting thing I’d like to point out is the way the prevalence of “mental retardation” is trending downward at the same time that “autism” is trending upward. Lets look at that a bit closer (Fig. 3):
Figure 3.
It almost looks as though the two curves are mirror images of each other. In order to check that hypothesis, a little mathematical manipulation is necessary.
First, I’ll add the numbers for “autism” and “mental retardation” together and then, to correct for the gradual rise in “all disabilities” (see Fig. 2), I’ll divide that number by the total of all IDEA disability categories and express the number as a percentage of all disabilities (Fig. 4):
Figure 4
Well, look at that! As a percentage of all children with IDEA categorized disabilities, the sum of “autism” and “mental retardation” has hardly changed a bit from 1993 to 2006. This would suggest that one of the following things is happening:
[1] Children who would previously have been categorized as “mentally retarded” are now being classified as “autistic”. As Roy Grinker has pointed out in his book, Unstrange Minds, this would generally be a good thing, since the “conventional wisdom” is that autistic children have a better prognosis, vis a vis learning, than children with mental retardation.
[2] Whatever is causing the “autism epidemic” is also reducing the prevalence of mental retardation. In fact, it seems that for every child who becomes autistic, one child is spared from developing mental retardation. (see: Zero-sum game)
Of course, the second scenario is extremely unlikely, as there is nothing I can discover which could both increase the prevalence of autism and decrease the prevalence of mental retardation.
So, it appears that the “hidden horde” and all of the autistic children that people didn’t see when they were in school may have been hiding in plain sight. After all, before autism became so well known (and morphed into what it is today), the most likely category for them to be placed in was “mental retardation”.
Of course, I realize that many of the people crying “Epidemic!” will persist (or should I say perseverate?) in asserting that there must be something - besides a shift in classification - driving the rise in autism prevalence.
They will continue to insist that something - something in the vaccines, something in the environment, something in the milk, something in the whatever, but something - is causing this rise in autism prevalence.
For them I have some good news:
Whatever that something is, it is simultaneously reducing the prevalence of mental retardation by an equal amount.
Well, that’s all for now.
Prometheus
Filed under: Autism Science, Critical Thinking | 24 Comments »
“We Want a Study!!!”
October 20th, 2007
After reading Sallie Bernard’s “dissenting view” of the recent study failing to link thimerosal to neurological disorders. Ms. Bernard, you may recall, is Executive Director of SAFEMINDS (Sensible Action For Ending Mercury-Induced Neurological DisorderS) and was invited by the authors to participate in the design of the study.
Now, much has been written about Ms. Bernard’s response, which seems to be based on her concerns that the study wasn’t done correctly. The cynic in me says that she didn’t like the results, which is why she thinks that the study was flawed. However, there are a number of people in the mercury-causes-autism movement who are agitating for “someone” (i.e. “The Government”) to “do the study” (i.e. to find the connection between thimerosal/mercury and autism).
Most of the time, when I ask people who want “someone” to “do the study” what it would take to convince them that mercury doesn’t cause autism, I get an answer like:
“I want them to admit that they poisoned my kid!”
or
“Nothing will ever convince me that thimerosal doesn’t cause autism!”
For people in that frame of mind, there is really no point in doing a study that doesn’t “prove” what they already “know”. In fact, there doesn’t seem to be any point in doing any study at all, since no possible outcome would change their minds.
However, there are people who sincerely want to know if thimerosal can cause autism. These people would welcome a study showing that thimerosal doesn’t cause autism because it would let them “move on”.
I recently had the opportunity to ask one of these open-minded folks what it was about all of the other studies that failed to convince them (as I have been convinced). The answer was simple:
“We want them [i.e. The Government] to look at kids who haven’t been vaccinated!”
That’s when everything became clear to me.
The average parent (even non-average parents like Ms. Bernard) hasn’t the education, training or experience to know how to do a good study. For that matter, there are a shockingly large number of clinical researchers who lack that basic ability, as well.
Let me run you all through the process of designing a study to show that thimerosal does or does not cause autism. Perhaps you will see why “nobody” has “done the study” that these parents are yearning for. And why probably nobody ever will.
Let’s start with a few assumptions – just to give us some real number to work with.
[1] We would want to study children between the ages of 6 and 17 years of age. This would limit our subjects to those born after the start of the “autism epidemic” and before thimerosal was removed from children’s vaccines.
[2] We will assume – for the sake of calculation – that the prevalence of autism in the US general population between 6 and 17 years of age is 65 per 10,000 (the latest CDC numbers). For those who don’t have their calculators handy, this works out to 1 out of 154. We will also look at what happens if we change this number.
[3] We will use the National Immunization Survey (NIS) to establish an upper limit for the percentage of completely unvaccinated children.
This data is collected by telephone survey subsequently validated up by a review of the medical records. As such, it under-represents people without a telephone as well as people who refuse access to their medical records. The CDC uses a statistical algorithm to correct for these shortcomings, but it is only fair to say that it may still undercount children who are completely unvaccinated.
However, the reports available to the public only give the numbers of children (ages 19 – 35 months) who have received all of the recommended doses of each vaccine that they list. As a result, children who are “behind” on their vaccinations will be counted as “unvaccinated” for the particular vaccine they are missing. As a result, the calculated number of “unvaccinated” children will always be higher than the actual number.
Let me run through this, since it is a critical assumption. The 2006 NIS data shows that 93.4% of children ages 19 – 35 months had received three doses of the HIB vaccine. That means that 6.6% of children had received <em>less</em> than three doses. This 6.6% includes children who had received no HIB vaccine, as well as children who had received one or two doses.
As a result, we can assume that the actual number of completely unvaccinated children will always be less than the percentage we calculate from the NIS data. To be fair, we will use half of the calculated number as our assumed prevalence of completely unvaccinated children.
Averaging the calculated number of unvaccinated children from 1995 to 2006 gives about 6%. Half of this would be 3%, which we will use as our basic assumption for the prevalence of completely unvaccinated children.
Now to design the study!
Clearly, a number of parents want to look at unvaccinated populations to see if their autism prevalence is lower than that of the general population. Advocates, such as Dan Olmsted, have suggested using the Amish or other group as a source of these “unvaccinated children”. There are, however a few problems with that approach:
[1] The Amish or other groups (e.g. Christian Scientists) will have a number of other differences from the general population. If we were to find a difference in autism prevalence, we would not know if it was due to vaccination or something else. The Amish, for instance, live a very different lifestyle from the general US population.
Part of the push behind using the Amish or any of the other groups suggested is the perception that these groups have lower autism prevalence than the general population. In the case of the Amish, at least, this may not be true. Although Dan Olmsted was unable to find any autistic Amish children (except for one child adopted from China), the staff at the Clinic for Special Children in Lancaster County, Pennsylvania see plenty of Amish children with autism. Just call them and ask, like I did.
No, the best way to do the study would be to pick children from the general population. There are a little over 49 million children in the US between 6 and 17 (Aug 2007 US Census estimate), so there should be about 1.47 million completely unvaccinated children and about 320,000 children with autism in that age range.
[2] By choosing to select subjects in a study based on their vaccination status, you are throwing away a lot of data. A perfect example of this is a recent telephone survey, which categorized children’s vaccination status as “unvaccinated”, “partially vaccinated” and “fully vaccinated”. This is a horrible waste of data.
Since there are 15 vaccinations recommended by age 35 months, the categories in this survey work out to be: zero vaccines, one to fourteen vaccines and fifteen vaccines. A much better design would be to use autism as a categorizing variable (autism: yes/no) and look at the number of vaccines received (vaccines: 0 to 15) in each group.
[3] Autism is a much harder variable to quantify, since all of the rating scales are terribly sensitive to the training and experience of the evaluator. In addition, children change their rating numbers as they get older. It would be much better, easier and less prone to later challenge if you simply said that a child had autism or didn’t, without getting into the furball of trying to quantify the severity.
Vaccinations, on the other hand, are easy to quantify – simply count them up.
[4] If you want to be able to use the data to find possible causes of autism, it works better to match cases of autism with controls who are the same age and sex. This way, you can look at more than one possible association. On the other hand, if you compare a group of unvaccinated children to a group of vaccinated children, all you can find are conditions that are associated with vaccination.
I suspect that many of the people calling for a “study” of vaccinated vs unvaccinated are absolutely convinced that a correlation will be found. However, in the all-too-likely event that they are wrong, a study of unvaccinated children will give absolutely no useful information about other potential causes.
[5] It is much better, from a statistical point of view, to select “case” subjects for the quality that is the least prevalent. This may not be intuitively obvious, but it has to do with the number of subjects you have to enroll in order to be able to tell, in a statistically significant way, if they are different in some quantity.
For example, if you select unvaccinated children and match them with completely vaccinated children, you would need almost 2,600 children (2,577) in each group (5,154 total) to detect a two-fold difference in autism prevalence. That means to see that the autism prevalence in vaccinated children is twice that of unvaccinated children, you would need to enroll over 5,000 kids, which is expensive and time-consuming. It’s also not necessary.
If, on the other hand, we selected “cases” of autistic children and then looked for “control” children of the same age, sex and region to match them with, we would have an easier time of it. Now it would only take 258 subjects in each group (516 total) to find a two-fold difference in the rate of being “completely unvaccinated”.
What if we change the numbers?
How about an autism prevalence of 1 in 56 (as has been touted on the Internet)? Then we still need 897 in each group (1,794 total) to find a two-fold difference in autism prevalence if we select “cases” and “controls” according to vaccination status.
What if 6% of the general population is unvaccinated (and the autism prevalence is 1 in 154)? Then we only need 123 children in each group (246 total) if we select “cases” and “controls” by autism diagnosis.
Or what if we go the other way and assume that only 1% of the population is unvaccinated? Then you need 1,611 subjects in each group (3,222 total) if you selected for autism first – still lower than looking for a group of unvaccinated children.
Only if you assume that the prevalence of autism is greater than the prevalence of unvaccinated children can you make a case for selecting “cases” and “controls” on the basis of vaccination status. And even then, you’re still throwing away a lot of data.
I know that I covered a lot of ground quickly, but I hope that you’ll come away with an appreciation of why (if not how) the decisions for research studies are made and why it sometimes makes more sense (and is better) to go against your “gut feeling”.
If you’d like to try your hand at this sort of statistical wizardry, trot on over to the DSS Research website and try their free tools.
Until next time,
Prometheus
Filed under: Autism Science, Critical Thinking | 11 Comments »
Testimonials: Listening to People’s Stories
October 15th, 2007
Evidence-based medicine is based on data. “Alternative” medicine has “alternative” data : testimonials.
So, what’s the matter with testimonials? They’re just people’s stories, right?
Absolutely! So are reports of alien abduction, Bigfoot sightings and pixies in the garden.
Are all of these people lying? No.
They’re just telling the truth as they see it.
Let me tell you a little story. This is a business plan that anybody with access to the Internet can do - if you decide to try it, please don’t mention that you saw it here.
……….
The Tale of the Lucky Stockbroker
A poor stockbroker (after the dot.com bubble burst) was trying to make a living when he decided that instead of buying and selling stock, he would sell his advice. The only problem was that he first needed to establish his reputation.
That’s when he came up with a brilliant idea.
He spent a little money to get the e-mail addresses of two thousand people who had purchased stock on-line and sent them each a message. To half of them, he sent a message saying that ABC Inc. stock would go up in the next week - to the other half, he sent a message that the stock would go down. It went up.
At the end of the following week, he sent an e-mail message to the thousand people who had received the “correct” stock advice. Half of these people got a message saying that XYZ Co. stock would go up in the next week - to the other half, he sent a message saying that the stock would go down. It went down.
He kept this up for four weeks, at which time he sent the remaining 125 potential clients a message saying that they could continue getting these amazing stock tips for a mere $5000 a year - payable in advance. Almost all of them paid for a year’s subscription.
The End.
……….
If this story seems too far-fetched to be true, remember: these people had just seen this stockbroker make four correct predictions in a row.
In fact, the stockbroker would be well-advised to arrange a meeting so that all 125 of the people who had gotten four straight correct predictions could get together and compare their amazing stories.
Furthermore, he could also have gotten a few customers from the people who had gotten three out of four correct predictions.
But, let’s take this a few steps further.
What if the stockbroker had said that the stock would go up within a month - or two months? And he could have told them that the stock might go down before it went up; that would give him a wider “window” for his “prediction” to come true.
And what if he told his potential clients that not every stock “worked” for every person - that you had to keep trying stocks until you found one that “worked” for you?
Well, at that point, most folks would start slowly backing away and looking for the door.
And what about all those people who only got one wrong prediction? Or those who got a mix of right and wrong predictions? Well, they aren’t around to tell the other “potential clients” their stories.
And that’s what it’s all about, right? Telling people’s stories?
Right?
Let’s recap the “stories” in this tale:
One wrong prediction (batting avg. 0.000) - 1000
One right and one wrong prediction (0.500) - 500
Two right and one wrong prediction (0.667) - 250
Three right and one wrong prediction (0.750) - 125
Four right predictions (1.000) - 125
Overall “batting average” - 0.500 - the same as flipping a coin.
So, out of the original two thousand (2000), 1500 (75%) think this guy is no better than flipping a coin (and 1000 think he’s worse). But those stories don’t get told.
The same thing happens in “alternative” autism therapies.
Don’t get me wrong. I firmly believe that the majority of “alternative” autism practitioners are sincere and truly think that they are helping people. Honesty and sincerity do not prevent them from being wrong, however.
But, with the Tale of the Lucky Stockbroker in mind, think about these points:
……….
[1] Autism is a syndrome of developmental delay, not developmental stasis - as I’ve said nearly a million times before. Children will continue to develop whether they are treated or no.
[2] Like their typical peers, autistic children develop in spurts, with periods of rapid change followed by longer periods of little or no change. This provides opportunities for “therapies” to appear to work. This is true for “mainstream” and “alternative” therapies, alike. The difference is that “mainstream” (evidence-based) therapies are tested using lots of subjects, along with placebos and blinded observers, which helps “zero out” random events.
[3] Studies have shown that as much as 19% of autistic children will “move off the autistic spectrum” before their seventh birthday. Thus, children “recover” or even get “cured” spontaneously. Even if some people don’t like to hear that.
[4] What happens to parents whose kids don’t get better? For the most part, they are encouraged - by other parents and by the practitioners themselves - to “keep trying”. They are also encouraged to pay close attention and to “think positively” - but they are never encouraged to doubt.
In most cases, parents who give up on “alternative” therapies simply fade away. They have enough going on in their lives that they don’t feel the need to “tell their story”. Especially when it’s not wanted.
If you have the opportunity, check in to one of the many “biomedically oriented” Internet groups and see what happens when somebody questions the idea that “biomed” can “recover” autistic children. At the least, they will be admonished - “Don’t stand in the way of other parents getting their children the help they need!”. More likely, they will be told to “Shut up!” and banned from the group. In some cases, they will be harassed and even threatened.
So, it’s not too likely that you’ll hear many dissenting “stories”. It seems that everyone’s story is valuable…unless it contradicts the received wisdom of “biomed”.
……….
Keeping in mind the story of the Tale of the Lucky Stockbroker, you can now see the value of testimonials in “alternative” autism therapy (not much). It’s not because the parents are lying; it’s not because the parents are stupid - it’s because the parents can only see their own experience.
Since they can only see their own experience, the “success stories” in “biomed” have no way of knowing if their child was going to be one of the 19% that is spontaneously “cured” or the unknown percentage who will show at least one period of rapid improvement. The only way to know that is to look at hundreds (if not thousands) of autistic children, some of whom received the treatment and some of whom didn’t.
And it would be best if the people evaluating the children didn’t know who got the treatment and who didn’t.
In other words, the only way to really know what works and what doesn’t is to use science.
Too bad about those testimonials.
Prometheus
Filed under: Autism Science, Autism Treatments, Critical Thinking | 26 Comments »
But it works…..!
October 10th, 2007
Or does it?
Why placebos “work”.
I doubt that a week goes by that I don’t hear from some parent upset or enraged that I have “dissed” their favorite “alternative” or “biomedical” treatment for autism. Their argument usually boils down to this:
“It worked for my child!”
So, why is it that I can doubt parents’ anecdotes about how [fill in the blank] helped their child’s autism?
Let me start out by stating that I do not suspect the parents of lying, deceiving, or otherwise fabricating their story. I am confident that they are telling the truth…as they know it.
So, why would people report that a treatment improved their children’s condition if that were not so? There are a number of potential reasons:
[1] Natural history of the disorder/disability
Change is the nature of the human experience. We fluctuate in weight, height (slightly), temperature, mood, energy level, appetite, etc. This is true also for people with diseases, disabilities or disorders – possibly even more so. A person with a “common cold” will experience a rapid increase in symptoms followed by a slower improvement. Even people with chronic diseases, such as emphysema, congestive heart failure and chronic pain have “good days” and “bad days”, “good months” and “bad months”.
Children with autism are not an exception to this general rule of humanity. Like “typical” children, they have days and weeks where they struggle to learn some new skill, followed by the sudden breakthrough and the slow attainment of mastery. This is then followed, in turn, by the struggle to learn some new skill. The disability of autism may make their progress slower, but it does not abolish it entirely.
So, when a parent is trying a new “therapy” on their child, be it “biomedical” or “mainstream”, there is always the chance that they will start this treatment just before their child has that “breakthrough”. The treatment had no impact on the “breakthrough” – it would have happened regardless – but the treatment gets the “credit”.
The chance of this sort of erroneous assignment of “credit” goes up dramatically if – as is often recommended by “biomedical” promoters – the parents are trying a number of different “therapies” one after the other (or in combinations). Since many of the promoters of “biomedical” therapies advise – in apparent good faith – that it may take months or even “years” for the “therapy” to work, they have a much longer “window” in which to receive the “credit” for improvement they did not cause.
Imagine, if you will, that you are suffering from a bad “cold”. At the peak of your suffering, you take an herbal remedy offered by a co-worker. He or she advises you that it may take “a few days” for it to work. Sure enough, you are better in two days! Was the herbal remedy a miracle cure or just a placebo? Since you took the remedy at the peak of your symptoms, it was predictable that you would be feeling better in a day or two, so the herbal remedy most likely had no effect.
Picture the same scenario, except now you are treating a child with a condition that is life-long and is characterized by slowed development. And also imagine you are told that the “remedy” may take up to two years to work. Is it not possible that at some time within that two year window there might be some improvement, just based on the child getting older?
Just speaking hypothetically, mind you.
[2] Placebo response
One of the reasons that clinical drug research uses placebos as a control is that people often feel better simply because they’re taking a medication they think will make them feel better. This often goes by the misnomer “placebo effect” but, since placebos (by definition) don’t have any effect, it is better to call it the “placebo response”.
The placebo response can occur in anyone – it doesn’t have to be just in the person taking the placebo. So, it is common for parents to feel that their child is doing better even when their child is getting a placebo. I saw this happen a number of times in drug trials – including the now-famous Repligen secretin trials.
Parents were often shocked when we told them (after the study was over) that their child had been receiving the placebo. More than one insisted that we give them the composition of the placebo – they were so convinced that it had helped their child that they wanted to continue giving it. That’s right – they wanted to keep giving the placebo.
By the way, nobody should be reading this with a smug, superior attitude. I’ve seen people from all walks of life, all levels of education and intelligence, fall for the placebo response. It has nothing to do with gullibility – all you have to do is want to have it work.
[3] “Accentuate the positive, eliminate the negative…”
It is simply the very natural tendency of human beings to give higher “weight” or authority to information that “confirms” what they already believe and to give less authority or “weight” to information that contradicts their beliefs. If this is done consciously, it is called “Cherry Picking”.
Parents treating their children have several psychological “pressures” pushing them to see improvement even when there is none. First, of course, is the innate desire of every parent to see their child healthy, happy and productive. Parents want to see their children get better, develop and succeed – it’s part of being a parent. And so, they will naturally tend to “accentuate the positive, eliminate the negative…”.
In addition, there is also the very human tendency to persist in a course of action once begun. This is doubly so if the course of action is taken at some cost. This “cost” can be either monetary or emotional – such as starting an unproven “therapy” that doctors, friends and family find questionable – or both.
Once parents have decided to take the “alternative therapy” route, they often feel committed to continue, if only to prove the “doubters” (e.g. doctors, friends and family) wrong. This only serves to increase the pressures to see improvement which, as mentioned above, are already a natural part of the human condition.
This is why parental reports are always viewed with a degree of suspicion by researchers. It’s not because the researchers think that the parents are lying – it’s because good researchers know the psychological pressures involved. And that’s why a good study will include a placebo – to help balance out the natural human tendency to want to see our children be healthy and happy.
[4] Errors in diagnosis
Given the variety of educational and experience levels of the people making the “diagnosis” of autism, it is not surprising that some of these diagnoses are in error. For that matter, the “diagnosis” of autism is so broad that it is quite likely that it includes many different conditions, with very different developmental trajectories.
This is not a problem limited to autism, to be sure. I often wonder how many of the “miraculous” cures attributed to “alternative” cancer therapies are due to an error in diagnosis.
At any rate, when parents of a child with a mis-diagnosis of autism try a therapy – “alternative” or “mainstream” – and their child improves faster than autistic children are “supposed” to improve (see: The Myth of Developmental Stasis), they attribute the “recovery” (or “cure”) to that therapy.
As alluded to above, there is a sub-category of “errors in diagnosis” that would be better named “errors in prognosis”. At the risk of sounding repetitive, there is an urban myth currently in circulation that children with autism do not develop without some sort of “therapy”, preferably “alternative”.
To be fair, there are also a number of medical practitioners who – to put it kindly – are “reading from an older edition”. They have not yet been brought up-to-date about the “New Autism” and are still telling parents that their autistic child will never talk, never make friends and will need to be institutionalized. They are just as wrong as those “lay experts” who say that autistic children will never talk, never make friends and will need to be institutionalized unless they receive chelation, HBOT, vitamin B6, methyl B12, …etc.
Summary:
There are a number of reasons why parents are not the best people to evaluate whether or not their children are responding to therapies. For that matter, the practitioners who recommend or administer the therapies are also not good, neutral observers.
Humans naturally try to find patterns in the world around them – even when there are no patterns. They also tend to give more credence to information that confirms what they already believe. This is not lying and it is not a moral failing – it is human nature.
The scientific methods of placebo and blinded observers (observers who do not know if the subject they are evaluating has received the treatment or placebo) were developed to keep us from fooling ourselves.
Prometheus
Filed under: Autism Science, Autism Treatments, Critical Thinking | 30 Comments »