Let’s run the vaccine risk/benefit numbers!
March 12th, 2010
While reading another ‘blog, I found - in the comments - one of the worst examples of “bad epidemiology” I have ever seen. I won’t embarass the commenter by name or by quoting, but the gist of their “argument” was that (in the US) the risk of vaccination exceeds the risk of the vaccine-preventable disease.
I think we’ve all heard that before.
There were many problems with their math, not the least of which was being too lazy to look up the actual numbers, but I realised that this was the tip of a much larger iceberg of innumeracy, especially as it pertains to understanding prevalence and risk.
So, to begin at the beginning, let us start with fractions.
Most of the time, risk or prevalence is expressed as a fraction, although it may not always look like a fraction (e.g. 1 in 100 is the same as the fraction 1/100). And - harkening back to our elementary school days - the two components of a fraction are the numerator (the top number) and the denominator (bottom number).
I bring up these apparently irrelevant mathematical issues because, in the world of risk and prevalence, there are two major types of errors: numerator errors and denominator errors (although, sometimes, there are errors of both).
In the example I mentioned above (the ‘blog comment), the risk of contracting a certain vaccine-preventable disease was calculated (wrongly, I must add) by dividing the number of people in the US who contracted that disease in a year by the population of the US. This was then stated - indirectly - to be the risk of an unvaccinated person contracting the disease.
Perhaps you’ve already noticed the error - it’s a denominator error. Since most people in the US are already immune to this disease (mostly by vaccination), the proper denominator would have been the number of unvaccinated people in the US. Let’s see how this changes the numbers:
If we use measles as an example, there were 140 cases of measles reported in 2008 (still compiling and verifying 2009 reports). At the end of 2008, according to the US Census Bureau, there were 300,459,786 people in the US. If we use the incorrect method from the example, that would give a “risk” of contracting measles of 0.47 per million per year (1 in 2,146,141 per year).
That seems a pretty low risk, doesn’t it? It’s a bit higher than your “risk” of winning the Powerball lottery, but still quite low.
But is it accurate? [Hint: No]
The correct way to calculate your risk of contracting measles is to divide the number of reported cases in the US by the number of vulnerable people in the US. By “vulnerable”, I mean those people who haven’t been vaccinated and haven’t had measles. That is a bit harder number to find.
The CDC’s NIS shows that, in 2008, 92.1% of children ages 19-35 months had received at least one MMR vaccination. Going back as far as 1994, that number seems fairly steady - about 90 - 92%. By the age of school entry, that percentage (in the 2007 - 2008 school year) was up to 94.9%. Even if we assume that this percentage doesn’t change, it would mean that - at most - 5% of the population is vulnerable to measles.
But even that isn’t an accurate number, because people born before the measles vaccine was available (1963) - and even the years immediately after the vaccine was introduced - would have gotten the disease if they weren’t vaccinated (it is highly contagious).
By 1968, the incidence of measles had dropped low enough to assume that anyone born after 1968 who was not vaccinated is not immune. So, that means that 5% of the US population age 41 or less is vulnerable to measles. This estimate compares with the value found by Hutchins et al (2004) for measles immunity in 1999, which supports the estimate.
According to the US Census Bureau, there are about 172 million people in the US age 41 years or less, so that gives us - at most - 8.6 million vulnerable people. Now the risk of contracting measles is 140 divided by 8.6 million or 16.3 per million per year (1 in 61,428).
After calculating the risk of contracting measles, we need to calculate the risk of death or serious complications. Measles has a case-fatality rate of 2 per thousand, so the risk of contracting measles and dying of it is about 0.03 per million per year - in the current situation, where 95% of the population is immune.
Other serious complications of measles include pneumonia (about 6% of cases) and encephalitis (1 per 1000 cases). Adding these to the risk of dying brings the total risk of serious complications to 0.8 per million per year. If we exclude pneumonia as a “serious” complication, the combined risk of contracting measles and having a permanent, life-altering (or life-ending) complication is 0.05 per million per year.
The risk of serious complications (i.e. death or permanent disability) from the MMR vaccine (discounting the as-yet-undemonstrated “autism connection”) is less than 1 per ten million doses (1 per million allergic reaction, less than 10% of which are “life threatening” = less than 1 per ten million), which (because the recommendation is two doses) works out to less than 0.2 per million per lifetime. With an average lifespan of 75 years, that works out to less than 0.003 per million per year, so the risk from the disease is over ten times greater than the risk of the vaccine even with 95% of the population immune.
Oh, and by the way - the MMR vaccine protects against three diseases, not just measles. We’ll just ignore that for right now.
And even this approximation doesn’t show the true risk of forgoing just the measles vaccine (let alone the MMR) because we haven’t considered how having a large immune population prevents spreading and how that has limited the number of measles cases reported.
Measles is transmitted from person-to-person, a single infection provides life-long immunity and it has no non-human reservoir and no known long-term carrier or dormant state. In this respect, it is similar to smallpox, polio, mumps, rubella, and many other vaccine-preventable diseases. If it is not transmitted, the measles virus “dies out”. It doesn’t “hang out” in the environment. That is why measles could be eradicated, just as smallpox was.
Currently (since 2000), measles is not endemic in the US, largely because there aren’t enough susceptible (non-immune) people in close enough contact to keep the virus going. Measles in the US is an imported disease that, until 2008, was rarely transmitted beyond the person importing it and any under-age (i.e. less than 2 years old) or immune-compromised people they came in contact with.
Starting in about 2008, the percentage of immune people in the US had slipped far enough that imported cases were able to spread locally in pockets of non-immune people. The August 22, 2008 edition of Morbidity and Mortality Weekly Report (MMWR) details two outbreaks of measles that occured in the US that year. In both cases, the outbreaks occured within groups that did not vaccinate for religious or philosophical reasons and were home-schooled.
This latter point is worth noting - even though these children did not attend a public or private school, they still contracted measles from one another.
Here is a telling statement from the MMWR report:
The number of measles cases reported during January 1–July 31, 2008, is the highest year-to-date since 1996. This increase was not the result of a greater number of imported cases, but was the result of greater viral transmission after importation into the United States, leading to a greater number of importation-associated cases. These importation-associated cases have occurred largely among school-aged children who were eligible for vaccination but whose parents chose not to have them vaccinated. [emphasis added]
As the percentage of non-immune people in the country rises, imported measles cases will spread to more people, further raising the risk of infection to non-immune people and increasing the already large benefit to risk ratio of vaccines.
What this shows is that those people who choose to not vaccinate should - at the least - take precautions against associating with other people who don’t vaccinate. This would help reduce their risk of infection to the levels I calculated above.
Perhaps they should wear some sort of lapel pin, similar to what many fraternal organisations (e.g. Masons, Rotarians, etc.) have. Except, of course, that instead of stepping forward and embracing when they see a fellow member (with or without secret handshake), they should immediately turn about and walk briskly in opposite directions, to avoid transmitting vaccine-preventable diseases.
As the events of 2008 showed us, there will not be a gradual increase in measles spread as vaccine coverage declines - there will most likely be an abrupt increase as the percentage of non-immune people (and their proximity to one another) crosses a critical threshold.
And it is important to note that non-immune people are not just the children of parents who choose not to have them vaccinated. They include children too young to be vaccinated and people who are immune-suppressed due to disease, cancer or genetic disorders. They include the elderly, whose immune systems are weaker, and those people who - for one reason or another - did not develop an adequate immune response to vaccination.
Those who choose to not vaccinate and think they are letting others take the risks for them are fooling themselves; they are taking the greater risk - even now.
Prometheus
Non-hyperlinked References:
Hutchins SS, Bellini WJ, Coronado V, et al. Population immunity to measles in the United States, 1999. J. Infec. Dis.. 2004; 189(Suppl 1):S91–7
Filed under: Critical Thinking, Help for the bewildered | 42 Comments »
Deductive Mis-reasoning
September 16th, 2009
At least once a day, I find myself confronting the aftermath of deductive reasoning gone wrong. Deductive reasoning is defined as “an argument (or reasoning) where the conclusion follows logically (or is a logical consequence) of its premises”. Many people - especially in ‘Blogland - are of the opinion that if their conclusions (or assertions) follow logically from their premises, that their conclusions must be true.
Unfortunately, that is not so.
Deductive reasoning (or deductive arguments) - according to the rules of Logic - can be either valid (if the conclusions are logical consequences of their premises) or invalid (if they are not). There is no “true”.
“Truth” (or “reality”, as I prefer to think of it) requires much more. For one, all of the premises of the argument must be correct (a place where many Internet writers of the conspiracy genre fail). Secondly - and most importantly - the conclusion must also be correct.
You see, there is usually more than one possible conclusion that can be logically drawn from a set of premises.
Let me use a nice neutral example to demonstrate this.
Surely the most famous champion of deductive reasoning is that creation of Arthur Conan Doyle, the great detective, Sherlock Holmes. In the opening pages of The Adventure of the Blue Carbuncle (1892), we find Sherlock Holmes making a number of startling conclusions about the owner of a hat [I will place the premise of each conclusions inside square brackets]:
“That the man was highly intellectual is of course obvious on the face of it [large hat size], and also that he was fairly well-to-do within the last three years [expensive hat], although he has now fallen upon evil days [wearing a hat three years out-of-style]. He had foresight [had a hat-securer installed on the hat], but has less now than formerly [elastic of the hat-securer is broken and not replaced], pointing to a moral retrogression, which, when taken with the decline of his fortunes, seems to indicate some evil influence, probably drink, at work upon him. This may account for the obvious fact that his wife has ceased to love him [the hat was dusty - not brushed. Presence of a wife inferred because the man was carrying a goose.].”
While these are all possible conclusions - thus, according to Logic, valid conclusions. There is, however, nothing to ensure that they are correct conclusions. Of course, they turn out to be correct in the story, but only because Sherlock Holmes had the author on his side. In real life, there is no such assurance. Here are some alternative conclusions that are equally valid and yet give a different picture of the owner.
[1] Large hat size: either a large gentleman (big people have big heads) or a large head size due to untreated (as it would be in that time) childhood hydrocephalus. The first would give no information about intellect, while the second would be consistent with reduced intellectual capacity. Of course, we now know that head size is no predictor of intellectual ability.
[2] Expensive hat: could have been purchased second-hand (which would also explain much of the other premises). Might have been a hand-me-down from a more wealthy relative or friend. Could have been stolen.
[3] Hat three years out-of-style: could have been purchased second-hand or given from a friend, relative, or generous stranger.
[4] Elastic hat-securer: see [2] and [3] above.
[5] Elastic of hat securer broken and not replaced: see [2] and [3] above.
[6] Dusty hat not brushed: too lazy to brush his own hat, slovenly, near-sighted and can’t see the dust. If we assume (as Holmes apparently did) that men of that time did not brush their own hats (even if they are noticeably dusty), he might have been a bachelor, widower or a transient visitor to London without his spouse. And even if we assume that men of that era did not cook geese, it does not mean that he was married. It could have been for a sister, the wife of a friend, his landlady, “significant other”, etc. who might cook a goose for him but not brush his hat.
So, with “alternative” conclusions that are consistent with the “data” at hand (i.e. “valid”), we draw a very different (or several very different) pictures of the hat’s errant owner.
This is a part of deductive reasoning that is not well-understood, apparently, by many people. There are usually several conclusions that can be drawn from any collection of data, but only one is correct.
On occasion, some of these conclusions can be eliminated by the simple fact that they are known (or reasonably presumed) to be false. A simple example of this is a GPS receiver, which can give you your position with information from only three satellites. Those who are adept at maths will realise that the position information from three satellites is the intersection of the surface of three spheres. Even under the best circumstances, this gives two points. Your GPS receiver resolves this apparent dilemma by rejecting the point that it presumes is nonsensical because it is either deep underground or out in space.
In most situations in science (and real life), however, the only way to resolve these multiple conclusions is to test them, through experimentation. There are generally two ways to do this: you can either directly test your conclusion (in the Sherlock Holmes example, this would be to find the hat’s owner) or you can gather more data and see if the new data contradict one or more of the possible conclusions.
Moving from fictional to real-life examples, let’s look at a few situations in which deductive reasoning has been misused.
Holmes et al, “Reduced levels of mercury in first baby haircuts of autistic children.” (Int. J. Toxicol., 2003 Jul-Aug;22(4):277-85.):
This is one of my favorite examples of poor reasoning (deductive or other), which is why I keep coming back to it. For those not intimately (or nauseatingly) familiar with this study, let me recap the “highlights”:
The authors studied hair saved from the “first baby haircut” of 94 autistic children and 45 age- and gender-matched non-autistic controls. The autistic group had a mean hair mercury of 0.47 ppm and the control group had a mean value of 3.63 ppm. They questioned the parents about various actual and potential mercury exposures during pregnancy and early childhood and came up with a “formula” that correlated these exposures to the control group’s hair mercury levels. This “formula” did not “predict” the hair mercury levels of the autistic group.
So, here are the “premises” that the authors used for their conclusion:
[1] Autistic children had far less mercury in their hair than the non-autistic controls.
[2] A formula that could take the “mercury exposures” (read the entire paper to see what this meant) of the control group and correlate it with their hair mercury levels did not predict the hair mercury levels of the autistic group.
And here is their conclusion:
“The lack of mercury in the hair of autistics may be due to a decrease in blood mercury levels feeding the hair follicles. This decrease is likely caused by the retention of the mercury inside the cells where it most likely causes its major biological damage.”
“Despite hair levels suggesting low exposure, these infants had measured exposures at least equal to a control population, suggesting that control infants were able to eliminate mercury more effectively.”
As it turns out, this conclusion was tested just about a year later. McDowell et al, in “Hair Mercury Levels in U.S. Children and Women of Childbearing Age: Reference Range Data from NHANES 1999–2000″ (Env. Health Perspect. 2004 Aug;112(11):1165-71), measured the hair mercury levels of 838 children ages 1 - 5 years (which includes the “first baby haircut” years). This much more extensive study found that the mean hair mercury level of these 838 children was 0.22 ppm, thus wiping out the first premise of Holmes et al’s conclusion.
However, even if we grant Holmes et al their obviously flawed premise (in which their “control” group had hair mercury levels which averaged over sixteen times the mean hair mercury of 838 children in the NHANES survey, there are “alternative” conclusions that do not require inventing “poor excretion”.
The first of these, although it seems obvious, is that - despite their attempts to quantify mercury exposure many years after the fact - the autistic children were actually exposed to less mercury than the control group. Given the known problems with getting accurate dietary and even medical data years after the fact, this seems a reasonable conclusion.
A second obvious conclusion is that the higher mercury exposure of the control group actually prevented autism. This seems a bit counter-intuitive, but it is logically consistent with the laboratory data reported in Holmes et al.
A third conclusion presents itself now that we know the control group in Holmes et al had hair mercury levels far out of the “normal” range (the NHANES study showed the 95th percentile to be 0.64 ppm). This conclusion is that either the control group had a massive and undiscovered exposure to mercury in their early years or the hair mercury analysis of this group was “botched” in the lab. I hope, for these children’s sake, that it was the latter.
In this example, not only did the authors fail to consider most of the alternative conclusions that were consistent with their premises (”data”), it turns out that one of their premises - the major one - was apparently incorrect.
James et al, “Cellular and mitochondrial glutathione redox imbalance in lymphoblastoid cells derived from children with autism.” (FASEB J., 2009 Aug;23(8):2374-83)
This study is a much “cleaner” example, in that there is no reason to suspect that its data is erroneous and it appears to be a well-designed study.
In this study, the authors examined cultured lymphoblastoid (precursors to lyphocytes) cells from ten autistic children (mean age 7.8 years) and ten control adults (mean age 27.7 years). They found that the cells from the autistic children had lower whole cell and mitochondrial reduced glutathione (GSH) and higher oxidised glutathione (GSSG) than the cells from the control adults.
They also found that the cells from autistic children were more sensitive to oxidative stress from thimerosal and nitric oxide (NO).
In their conclusions, the authors state:
“A potential role of subclinical mitochondrial dysfunction and altered redox homeostasis in a subset of children with autism has been previously proposed. Within the limitations of an in vitro cell model, the baseline differences in intracellular and mitochondrial glutathione redox status in autism and control LCLs cultured under identical conditions would support this possibility.” [emphasis added]
“These potentially vulnerable subpopulations need to be identified and evaluated independently because large population epidemiologic studies do not have the sensitivity to detect minor high-risk subpopulations.” [emphasis added]
As I’ve stated above, I have no issue with the methods of this study. But the parts of their conclusions that I’ve highlighted are glaringly invalid (i.e. they do not logically flow from their premises/data).
Some of you have probably already figured it out, but let me give the rest of you a hint: they studied cells from only ten autistic children.
If this were a “subpopulation” problem, a problem seen in only a “subset” of autistic children who were part of a “minor high-risk subpopulation”, why did they see it so clearly in cells from only ten children?
The only criteria used for selecting the autistic subjects was that they had to have ”at least one affected male sibling” - so they were looking only at familial autism. Even so, it is remarkable that they were able to find this “minor high-risk subpopulation” with so few subjects. This would suggest that having two autistic children is a marker for redox imbalance.
If we conclude - as the authors apparently did - that having two autistic children is a marker of redox imbalance, then it should be a relatively simple matter to test this conclusion, since the “minor high-risk subpopulation” is easily identified. I eagerly await the results of the study that looks into this.
And even if we accept the implication that all strongly familial autism is accompanied by this redox imbalance, there is still nothing in this study that shows causation. It is equally valid to conclude - from these data - that strongly familial autism is accompanied by a redox imbalance. There is nothing in this study to show causation.
In fact, since thimerosal (the only exogenous oxidant tested) was removed from childhood vaccines at a time roughly equal to the mean age of the study subjects, at least half of them had no significant exposure to thimerosal.
So, we see that we have to be very careful about blindly accepting the conclusions of scientific papers - even from studies that are otherwise well-designed and well-executed. We are fortunate that scientific papers generally include enough detail for us to determine if the data support the conclusions and if the conclusions reached by the authors are the only valid conclusions.
And if this is true for scientific papers, just imagine what is happening on ‘blogs, websites and the like.
Bottom line: ask people to “show their work”. If they won’t……well, use caution.
Prometheus
Filed under: Autism Science, Critical Thinking, Help for the bewildered | 16 Comments »
Just call me “Captain Buzz-Kill”
July 24th, 2009
Over the years that I’ve been writing this ‘blog, I’ve often been accused of keeping people from “curing” or “helping” their autistic children. I’ve always wondered what they meant by that, since I’m not doing anything I know of to prevent people from seeking whatever treatments they want for themselves or their children.
All I’m trying to do is give people information. If they are already happy with the information they have, if they have already found “The Truth”, all they have to do is ignore me.
It’s really that simple.
During a recent conversation, it finally came to me what all of those parents (and not a few practitioners) have really been telling me.
What actually bothers people is not that I am somehow standing in the way of them getting the treatments they think their children need – which is patently ridiculous - but that I am threatening their false hope. To borrow from the jargon of my youth, I’m killing their buzz. [Note: for people who were not part of the psychedelic '60's, this means that I am interfering with their enjoyment of an unreal state of mind]
Now it all makes sense! You see, while real hope is based in reality, false hope is destroyed by reality.
At some level, these angry parents know that their hope isn’t real - that’s why they are so angry with me for speaking truth to fantasy. If they were truly convinced that they were right, they wouldn’t get so upset.
If - for example - they were treating their child with antibiotics for pneumonia and somebody on a ‘blog somewhere in cyberspace was writing that antibiotics are worthless for treating infections and are all just a sham to make money for doctors and “Big Pharma” (which, sad to say, has been written on at least one ‘blog), I doubt that they would be upset. Amused, perhaps, but not upset.
One of the problems with false hope is that it is - at its heart - a lie. The people who provide the false hope may not consciously know that it is a lie and the people who believe in the false hope almost certainly aren’t consciously aware that it’s a lie, but it is a lie nonetheless. And, as most of us have discovered, one lie is never enough - to keep the first lie going, you have to tell more lies. This is true even when you are lying to yourself. You can see this in the way that many of the so-called “alternative” autism therapies are rationalized.
Let’s look at thimerosal and chelation, as an example. First, it was claimed that the rise in autism prevalence was due to mercury in the vaccines and that chelation would produce a cure. At first, it was claimed that a “few months” of chelation would “cure” or “recover” most autistic children. When this didn’t happen, the claims slipped from “months” to “years” without explanation.
When thimerosal was taken out of children’s vaccines, it was claimed that the continued rise in autism prevalence was due to “trace” amounts of mercury still in vaccines (disregarding the fact that the amount of mercury was far less than before the “autism epidemic”). When that was challenged, it was claimed that mercury from power plants, crematoria and even drifting across the ocean from China was the source.
When that proved untenable, the claim morphed from mercury to more vague (and harder to refute) “toxins” and then to the lamentable “too many, too soon” argument - the acme of vague and undefined. All this to prevent a group of people from confronting the fact that they are living on (or profiting by) false hope.
This is probably a good time to introduce the concept of “cognitive dissonance”. Cognitive dissonance – in simple terms – is the sense of unease people feel when they are trying to reconcile two contradictory ideas. In most people, this occurs when what they believe to be true (fantasy) is in conflict with what they know is true (reality). This is an uncomfortable place to be, psychologically, and most people (so my psychologist friends tell me) will avoid it if they can.
The best way to deal with cognitive dissonance is to resolve it – to consciously decide which of the contradictory ideas to keep and which to discard. Apparently, in conflicts between fantasy and reality, it makes no difference to your sense of well-being whether you discard the fantasy or the reality. Of course, if you discard reality in favor of fantasy, you will sooner or later encounter new opportunities to experience cognitive dissonance, as your false view of reality repeatedly comes in conflict with reality itself. It’s a bit like someone deciding that they would rather be driving through Colorado and so uses a Colorado road map, despite the fact that they are in Kansas – they keep bumping into situations where their false reality is contradicted (often quite forcefully).
However, my psychologist friends tell me that most people faced with cognitive dissonance use “avoidance” rather than facing their conflict and resolving it. They simply avoid, ignore or flee from situations where their fantasy is in conflict with reality. Over time, the “places” (physical and psychological) that they have to avoid grow in number. Interestingly, they also tend to lash out at people who remind them of their unresolved conflict, often accusing them of some sort of personal attack when none exists.
Does this sound like anyone you know?
Strangely enough, this insight provides me with a faint glimmer of hope. Clearly, the people who lash out when they are confronted with information have not yet resolved to abandon reality. They are still bothered – at some level – by the fact that their fantasies about autism are not supported by reality. There is still some hope that they will eventually – when the pain of their cognitive dissonance is severe enough – decide to give reality a chance.
I guess my job is to keep flinging reality at these folks until they are forced to make a choice. And hope they choose reality.
Just call me “Captain Buzz-Kill”.
Before someone accuses me of gloating or schadenfreude, let me say that I was one of those parents who bought the false hope and experienced the cognitive dissonance. I have immense sympathy for parents who find themselves in the same position today (or tomorrow). My purpose is not to ridicule them but to help them to find a way out of their dilemma. I’ve been there and I know a way out.
Of course, there are a few people who will be unable to make a choice – or will feel compelled to choose fantasy over reality. They are people who have stepped so far into the false hope of autism that they feel they cannot turn away. Some of them have “invested” their reputations in a very public way (often publically excoriating – and even slandering – people who disagreed with them) and may feel that the humiliation of admitting that they were wrong would be greater than the humiliation of persisting in error.
Others have profited greatly from the false hope they believed in and feel they cannot choose reality without facing real consequences. For these people, the cognitive dissonance is far less than the fear of the wrath of the parents who bought their false hope. I note that while several ”DAN!” (a group emphasizing the “alternative” and “biomedical” treatment of autism) practitioners have quietly dropped out of the “autism business”, few (or none) have publically admitted they were wrong. Perhaps none of them do think that they were wrong and they really dropped out of “alternative” autism practice to “spend more time with their family” or “to limit their practice”, but I like to think that at least one or two of them “saw the light” and quietly slipped away.
I don’t have any easy solutions for those “alternative” autism practitioners who find themselves facing the conflict between their autism fantasies and cold reality. They should have known better - at least better that the parents who trusted their expertise and followed their suggestions. However, even doctors are human and make mistakes. The best solution is to admit the mistake – at least to themselves – and move on in the proper direction. It would be an even larger mistake to keep on “believing” in order to evade the consequences.
Prometheus
Filed under: Autism Practitioners, Autism Treatments, Critical Thinking, Help for the bewildered | 33 Comments »