Deductive Mis-reasoning
September 16th, 2009
At least once a day, I find myself confronting the aftermath of deductive reasoning gone wrong. Deductive reasoning is defined as “an argument (or reasoning) where the conclusion follows logically (or is a logical consequence) of its premises”. Many people - especially in ‘Blogland - are of the opinion that if their conclusions (or assertions) follow logically from their premises, that their conclusions must be true.
Unfortunately, that is not so.
Deductive reasoning (or deductive arguments) - according to the rules of Logic - can be either valid (if the conclusions are logical consequences of their premises) or invalid (if they are not). There is no “true”.
“Truth” (or “reality”, as I prefer to think of it) requires much more. For one, all of the premises of the argument must be correct (a place where many Internet writers of the conspiracy genre fail). Secondly - and most importantly - the conclusion must also be correct.
You see, there is usually more than one possible conclusion that can be logically drawn from a set of premises.
Let me use a nice neutral example to demonstrate this.
Surely the most famous champion of deductive reasoning is that creation of Arthur Conan Doyle, the great detective, Sherlock Holmes. In the opening pages of The Adventure of the Blue Carbuncle (1892), we find Sherlock Holmes making a number of startling conclusions about the owner of a hat [I will place the premise of each conclusions inside square brackets]:
“That the man was highly intellectual is of course obvious on the face of it [large hat size], and also that he was fairly well-to-do within the last three years [expensive hat], although he has now fallen upon evil days [wearing a hat three years out-of-style]. He had foresight [had a hat-securer installed on the hat], but has less now than formerly [elastic of the hat-securer is broken and not replaced], pointing to a moral retrogression, which, when taken with the decline of his fortunes, seems to indicate some evil influence, probably drink, at work upon him. This may account for the obvious fact that his wife has ceased to love him [the hat was dusty - not brushed. Presence of a wife inferred because the man was carrying a goose.].”
While these are all possible conclusions - thus, according to Logic, valid conclusions. There is, however, nothing to ensure that they are correct conclusions. Of course, they turn out to be correct in the story, but only because Sherlock Holmes had the author on his side. In real life, there is no such assurance. Here are some alternative conclusions that are equally valid and yet give a different picture of the owner.
[1] Large hat size: either a large gentleman (big people have big heads) or a large head size due to untreated (as it would be in that time) childhood hydrocephalus. The first would give no information about intellect, while the second would be consistent with reduced intellectual capacity. Of course, we now know that head size is no predictor of intellectual ability.
[2] Expensive hat: could have been purchased second-hand (which would also explain much of the other premises). Might have been a hand-me-down from a more wealthy relative or friend. Could have been stolen.
[3] Hat three years out-of-style: could have been purchased second-hand or given from a friend, relative, or generous stranger.
[4] Elastic hat-securer: see [2] and [3] above.
[5] Elastic of hat securer broken and not replaced: see [2] and [3] above.
[6] Dusty hat not brushed: too lazy to brush his own hat, slovenly, near-sighted and can’t see the dust. If we assume (as Holmes apparently did) that men of that time did not brush their own hats (even if they are noticeably dusty), he might have been a bachelor, widower or a transient visitor to London without his spouse. And even if we assume that men of that era did not cook geese, it does not mean that he was married. It could have been for a sister, the wife of a friend, his landlady, “significant other”, etc. who might cook a goose for him but not brush his hat.
So, with “alternative” conclusions that are consistent with the “data” at hand (i.e. “valid”), we draw a very different (or several very different) pictures of the hat’s errant owner.
This is a part of deductive reasoning that is not well-understood, apparently, by many people. There are usually several conclusions that can be drawn from any collection of data, but only one is correct.
On occasion, some of these conclusions can be eliminated by the simple fact that they are known (or reasonably presumed) to be false. A simple example of this is a GPS receiver, which can give you your position with information from only three satellites. Those who are adept at maths will realise that the position information from three satellites is the intersection of the surface of three spheres. Even under the best circumstances, this gives two points. Your GPS receiver resolves this apparent dilemma by rejecting the point that it presumes is nonsensical because it is either deep underground or out in space.
In most situations in science (and real life), however, the only way to resolve these multiple conclusions is to test them, through experimentation. There are generally two ways to do this: you can either directly test your conclusion (in the Sherlock Holmes example, this would be to find the hat’s owner) or you can gather more data and see if the new data contradict one or more of the possible conclusions.
Moving from fictional to real-life examples, let’s look at a few situations in which deductive reasoning has been misused.
Holmes et al, “Reduced levels of mercury in first baby haircuts of autistic children.” (Int. J. Toxicol., 2003 Jul-Aug;22(4):277-85.):
This is one of my favorite examples of poor reasoning (deductive or other), which is why I keep coming back to it. For those not intimately (or nauseatingly) familiar with this study, let me recap the “highlights”:
The authors studied hair saved from the “first baby haircut” of 94 autistic children and 45 age- and gender-matched non-autistic controls. The autistic group had a mean hair mercury of 0.47 ppm and the control group had a mean value of 3.63 ppm. They questioned the parents about various actual and potential mercury exposures during pregnancy and early childhood and came up with a “formula” that correlated these exposures to the control group’s hair mercury levels. This “formula” did not “predict” the hair mercury levels of the autistic group.
So, here are the “premises” that the authors used for their conclusion:
[1] Autistic children had far less mercury in their hair than the non-autistic controls.
[2] A formula that could take the “mercury exposures” (read the entire paper to see what this meant) of the control group and correlate it with their hair mercury levels did not predict the hair mercury levels of the autistic group.
And here is their conclusion:
“The lack of mercury in the hair of autistics may be due to a decrease in blood mercury levels feeding the hair follicles. This decrease is likely caused by the retention of the mercury inside the cells where it most likely causes its major biological damage.”
“Despite hair levels suggesting low exposure, these infants had measured exposures at least equal to a control population, suggesting that control infants were able to eliminate mercury more effectively.”
As it turns out, this conclusion was tested just about a year later. McDowell et al, in “Hair Mercury Levels in U.S. Children and Women of Childbearing Age: Reference Range Data from NHANES 1999–2000″ (Env. Health Perspect. 2004 Aug;112(11):1165-71), measured the hair mercury levels of 838 children ages 1 - 5 years (which includes the “first baby haircut” years). This much more extensive study found that the mean hair mercury level of these 838 children was 0.22 ppm, thus wiping out the first premise of Holmes et al’s conclusion.
However, even if we grant Holmes et al their obviously flawed premise (in which their “control” group had hair mercury levels which averaged over sixteen times the mean hair mercury of 838 children in the NHANES survey, there are “alternative” conclusions that do not require inventing “poor excretion”.
The first of these, although it seems obvious, is that - despite their attempts to quantify mercury exposure many years after the fact - the autistic children were actually exposed to less mercury than the control group. Given the known problems with getting accurate dietary and even medical data years after the fact, this seems a reasonable conclusion.
A second obvious conclusion is that the higher mercury exposure of the control group actually prevented autism. This seems a bit counter-intuitive, but it is logically consistent with the laboratory data reported in Holmes et al.
A third conclusion presents itself now that we know the control group in Holmes et al had hair mercury levels far out of the “normal” range (the NHANES study showed the 95th percentile to be 0.64 ppm). This conclusion is that either the control group had a massive and undiscovered exposure to mercury in their early years or the hair mercury analysis of this group was “botched” in the lab. I hope, for these children’s sake, that it was the latter.
In this example, not only did the authors fail to consider most of the alternative conclusions that were consistent with their premises (”data”), it turns out that one of their premises - the major one - was apparently incorrect.
James et al, “Cellular and mitochondrial glutathione redox imbalance in lymphoblastoid cells derived from children with autism.” (FASEB J., 2009 Aug;23(8):2374-83)
This study is a much “cleaner” example, in that there is no reason to suspect that its data is erroneous and it appears to be a well-designed study.
In this study, the authors examined cultured lymphoblastoid (precursors to lyphocytes) cells from ten autistic children (mean age 7.8 years) and ten control adults (mean age 27.7 years). They found that the cells from the autistic children had lower whole cell and mitochondrial reduced glutathione (GSH) and higher oxidised glutathione (GSSG) than the cells from the control adults.
They also found that the cells from autistic children were more sensitive to oxidative stress from thimerosal and nitric oxide (NO).
In their conclusions, the authors state:
“A potential role of subclinical mitochondrial dysfunction and altered redox homeostasis in a subset of children with autism has been previously proposed. Within the limitations of an in vitro cell model, the baseline differences in intracellular and mitochondrial glutathione redox status in autism and control LCLs cultured under identical conditions would support this possibility.” [emphasis added]
“These potentially vulnerable subpopulations need to be identified and evaluated independently because large population epidemiologic studies do not have the sensitivity to detect minor high-risk subpopulations.” [emphasis added]
As I’ve stated above, I have no issue with the methods of this study. But the parts of their conclusions that I’ve highlighted are glaringly invalid (i.e. they do not logically flow from their premises/data).
Some of you have probably already figured it out, but let me give the rest of you a hint: they studied cells from only ten autistic children.
If this were a “subpopulation” problem, a problem seen in only a “subset” of autistic children who were part of a “minor high-risk subpopulation”, why did they see it so clearly in cells from only ten children?
The only criteria used for selecting the autistic subjects was that they had to have ”at least one affected male sibling” - so they were looking only at familial autism. Even so, it is remarkable that they were able to find this “minor high-risk subpopulation” with so few subjects. This would suggest that having two autistic children is a marker for redox imbalance.
If we conclude - as the authors apparently did - that having two autistic children is a marker of redox imbalance, then it should be a relatively simple matter to test this conclusion, since the “minor high-risk subpopulation” is easily identified. I eagerly await the results of the study that looks into this.
And even if we accept the implication that all strongly familial autism is accompanied by this redox imbalance, there is still nothing in this study that shows causation. It is equally valid to conclude - from these data - that strongly familial autism is accompanied by a redox imbalance. There is nothing in this study to show causation.
In fact, since thimerosal (the only exogenous oxidant tested) was removed from childhood vaccines at a time roughly equal to the mean age of the study subjects, at least half of them had no significant exposure to thimerosal.
So, we see that we have to be very careful about blindly accepting the conclusions of scientific papers - even from studies that are otherwise well-designed and well-executed. We are fortunate that scientific papers generally include enough detail for us to determine if the data support the conclusions and if the conclusions reached by the authors are the only valid conclusions.
And if this is true for scientific papers, just imagine what is happening on ‘blogs, websites and the like.
Bottom line: ask people to “show their work”. If they won’t……well, use caution.
Prometheus
Filed under: Autism Science, Critical Thinking, Help for the bewildered | 16 Comments »