A “Made for Court” Study?
October 24th, 2009
This month, the journal Neurotoxicology published a study about vaccines, mercury and neurolgical delay:
Hewitson L, Houser LA, Stott C, Sackett G, Tomko JL, Atwood D, Blue L, Railey White E, Wakefield AJ. “Delayed acquisition of neonatal reflexes in newborn primates receiving a thimerosal-containing Hepatitis B vaccine: Influence of gestational age and birth weight.” Neurotoxicology. 2009 Oct 2. [Epub ahead of print]
The full text of this article has been thoughtfully provided by the folks at “Thoughtful House” in Austin, Texas.
Normally, when I read a scientific journal article, I like to first know a bit about the qualifications and previous work of the authors. It helps provide a context to evaluate their methods and conclusions.
The Players:
Laura Hewitson:
Adjunct Associate Professor of Obstetrics, Gynecology & Reproductive Sciences and Environmental and Occupational Health at the University of Pittsburgh School of Medicine Pittsburgh Development Center Magee-Womens Research Institute.
She is also on the staff of “Thoughtful House” in Austin, Texas.
She has a PhD in Biology and has done a fellowship in reproductive biology
She is also mother to Joshua Hollenbeck, who is a petitioner in a lawsuit filed with the National Vaccine Injury Compensation Program (NVICP) [petitioner #437 of 828 – her name is misspelled]. With the hypothesis of autism being caused by the MMR vaccine alone or in combination with thimerosal in shambles (from the petitioners’ point of view), she has motivation to search out another hypothesis of causation to bring before the National Vaccine Injury Compensation Program (NVICP).
Carol Stott, PhD:
A psychologist who worked with Dr. Wakefield in the UK and later followed him to “Thoughtful House” in Texas. Famous for her harassing and threatening e-mails to Brian Deer (see here, here and here).
David A. Atwood:
Associate Professor of Chemistry and the University of Kentucky. He holds a patent (US patent # 6,586,600) on benzenediethanethiol (BDT) environmental chelators, one of which [N,N’-bis (2-mercaptoethyl)isophthalamide] is being marketed by Boyd Haley (also from the University of Kentucky) as “Oxidative Stress Relief” (OSR) and is touted to be a “cure” for autism, specifically treating the putative mercury toxicity widely assumed (but never shown) to cause autism. Dr. Haley is marketing OSR as a “dietary supplement”, even though it is not a substance found in food (or nature) and has not been adequately tested for safety as a drug. The FDA has objected to this, but the DSHEA prevents them from being able to prevent Dr. Haley from marketing this compound as a “supplement”, so long as he calls it a “supplement”. Dr. Atwood’s role in the marketing of “OSR” is unclear.
Lisa Blue:
One of Dr. Atwood’s graduate students.
E. Railey White:
As of 2007, an undergraduate at the University of Kentucky who worked with Dr. Atwood on the benzenediethanethiol (BDT) environmental chelator project – the product which later generated “OSR”
Andrew Wakefield:
A British gastroenterologist who has published a famous (infamous?) article linking the measles vaccine strain (found in the MMR vaccine) to autism. Unfortunately, that hypothesis has not been supported by independent data, with his “landmark” paper being marred by the withdrawal of most (ten of thirteen) of the authors on that paper.
Facing a General Medical Council investigation in the UK, Dr. Wakefield moved to the US, eventually settling at “Thoughtful House” in Austin, Texas.
Given the many of the authors’ past history, it seems odd that this paper doesn’t mention the word “autism” even once. That’s right - the word appears for the first time in the acknowlegments (in “Autism Research Institute”, one of their funding sources). Even with that apparent oversight, it seems apparent that the focus of this research is probably on linking hepatitis B vaccine – with or without thimerosal – to autism. So, now that we have a picture of most of the authors, let’s move on to the introduction of the paper.
Introduction – setting the stage:
The first question the authors felt compelled to answer in their introduction was why they were studying thimerosal-containing hepatitis B vaccine, given that this formulation hasn’t been available in the US since 2001. Their explanation was that thimerosal-containing hepatitis B vaccines are still available in “developing countries”, where they are needed because refrigeration isn’t as widely available as in the US. This is an important question for the authors to answer, because their research is moot, otherwise.
Methods:
This is where the study starts to show some “quirks”.
Although the authors don’t mention it, this study appears to be an extension of a study that has been underway for some time. It is amazingly similar to a study mentioned in a 2008 abstract. This is not - by itself - an issue; what is an issue is that while the number of treated animals remained the same (13) the number of control (untreated) animals more than doubled (from 3 to 7) since that abstract and the number of reflexes tested went from 18 to 13 (see also here and here).
Perhaps this could explain why the “randomization” and the control treatment in this study was so….eccentric. The authors make an off-hand mention of the unusual way they randomized their subjects, calling it “semi-random”. The rhesus macaque infants were grouped into “peer groups” that were less than four weeks apart in age – apparently sequentially. The infants were given either a thimerosal-containing hepatitis B vaccine (they had to add the thimerosal themselves) or – and here’s one of the ”quirky” parts – either no injection or an injection of saline (control). In this paper, three of the control animals received no injection at all and four received a saline injection - further evidence that the additional control animals were added only after the 2008 abstract.
Another ”quirky” part of the methods is their mention of each newborn being assessed by a “Simian APGAR” [sic] (NOTE: the “Apgar” score was developed by Virginia Apgar, MD in 1952 – the term is a proper name, not an acronym). This is odd because they never mention these Apgar scores again. Although they mention that the birth weights and gestational ages were similar between the treatment and control groups, they never, ever, mention the Apgar scores.
Curious.
The infant macaques were evaluated for the time it took them to develop ceretain reflexes: root, snout, suck, auditory startle, grasp (hands and feet, since macaques can grasp with their feet), clasp (hands and feet), auditory orienting, visual orienting (near and far) and visual following (near and far) – a total of thirteen reflexes.
The final ”quirky” thing about the methods is their choice of Laura Hewitson (”LH”) to perform the neonatal assessments [NOTE: as a reader pointed out, the neonatal assessments were done by "L.A.H." rather than "L.H." - my bad for not noticing that "Lisa A. Houser" was the "L.A.H." in question]. By their own admission, “L.A.H.” has no prior training or education that would make her a good choice for this job. In fact, she required training to be able to carry out the assessments. Perhaps it would have been better to hire one of her trainers to carry out the evaluations – this would have made the comparisons made at the beginning of the study comparable to those made later. As it was, we have no way of knowing where “L.A.H.” was on her “learning curve” when each group was assessed. This is critically important because it is pretty clear that a second control group was added much later.
The fact that the authors devoted almost a full paragraph to describing “L.A.H.’s” training is indicative of how vulnerable they felt on this issue.
Two important points to note before we leave the methods section - one is that the assessments were performed “..always daily…” (an odd choice of words). Remember that – “…always daily…”. The second is that there were 13 reflexes studied in this study, where there were 18 reflexes studied in the 2008 abstract. These points will be important later.
Results – the meat of the paper:
Again, the authors emphasize that the birth weights and gestational ages of the treatment and control groups were similar. Once again, they fail to mention anything about the Simian Apgar scores.
They found a statistically significant difference between the treatment and control groups on only three (3) reflexes (root, p=0.004; suck, p=0.002 and snout, p=0.03) and claim that startle (p=0.11) and grasp hand (p=0.07) “approached significance”.
Unfortunately, neither the methods nor the results mention any correction for multiple comparisons. After all, with a cutoff of p<=0.05 and 13 comparisons, there is a 48% probability that at least one of them will be “positive” (have a p-value less than 0.05) just by chance. If, in fact, they had originally looked at 18 reflexes, that probability rises to 60%.
The authors then did Cox proportional hazards regression analysis of the results (see here for a brief explanation) . This technique makes the assumptions that there is a proportionality between the “hazard function” (the mathematical function that describes the “hazard rate”) and the log-linear function of the independent variables (covariates). It also assumes that the relationship between the covariates and the hazard function does not depend on time.
The Cox analysis showed a relationship between exposure (to both hepatitis B vaccine and thimerosal, since they were administered together) and delayed root and suck reflexes, but not for snout. When they added gestational age as a covariate, they found a significant main effect of gestational age on visual follow far, which did not have a statistically significant independent difference between the two groups.
In the end, they found contributions of gestational age and “exposure” to the differences they saw in the time to acquire the reflexes.
This might be a good time to discuss the reflexes themselves. Human infants generally have the same reflexes (although they aren’t as dexterous with their feet), most of them (esp root, snout, suck, startle and grasp) present at birth. In humans, they are collectively referred to as “primitive reflexes”. It seems odd that human infants, who are generally “behind” the newborns of other animal species in their acquisition of motor skills, should have these reflexes at birth when the macques do not. I would appreciate any input from people who have experience working with primates on what this might imply.
Table 2 gives the mean time-to-criterion (the time it took the infant macaques to meet the “criterion” level of skill in that reflex) for all thirteen reflexes. One nit-picking detail I must point out is that even though the evaluations were done once a day, they show the results as means, using fractional days. This is deceptive (and very commonly done, yes I know) because it gives the impression of much higher precision than was attained.
For example, if one of the little macaques attained “criterion” in their suck reflex an hour (or even a minute) after the evaluation, they would have been “scored” as having taken a full day longer to reach “criterion”. This gives a particular “graularity” to the data and adds to the potential error. In the example given above, the difference between the actual time to reach “criterion” and the measured time would be almost 24 hours - more if the interval between evaluations was longer.
Another odd bit in this study is their graph of the results (figure 1), which shows that the evaluations were done at irregular intervals and not as described in the “methods” section (”…always daily…”). There were, in fact, several intervals that exceeded 24 hours and also many that were less than 24 hours. It may just be that their description in the “methods” section was poorly worded, but it gives the impression of sloppy work.
Discussion – explanations, rationalizations and excuses:
I’ll let you read the authors’ discussion for yourself, but I will include a few “highlights”:
“Our study design does not enable us to determine whether it is the vaccine per se, the exposure to Th [sic – thimerosal], or a combination of both, that is causing the observed effects [note: absolutely true – this is one of the main flaws of their design] None-the-less [sic], the developing brain is considered the most vulnerable organ to mercury exposure…”
After admitting that they couldn’t distinguish between the vaccine and the thimerosal – because of their own poor experimental design – they turn right around and say, in essence, “But we know it was the mercury…”
“Although the basis for the effect of birth weight is not known, it is plausible that lower birth weight infants are more susceptible to what may be a dose-dependent toxicity of Th [sic – thimerosal] or some other HB [sic – hepatitis B] vaccine component, such as aluminum.”
With absolutely no data to support this conjecture, the authors throw out aluminium as a potential vaccine-related toxin. It is important to note that there was no citation given to support this idea. It is telling that they do not consider what aspects of their study design might have led to these inexplicable results.
Conflict of interest statement:
In most papers, this is pretty dry and uninteresting and as I’m not a person to “dismiss out of hand“ a study because of potential conflicts of interest, I usually don’t give them a great deal of weight. [section removed]
[NOTE: as mentioned above, the neonatal assessments were performed by "L.A.H" - "Lisa A. Houser", so the conflict of interests statement is not incorrect, however much it may "whitewash" the previous actions by Drs. Wakefield and Scott.]
My conclusion:
This study raises more red flags than a Kremlin May Day parade in the old Soviet Union. It appears that control subjects were added post hoc and that unproductive reflexes were dropped from the data. One of the authors – the “anchor” author – has been implicated in research and ethical misconduct and is awaiting adjudication. Other authors have significant personal conflicts of interest.
But these are methodological problems. Apart from that, the study also has some serious flaws in its conclusions - even if we assume their methods produced good data.
First, there appears to be no attempt to correct for multiple comparisons. In fact - as mentioned above - there appears to have been an attempt to “prune” the data set. Since I doubt that they would have eliminated data that supported their point, I can only surmise that these data were “unproductive”.
Second – and most fatal for this study – is that there was not an increase in human infants who were late to suck, root, etc. following the introduction of neonatal hepatitis B vaccination in 1991. Given that the “uptake” of this vaccine at birth was close to (and often exceeded) 90%, an effect of this magnitude would have been noted. Nor is there a difference between the US – which has continued hepatitis B vaccination at birth – and the UK – which almost never gives the hepatitis B vaccine at birth – in developmental delay or autism prevalence.
Now, the authors – or their supporters – might argue that this delay would only be seen in a small number of human infants and so wouldn’t be seen against the background of the larger population. However, if this study could find a difference with only twenty subjects, it should be rampant in the general human population, if it is a real effect. Since the results of this study do not correlate to what we see in the human population - which has received the same exposure - it seems unlikely that it is relevant to human infants.
In short, this study was irrelevant from the moment it was published. At least, from a scientific perspective.
However, from the perspective of people facing the demolition of their argument that the MMR vaccine - with or without thimerosal - caused autism in their children, finding evidence - even spurious evidence - that a different vaccine might be to blame could be highly relevant. Could this be the explanation behind the study design and the curious collection of authors?
I guess we’ll know soon enough.
Prometheus
Filed under: Autism Science
October 25th, 2009 at 10:03 am
Very thorough analysis.
The IMFAR abstract looked like litigation-generated research to me. This paper, however, appears to be pretty easy for the defense to shoot down, especially if the 4 monkeys were in fact added post-hoc. They can even bring up the inconsistency in the conflicts of interest statement.
In fact, if this is an attempt to produce evidence for court, it’s a rather clumsy attempt with zero chance of succeeding, isn’t it?
October 25th, 2009 at 1:14 pm
Nah, they were just lying.
October 25th, 2009 at 3:03 pm
One wonders how this got past the peer reviewers. I have no idea of the standing of Neurotoxicology as a publication, but non-standard capitalization (”NeuroToxicology welcomes Original Research Papers, Short Communications, Reviews, Commentaries, Letters to the Editor, Teaching Monographs, and Position Papers. Periodically the journal also publishes Announcements, Meeting Reports, Abstracts and peer-reviewed Proceedings of Meetings.”) always raises a flag (the price of being raised by near-pedants).
This paper’s publication does raise an interesting question for me. At what point does the history of someone such as Wakefield become an obstacle to their being published in a respected journal? Have I answered my own question?
October 25th, 2009 at 3:57 pm
Ah! The infamous monkey study finally breaks surface, after obviously having been extensively Waterboarded.
It’s so poor quality I can’t quite believe it’s been published. Any ideas how it got pass Peer Review?
You didn’t mention the circumstances of Stott leaving the University of Cambridge, nor the entirely co-incidental leaking of data from one of their unfinished studies.
*sigh* You boffins just will never master the Fine Art of Muckraking.
October 25th, 2009 at 7:21 pm
Prometheus,
Quick question: You wrote that Laura Hewitson (”LH”) performed the neonatal assessments. However, in the manuscrit (in section 2.6 “Neurodevelopmental Testing”) it says that “L.A.H.” performed these assessments. So, are they referring to Lisa A. Houser and not Laura Hewitson?
Not that this would invalidate any of your other arguments concerning the article.
Thanks again for your blog.
October 25th, 2009 at 8:33 pm
MDP,
Thank’s for picking that up - I completely missed the fact that one of the other authors had the same first and last initials. “My bad”, as the kids say.
I’ve corrected the post and left my own errors out where everyone can see them - no “memory holes” for me.
Prometheus
October 25th, 2009 at 8:40 pm
Joseph (and others),
While this may be an easy study to shoot down scientifically, there is still an effort being made to generate a quantity of studies that “support” the “vaccines/thimerosal-cause-autism” hypothesis.
It may even be that the people who are doing these studies are sincere in their beliefs and are simply trying to “do the work”.
However, this study was so poorly designed that it could not differentiate whether it was the vaccine and thimerosal that caused any effects they observed. This was either stupidity or a deliberate effort to mimic the hepatitis B vaccine as it was given prior to 2001. I think that their stated concern about children in developing countries was simply “window dressing”.
Prometheus
October 25th, 2009 at 9:56 pm
I missed the non-use of Apgar scores in the results but I did note that they divided them into ‘peer groups’ for later social testing but also, never mentioned them again. The way they did this could certainly introduce observer bias.
It is indeed curious as to why they used 18 neonatal reflex measurements in the IMFAR abstracts, reported that they examined the monkeys for 30 days, and incidentally claimed to have found 4 reflexes that were significantly different in the exposed group, but did not have the same method and results for this study.
October 27th, 2009 at 7:13 pm
OK, I tried to “line out” my errors, but apparently that makes them “invisible”. Drat my html inadequacies!
In order to make the post look less “redacted”, I’ve deleted most of the erroneous material about “L.A.H.” and left my correction notes.
Prometheus
October 28th, 2009 at 3:39 pm
This is the Elizabeth Birt memorial garbage. Classic Wakefield. Think about it: unless thimerosal was leaving every other baby chronically disabled, how would you ever get a statistically significant result with this number of monkeys?
Same with his original crap: all the kids had a new syndrome, then all the kids had measles virus. If these people had just been a little less gluttanous in their scams they might have gotten away with it.
Plainly, there’s retrospective subgroup analysis, manipulated controls and every other device known to these crooks, quacks and idiots.
October 28th, 2009 at 7:15 pm
Nice analysis.
Apgar was both the anesthesiologist’s name (as you mentioned) as well as an acronym (Activity, Pulse, Grimace, Appearance, and Respiration).
Joe
October 29th, 2009 at 9:48 pm
Oh, Prometheus, you’re no fun, raining on the kiddies’ parade with all those mean statistics.
(Good for you!)
Did they really say the thimerosal is necessary for when there’s no refrigeration? I have never been under the impression that temperature had anything to do with it.
October 30th, 2009 at 12:11 pm
Isles,
Actually, the article didn’t address why there is still thimerosal in (some) vaccines used in “developing countries” - they leave it to their readers’ imagination. However, there are only two reasons to put thimerosal into vaccines:
[1] It is the only preservative approved for use in vaccines that will reliably prevent bacterial/fungal growth at room temperature.
[2] It is one of the few preservatives approved for use in vaccines that will reliably prevent bacterial/fungal growth in vaccine vials that are used for multiple doses.
Since “developing countries” tend to buy the multi-use vials (much cheaper) and also tend to have less widespread availability of refrigeration, thimerosal is a good choice for them.
There are probably other preservatives that could do the job as well as thimerosal, but there is little incentive for vaccine manufacturers to develop, test and get approval for them.
In addition, we already know a lot about organo-mercury compounds, having had over one hundred years experience with them. A new preservative could have potential problems that we wouldn’t know about for decades. Better the devil you know…
If the people shouting about thimerosal would put one tenth of the money they spend on rallies and full-page adverts in major newspapers into research to find a better vaccine preservative, maybe we could offer “developing countries” a choice. As it is, their efforts are largely convincing researchers to shun vaccine (and autism) research. Those that don’t (e.g. Dr. Paul Offit) are either masochists or saints.
Prometheus
November 11th, 2009 at 6:33 pm
I blogged a related abstract in July 2008: http://daisymayfattypants.blogspot.com/2008/07/autism-no-rats-will-be-harmed-but-keep.html
The Safe Minds link to it is now broken, but I quote from it, and it’s available here: http://www.safeminds.org/research/research3.html. It’s equally indefensible.
My guess is that reviewers (who oughta be ashamed) asked about the training question, hence the defensive and telling elaboration.
Love that startle p value “trend.” I wish I could take a p value of 0.11 with no corrections for multiple comparisons and call it near significance. No, really. I don’t wish that.
The n values for the groups in this study were nowhere near large enough for the “granularity” introduced by the way criterion intervals were measured. And the lack of Bonferroni correction or whatever correction would be needed undermines any putative significance. My guess is that it would be effaced completely with the appropriate stats.
But most disturbing is the introduction of that “extra” set of controls. Well, most disturbing is the utter lack of relevance of the study question and that they did this to a bunch of baby macaques for no damned good reason.
Neurotox has an impact factor of about 2.5. Even that low, it seems a little high for a journal that let this one get by. Here is a link to a listing of their associate editors and editorial board in case anyone wants to let them know: http://www.elsevier.com/wps/find/journaleditorialboard.cws_home/621355/editorialboard. In particular, I’d look into R.A. Yokel
Lexington, KY, USA. Why? Because of these related authors:
D. Atwood , Chemistry, University of Kentucky, Lexington, KY
L. Blue, Chemistry, University of Kentucky, Lexington, KY
E. R. White , Chemistry, University of Kentucky, Lexington, KY.
Yokel: http://pharmacy.mc.uky.edu/faculty/RobertYokel.php. “Dr. Yokel’s research focuses on neurotoxic metals, their toxicokinetics and chelation.”
November 11th, 2009 at 11:22 pm
heh. was the hepatitis A vaccine given at birth in 1998? If it was, can I blame it for my 11 year old’s advanced speech?
November 11th, 2009 at 11:27 pm
“This was either stupidity or a deliberate effort to mimic the hepatitis B vaccine as it was given prior to 2001.” I should read more.
November 11th, 2009 at 11:33 pm
heh, rereading. Okay so hep B, not A, was given after 1991, so therefore I can blame it on my daughter’s advanced language! I can also blame it on her wonderful ability to nurse, with absolute refusal of anything but the real thing. ha!
November 18th, 2009 at 7:37 pm
If Wakefield et al added control animals without changing the total, couldn’t this mean they simply “reclassified” vaccinated animals as unvaccinated, ie LIED?
I also strongly suspect that some kind of fraud was used in getting a journal’s staff to publish anything by Wakefield. I find some substantiation in the fact that acceptance took exactly 3 months, which appears slightly faster than usual. Anyone else have thoughts on this?
November 20th, 2009 at 11:08 am
I have been doing readings on the emergence of the Y chromosome - 600 million years ago.
Somewhere in this reading I came across a study that linked autism to excessive amounts of testosterone?
I’d appreciate your always complete comments on this vein/vane/vain of thought.
All should be as such,
d
February 12th, 2010 at 10:12 am
This article was withdrawn from press. I cover it today on my blog.
February 14th, 2010 at 8:31 pm
[...] undisclosed conflict of interest from lead author Laura Hewitson or simply the fact that it is an atrocious piece of work. Meanwhile his fan base are doing their best with a number of [...]
February 18th, 2010 at 8:06 am
[...] war schon länger wegen offensichtlicher Fehler im Studiendesign kritisiert worden (Orac, Prometheus, Just the Vax), ein Blogger rief sogar zum Boykott des Verlagshauses Elsevier auf. Anfang der Woche [...]
February 20th, 2010 at 9:36 pm
[...] and taken another look at Wakefield’s study and decided that the faults identified by Prometheus Orac and others were too serious to be [...]
February 22nd, 2010 at 6:56 am
[...] Research, Part II. The excellent skeptical blogger Prometheus also deconstructed the study in A “Made for Court” Study? Read those posts if you want the gory details. Suffice it to say, that once again Wakefield was [...]
March 10th, 2010 at 6:06 pm
[...] Wakefield’s “monkey study,” which I blogged about in 2008 and again in 2009. Prometheus and Science Mom have also eviscerated the study, which, as you may also recall, had somehow managed [...]
March 17th, 2010 at 7:05 am
[...] mysteriously resigned from his position at Thoughtful House, the autism center he founded. Further, a poorly designed study on the Hepatitis B vaccine he was involved in was denied publication while another study to be published in the Pediatric Infectious Disease [...]