So, after years of campaigning, the NHMRC of Australia has finally released the first draft of it’s infamous report on Homeopathy.
The second version was released to great media fanfare in 2015 generally proclaiming the end of Homeopathy. Over time, the HRI and others began exposing the serious and likely deliberate methodological issues in this report. Eventually they initiated Freedom of information requests, and discovered even more methodological irregularities. These are detailed in the following video.
In short the points wrong (taken with gratitude from the HRI) with this report were:
- NHMRC did the homeopathy review twice, producing two reports, one in July 2012 and the one released to the public in March 2015.
- The existence of the first report was not disclosed to the public – it was only discovered through Freedom of Information (FOI) requests.
- NHMRC say they rejected the first report because it was poor quality despite it being undertaken by a reputable scientist and author of NHMRC’s own guidelines on how to conduct evidence reviews.
- FOI requests have revealed that a member of NHMRC’s expert committee overseeing the review process – Professor Fred Mendelsohn – confirmed the first review to be high quality saying – “I am impressed by the rigor, thoroughness and systematic approach given to this evaluation [….] Overall, a lot of excellent work has gone into this review and the results are presented in a systematic, unbiased and convincing manner.”
- NHMRC said the results of the second report published in 2015 were based on a “rigorous assessment of over 1800 studies”. In fact results were based on only 176 studies.
- NHMRC used a method that has never been used in any other review, before or since. NHMRC decided that for trials to be ‘reliable’ they had to have at least 150 participants and reach an unusually high threshold for quality. This is despite the fact that NHMRC itself routinely conducts studies with less than 150 participants.
- These unprecedented and arbitrary rules meant the results of 171 of the trials were completely disregarded as being ‘unreliable’ leaving only 5 trials NHMRC considered to be ‘reliable’. As they assessed all 5 of these trials as negative, this explains how NHMRC could conclude that there was no ‘reliable’ evidence.
- Professor Peter Brooks, Chair of the NHMRC committee that conducted the 2015 review, signed conflict of interest form declaring he was not “affiliated or associated with any organisation whose interests are either aligned with or opposed to homeopathy”, despite being a member of anti-homeopathy lobby group ‘Friends of Science in Medicine’
- NHMRC’s guidelines state that such committees must include experts on the topic being reviewed, yet there was not one homeopathy expert on this committee.
One of the most noteworthy problems which emerged during HRI’s investigation was the fact that the report was completed twice. One draft was finished in 2012, and a completely separate team was assembled to then produce the second publicly released version of the report, in 2015.
The first draft was concealed from the public, to the extent it was only uncovered by the Freedom of information act requests of the HRI and an Australian Homeopathic Association. This report, which can be seen here, is all the more unique in that it’s conclusions were quite different than the 2015 version.
The 2012 Version found Promising C grade evidence of Homeopathy in 5 conditions, Fibromyalgia, Otitis Media, Post Operative Ileus, Upper Respiratory Tract Infections in Adults, and Side Effects of Cancer treatment.
Furthermore, commentary on the first page clairified the position of NHMRC on Homeopathy, that the 2015 report did not find no evidence for Homeopathy in general, merely no evidence for any specific condition, and that with the change in methodology discussed above.
The change between versions of reports is critical, and lends even more weight to the idea that the 2015 report engaged in outcome shopping, due to not wishing to find positive evidence for Homeopathy.
I will in the future explore this 2012 report, and compare it to the 2015 version much more extensively.
I also will be presenting a webinar on Homeopathy research in the near future! I hope you will be able to join me for it. More on that in the future!