Hi Everyone!
So as many of you know, I have a very lively ( and occasionally snarky) twitter account, which can be found under the name @DrPaulND. As in any human endeavor on twitter, there is quite a bit of trolling and unpleasant discourse that occurs. However, being on twitter does give me a great deal of insight into current trends in the discourses of those ideologically opposed to Homeopathy and Naturopathic medicine.
As I detailed in a previous blog post, the most recent and most heavily propagandized systematic review on Homeopathy was the NHMRC review. Using Fake News Tactics, a number of groups promoted this report, despite it’s terrible methodology and numerous problems. Most tellingly, no skeptical organization has, to my knowledge, explored or even acknowledged these problems.
Rachel Roberts of the Homeopathy research institute describes the faults in the study in the following video.
These irregularities are summarized as follows (Taken from my previous blog on Homeopathy research)
- NHMRC did the review twice
- The first review, and even it’s existence, was not disclosed to the public
- the NHMRC, upon questioning, responded that the first review was low quality despite being conducted by the individual responsible for developing NHMRC’s guidelines on how to conduct evidence reviews
- FOI requests confirm that a member of NHMRC, Fred Mandelsohn, confirmed the first review to be high quality stating ““I am impressed by the rigor, thoroughness and systematic approach given to this evaluation [….] Overall, a lot of excellent work has gone into this review and the results are presented in a systematic, unbiased and convincing manner.”
- NHMRC stated their results were based on over 1800 studies, when in fact they were based on only 176
- NHMRC has used a method which has never been used in any other review, declaring that only trials of over 150 participants would be accepted, excluding the vast majority of high quality homeopathic trials, which due to lack of funding tend to be smaller, and despite the fact the NHMRC routinely conducts studies of less than 150 participants
- The above rules resulted in exclusion of 171 of 176 studies, leaving only 5 to be used as the basis of the study
- The chair of the second review, Peter Brooks, signed a conflict of interest form declaring himself “free from any association with any organization whose interests are either aligned or opposed to Homeopathy” when he was a member of the anti homeopathy lobby group “friends of science in medicine”
- The NHMRC review included no homeopaths or experts in Homeopathy research, despite the NHMRC guidelines requiring such an inclusion
As a result of these irregularities, Rachel made a complaint to the ombudsman of Australia on this report, which being a government agency, is accountable to the public. The Australian Ombudsman, who reviewed the complaint, found it of sufficient merit as to investigate it directly. They have proceeded to investigate this complaint since then, and according to Roberts, this investigation will soon reach a conclusion, as she discusses in this video.
This investigation has recently been maligned by a number of coordinated online campaigns. The skeptical community appears to have been spreading the rumour that this investigation has not, in fact occurred due to the large length of time since the making of the complaint. This is a complete lie. The ombudsman of Australia has very clear privacy policies, listed here. As is the case in most countries with protection of privacy legislation, information on ongoing investigations is held confidential until reports are issued.
The length of the investigation is indeed lengthy. However indeed, some investigations, particularily complex ones, often take months, as is mentioned in this FAQ on the Ombudsman’s website. Lengthy investigations are indeed not unprecedented, particularly in complex matters, as an investigation of a lengthy report in a contentious field would undoubtedly would be.
Secondly the complaints process is generally only communicated to the people in question who have started the complaint. Rachel Roberts, again, being that person, is the only person privy to that information, and her testimony on that subject is, as we have seen above, quite clear. The people spreading these rumours are not privy to the complaint process, and are spreading heresay in an attempt to discredit the fact that the Ombudsman found the complaint made by HRI to be of sufficient merit to warrant the Australian government investing it’s own resources in a very lengthy investigation into wrongdoing by NHMRC in this matter.
Furthermore, to add some extra spice to this already interesting set of circumstances, The Australian Senate recently questioned the NHMRC. One senator, Sterling Griff, questioned the department of health on the NHMRC review of Homeopathy. The answers were provided in writing and are available online see questions 214, 215, 261, 264, 268, 269, 270, 273, 274, and 275 ( And thank you so much to the website yourhealthyourchoice.com for this blog post which went into detail on this subject, who’s work I am largely repeating). While these answers will be discussed in future blogs, I encourage you to read them on your own.
The blog post by yourhealthyourchoice.com was quite succinct in detailing the inconsistencies in the NHMRC’s response to the Senator’s question 268 on the arbitrary limit of 150 per study. Both Question and answer are reproduced below:
Question:
The Homeopathy Review created a rule that trials with less than 150 participants were not ‘reliable’, meaning their results did not “warrant further consideration of their findings” (Information Paper, Appendix C). Does the NHMRC apply this rule to other trials it funds and/or reviews? If so, where else has this rule been applied and what internationally accepted scientific standards were used to authenticate it?
Answer:
Studies with less than 150 participants were not excluded from consideration in the evidence assessment. The Homeopathy Working Committee (HWC) deemed that, for the purposes of drafting evidence statements, the outcomes of studies with less than 150 participants were not sufficiently powered to reliably inform conclusions on the effectiveness of homeopathy to treat a particular clinical condition. NHMRC establishes expert working committees, such as the HWC, under section 39 of the NHMRC Act, to advise NHMRC on the methodology and parameters of evidence evaluation when developing health advice and guidelines. This is in line with international best practice for formulating evidence-based health advice. The HWC has expertise in evidence-based medicine, clinical trials and complementary medicine. There is no ruling as to the number of participants that is appropriate for a trial that applies for funding by NHMRC. Applications are selected for funding through a competitive peer review process. The grant review panel assesses the application against published criteria. Issues such as trial sample size, experimental design and the power are usually assessed under “scientific quality”. If, in the judgement of the expert assessors, the scientific quality is compromised by inadequate trial size, this will be reflected in the score for that assessment criterion.”
This answer is remarkable in it’s deceptiveness. As shown by the Your Health blog, the question was never actually answered. NHMRC, yet again, did not specify why the cutoff of 150 was chosen, and failed to justify it. This is particularily ironic in that NHMRC itself regularily funds and conducts trials of less than 150, as YourHealthYourChoice demonstrated.
Secondly, The answer incorrectly states that the sample size of 150 was not used to exclude trials. However, the NHMRC information paper on page 34-35 and Optum overview report appendices on page 275 both do confirm this. In effect NHMRC specifically crafted it’s protocol to exclude unreliable studies, and then defined, without justification, anything less than 150 as unreliable, reducing massively the pool of studies that they could draw upon to make their conclusion.
Furthermore, their answers to the Senator Griff concealed this from the government, either deliberately, or through ignorance on part of the answering person.
This act of deception, more than anything else, likely explains the length of this investigation. It cannot be expected that a body that went to such length to pervert science to create a given outcome will be particularly with a government investigation of their practices and, dare I say it, motivations for doing so.
I encourage everyone to read the following blog post going into great detail
NHMRC under scrutiny in the Senate – Full and accurate answers not provided