Homeopathy does have a Positive Evidence base

Hi Everyone. This post has been brewing for a considerable period of time. I have decided to release it finally, rather than have the same conversation with people, over and over again, about the evidence base of Homeopathy.

In short: Homeopathy does have an evidence base. This evidence base is, when taken as a whole and evaluated in light of the way Homeopathy is practiced, positive. Homeopathic methodology does indeed contain many aspects which makes evaluating Homeopathy in a clinical trial difficult, but these problems are slowly being overcome through improved clinical methodologies. Other lines of evidence supporting Homeopathy include laboratory studies and historical records of Homeopathy’s use, particularly in infectious diseases.

Now to begin, clinical trials are the standard method by which modern Allopathic medicine evaluates the outcome of most interventions, and have been taken up by other branches of medicine as well. This method is, on the whole, designed to compare a treatment, and a non treatment (often termed a placebo) a treatment deliberately calculated to be ineffective. The results are numerically calculated in some way and compared to one another using a number of statistical tools. Results are either statistically significant (i.e. found by the methods being used to be greater than what would occur by chance) or statistically insignificant (found by methods to be within the bounds of what would occur by chance).

Homeopathy has been evaluated by means of clinical trials for hundreds of years. However the nature of Homeopathic prescribing, the deeply detailed nature of casetaking, the therapeutic nature of casetaking itself, and the multiple adjustments and refinements of potency needed during therapy, which makes placebo control difficult all combine together to make clinical trials challenging to undertake for genuine, pure Homeopathy. Methodological refinements have indeed been made to the clinical trial procedure over the past several difficulties, but they have not been completely successful, often creating outcomes in which the controls of a clinical trial result in data which are less than what one would expect from clinical practice.

Within the basic framework of a clinical trial, many variables can influence results, biasing the results in a particular direction. The kinds of influence have been assessed by researchers over the years, and tabulated. Several such measures exist, but the one in this link is particularly widely used.

Homeopathy has been examined by multiple clinical trials to determine efficacy throughout the years. Many of these trials have been negative. Many have been positive, until recently, the compilation of the data from these trials had not been done in such a way as to examine the validity of Homeopathy as a whole. Recently however this strategy has been adapted. Termed a Meta-analysis, this method has been used in a number of fields with sufficient research, allowing for larger conclusions to be drawn. I will go through them, followed by a summary table at the end of this discussion.

The first meta-analysis was preformed in 1991 by Kleijnen and published in BMJ. It found, in the small amount of clinical trials available at the time, a positive result.

The next meta-analysis was published in the Lancet in 1997 by Linde. Of 185 trials found, 115 met inclusion criterea, with 89 having adequate data for inclusion. Linde did not differentiate methodology, but pooled studies of all varieties of Homeopathy regardless and found an Odds ratio of 2.45 (with a 95% confidence interval of 2.05-2.93). This is a strongly positive result.When corrective factors for publication bias were added, the odds ration decreased to 1.78 (1.03-3.1), weaker but still positive. Analyzing only high quality trials resulted in results of 1.66 (1.33-2.08). Linde was repeatedly criticized for lumping data of different quality together, and for lumping together trials of individualized and nonindividualized homeopathy, so further studies were conducted.

Ernst also conducted several reviews in this period this first (Ernst E: Are highly dilute homeopathic remedies placebos? Perfusion 1998;11:291.) purported to reanalyze Linde’s 1997 analysis in light of criticisms that it evaluated many conditions, included nonindividualized trials, and include lower potency preparations that could potentially contain molecules of the original product.His analysis of the, in his view, higher quality studies, shows an odds ratio of 0. Hahn has criticized this study as including only 5 studies, and thus far less valid than Linde’s original 1997 analysis, despite it’s flaws.

In 1998 Linde conducted a study of clinical trials of individualized Homeopathy. he found a number of methodological and quality issues, but pooling the data regardless, found a positive result with an odds ratio of 1.62 (95% CI 1.17-2.23). However, when the analysis was restricted to only higher quality trials, no significant effect was seen, though a positive trend was observed Odds ratio of 1.12 (0.77-1.44). As discussed above, the many issues in clinical trials of Homeopathy at the time, make this conclusion not terribly surprising, though this problem is remedied in more current research. Linde explored this in a subsequent article in 1999 exploring the effect of trial quality on effect size in Homeopathy studies. He found that higher trial quality often produced smaller effect sizes. This is important, as many critics seized upon this as proof of lack of efficacy of Homeopathy, forgetting that other research has shown this effect to be consistent throughout medical research ( ie not a specific problem to Homeopathy).

In 2000 Ernst again published another Meta-analysis, in which he criticized Linde’s 1998 analysis as flawed due to the weaknesses of the studies used. His response was to extrapolate trends based on Linde’s 1999 study of trial quality, and suggest a linear trend (when in fact the data is asymptotic), that trials with a perfect quality would theoretically move towards results of 0. This is strongly criticized by Hahn as prioritizing imagined and extrapolated data over real data, particularly bizarre when he had access to the real data.

The next meta-analysis was preformed by Cucherat et al in 2000 and published in the European Journal of Pharmacology. It took 118 trials, judged 16 to meet inclusion criteria, and then combined p values together to form a grand p value, an extremely unusual method of analysis, one which Hahn notes to be likely to give the least favorable statistical result to Homeopathy). Despite this odd method of analysis, the results of the analysis were positive with a grand P value of <0.000036 (i.e. the results had a less than 3.6 out of 100 000 chance of being due to chance). Studies with less than 10% dropouts had a slightly lower p value of 0.084 (8.4 out of a 1000 chance of being due to placebo) and studies with less than 5% dropout had a p value of 0.082, on marginally above the generally accepted threshold for statistical significance. However this standard was extraordinarily rigorous, with dropout rates of less than 20% being commonly accepted as adequate.

The next Meta-analysis of Homeopathy as a whole was Shang, published by the Lancet in 2005. This study took an extraordinarily bizarre approach of choosing 110 randomized controlled double blind trials, and then “matching” them to 110 comparable allopathic trials, with some high quality Homeopathy studies being excluded solely due to a comparable allopathic trial not being able to be found. The data in these 110 trials were not combined to creat an odds ratio. Instead the data from all but 21 trials of Homeopathy were excluded due to unspecified quality measures. The authors then excluded, for unclear reasons, all but 8 trials and came up with an insignificant odds ratio of 0.88 (0.65-1.19). Fascinatingly, the identity of these 8 studies were not listed in the original publication, but were released several months later, after the media circus around this result had died down. Skeptical organizations created a vast amount of propaganda using what we now call Fake News something we will also see occurring with the more recent NHMRC report. In both cases, the subsequently revealed errors were not reported in either the popular or skeptical presses.

These results have been heavily criticized by Homeopaths and statisticians, with the overall opinion that this trial should not have been published. These arguments are summarized by the Homeopathy Research Institute, but the most significant perhaps belong to Ludtke and Hahn. Ludtke analyses Shang to find that the results gained out of his analysis are highly dependent on the subsection of trials selected, with his results being almost entirely due to a single large nonindividualized trial of Homeopathy. Hahn comments that the funnel plot tool used by Shang is completely inadequate to evaluate treatment effects in different medical conditions and that in order to reach his conclusions of Homeopathy lacking effect beyond placebo, 90% of clinical trials must be excluded. In fact, it has been determined by Ludtke, that if Shang had included all 21 higher quality trials, even with his bizarre selection criteria, he would have found a statistically significant effect greater than placebo.

The next major metanalyses of Homeopathy was in 2014, by Mathie, published in Systematic review. It included all available RCTs, and rated all of them with several quality assessment measures. Unfortunately, again likely due to methodological issues, no trial rated as no risk of bias in all domains. However the trials that had low or unclear risk of bias in one domain were analyzed. 22 trials were had extractable data, and were subject to analysis creating a significant odds ratio of 1.53 (1.22-1.91). Restricting the analysis to trials rated as reliable evidence created an even higher odds ratio of 1.93 (1.16-3.38). This is explained wonderfully by Mathie himself in the following short presentation:

This analysis stands at the present day (April 2018 as of writing) as the most current, complete and unbiased of all reviews of Homeopathy. The low quality of evidence does merit caution, but the results do clearly indicate that the existing data do suggest that Homeopathy has an effect greater than placebo. This is the current state of the evidence from clinical trials, and to deny this is to go against the existing data.

Ernst has criticized the results as excluding one of his own trials. Mathie graciously explained on Ernst’s blog the exclusion criteria, but added Ernst’s data to his despite not meeting the criteria, finding the same results. This is published in a pdf separate from Mathie’s original article.

The next, and probably lowest quality of analysis is the NHMRC review done in 2015, commissioned by the Australian government to evaluate the evidence of Homeopathy. This review has a number of methodological problems, which have been summarized expertly by the Homeopathy Research Institute. I will summarize them as follows (and much gratitude to HRO for presenting these points so succinctly):

• NHMRC did the review twice
• The first review, and even it’s existence, was not disclosed to the public
• the NHMRC, upon questioning, responded that the first review was low quality despite being conducted by the individual responsible for developing NHMRC’s guidelines on how to conduct evidence reviews
• FOI requests confirm that a member of NHMRC, Fred Mandelsohn, confirmed the first review to be high quality stating ““I am impressed by the rigor, thoroughness and systematic approach given to this evaluation [….] Overall, a lot of excellent work has gone into this review and the results are presented in a systematic, unbiased and convincing manner.” 
• NHMRC stated their results were based on over 1800 studies, when in fact they were based on only 176
• NHMRC has used a method which has never been used in any other review, declaring that only trials of over 150 participants would be accepted, excluding the vast majority of high quality homeopathic trials, which due to lack of funding tend to be smaller, and despite the fact the NHMRC routinely conducts studies of less than 150 participants
• The above rules resulted in exclusion of 171 of 176 studies, leaving only 5 to be used as the basis of the study
• The chair of the second review, Peter Brooks, signed a conflict of interest form declaring himself “free from any association with any organization whose interests are either aligned or opposed to Homeopathy” when he was a member of the anti homeopathy lobby group “friends of science in medicine”
• The NHMRC review included no homeopaths or experts in Homeopathy research, despite the NHMRC guidelines requiring such an inclusion

Rachel Roberts of the HRI presents this set of circumstance expertly in the video below.

The evidence against the NHMRC in frank academic bias and misconduct is so strong the HRI is pursuing a complaint with the relevant Ombudsman in Australia. The initial investigation has found sufficient evidence to warrant a full investigation of NHMRC’s conduct, which is ongoing.

The Skeptical media had a field day with this study, producing a deluge of articles and media misrepresenting the conclusions of the NHMRC review and using Fake News strategies to expand it’s reach to multiple audiences. The subsequent faults and criticisms of this report, and it’s current status of being investigated by the government has gone unreported.

The most recent metaanalysis of Homeopathy was done by Mathie in 2017, and focused on all nonindividualized trials of Homeopathy. This study focused on Randomized, double blinded control trials of nonindividualized Homeopathy, and found 75. 48 trials had high risk of bias, 23 uncertain and 3 had low risk of bias and were listed as reliable. The standard mean deviation was the measure used. 54 trials had extractable data and pooled data shows a SMD of -.33 (CI -.44,-.21) which was statistically significant. Adjusting for publication bias this was adjusted to a still significant -0.16(CI-.46, -.09). Reliable data resulted in an insignificant result of -.18 (CI -.46,+0.09). This result has gone unremarked in the greater skeptical literature on Homeopathy, despite both major negative reviews of Homeopathy (Shang and NHMRC) failing to differentiate these two types of studies.

Skeptical organizations and their sympathizers in the media have utilized the flawed negative studies, presented them without context, and used them as a justification for their already held conclusions that Homeopathy has no effect greater than placebo. This is not just a problem with skeptical organizations, as much of science journalism suffers from similar issues. Skeptical organizations however have embraced Fake News tactics in order to spread their message, having just as much culpability for the modern state of disinformation as other sponsored state actors using those same tactics to advance their political goals.

Conclusion:

This blog post reviews all the relevant meta-analyses of Homeopathy as a whole preformed. All meta-analyses preformed in at least a somewhat open and rigorous manner have found statistically significant effects suggesting Homeopathy has a greater than placebo effect, or at least a strong trend in that direction. The meta-analyses with questionable methodology, one of which is undergoing government investigation for academic irregularities, found negative results, which have been demonstrated to be a direct result of their exclusion of vast swathes of the Homeopathic clinical trial literature based on arbitrary and unexplained criteria.

The clinical data are flawed. Issues with methodology of randomized controlled clinical trial use in Homeopathy, combined with a lack of research funding have produced a lack of high quality trials and data. However, the data we have point towards Homeopathy having an effect greater than that of placebo.

There can be no argument with this conclusion, aside from new data emerging. Anyone who disputes this is going against the existing set of highest quality evidence on Homeopathy.