Have four out of five systematic reviews concluded that homeopathy works? [duplicate]

Question

In this article in the Guardian, Rachel Roberts claims that:

"Five major systematic reviews have also been carried out to analyse the balance of evidence from RCTs [randomized controlled trials] of homeopathy – four were positive (Kleijnen, J, et al; Linde, K, et al; Linde, K, et al; Cucherat, M, et al) and one was negative (Shang, A et al)."

Is this claim true? And if so, is it representative?

I am aware of the risk of cherry-picking trials to support one's hypothesis, but I would expect that systematic reviews would remove this selection bias.

To be clear, I'm not interested in whether there is a scientific basis for homeopathy (since the author of the article does not argue that there is one, apart from anything else).

Answer 1

This statement is only true if you heavily distort the meaning of the word "positive". The following are quotes from the abstract of all four meta-analyses the article claimed are positive:

At the moment the evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias. This indicates that there is a legitimate case for further evaluation of homoeopathy, but only by means of well performed trials.

 

We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results.

 

There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results.

 

the results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition. Further research on homeopathy is warranted provided it is rigorous and systematic.

One of these didn't actually address the question whether homeopathy works or not, they examined the influence of study design and came to the conclusion that better studies tend to show more negative results on homeopathy. All other three essentially said that their results are not conclusive and that more research is necessary.

They all state that the quality of the studies they're based on is often low, and that this affects their conclusion. It's very misleading at least to claim that these meta-analyses support homeopathy.

Answer 2

The claim is loosely worded, but a reasonable interpretation is that there are only five systematic reviews (otherwise, the whole point of avoiding cherry-picking a study is negated). This meta-meta review found eleven independent reviews, and found "Collectively they implied that the overall positive result of this meta-analysis is not supported by a critical analysis of the data. " https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1874503/

And another article looked at 57, and found "Though that body was mixed in size and quality, no clear signal of effectiveness emerged from the higher quality studies." http://blogs.bmj.com/bmj/2016/02/16/paul-glasziou-still-no-evidence-for-homeopathy/

Also, the JREF prize can be considered a review, and no one has claimed it.

Finally, there is an idea that we should approach claims with an "open mind", and entertain all hypotheses, but there is a point at which a hypothesis is so absurd that it cannot be meaningfully evaluated. Even if there were studies that showed that homeopathy works, that would do nothing to prove homeopathy; homeopathy basically requires that there be some supernatural being keeping track of what water has been in contact with what substance, and altering people's biology accordingly. And once you entertain that hypothesis, you might as well entertain the hypothesis that the studies don't actually support homeopathy, but aliens are mind-controlling scientists into thinking they do.

Answer 3

Long comment - Wrote this before I had commenting privileges.

Homeopathy most certainly does not work, except as a placebo.

You linked to an article in The Guardian, written by Rachel Roberts. Click her bio on The Guardian, and what'd you know, she's a 'professional homeopath':

Rachel Roberts is a professional homeopath, who qualified in 1997, and a registered member of the Society of Homeopaths. Holding a degree in biological sciences, Rachel lectures in both homeopathy and medical sciences at various colleges in the UK and overseas. She currently holds the posts of research consultant for the society of homeopaths and executive co-ordinator of the Homeopathy Research Institute

Her article is from 2010, and that's the only thing she wrote for The Guardian. Purely a fluff piece used to prop up her quack pseudoscience.

Nearly eight years later, nothing has changed. Homeopathy still doesn't work.

That's on top of the criticisms of the actual studies put forth by user Mad Scientist.

Rest assured, Homeopathy has not been vindicated in the slightest by this.

Edit: I see some criticism that I didn't address the truth or falsehood of the studies. That's because someone else already did that, and I referred to that person - Mad Scientist - for the answers to that. I addressed the potential conflict of interest that further casts shadow on her credibility in reporting on these studies.

Answer 4

The article is formally correct in its statements (four of these five reviews indeed say that), but it is either deliberately misleading or abysmally researched, and written by someone lacking professional knowledge (or, both).

It took me 20 seconds on Medline to find another four systematic reviews (although in the Guardian author's defense, three of them published after the Guardian article), and a reference to a publication of around the same time as the Guardian article which did a more profound research on the Cochrane database of SRs, coming to a much different conclusion:

The findings of currently available Cochrane reviews of studies of homeopathy do not show that homeopathic medicines have effects beyond placebo.

The Guardian article's author is very obviously in favor of homeopathics and writes in an heavily argumentative, anecdotical-emotional, and if I'm allowed to say almost polemic way (some emphasis added by me):

(An anecdote fo how I ruined a dinner party)
[...]
I know homeopathy works
[...]
The facts, it seems, are being ignored. By the end of 2009, 142 randomised control trials (the gold standard in medical research) comparing homeopathy with placebo or conventional treatment had been published in peer-reviewed journals.
[...]
The "sceptics" campaign had a breakthrough

The facts. The facts are that the quoted reviews state that the overwhelming majority of trials, if they had any interpretable results (apparently there were others too, no information), was of sheer embarrassing quality. Gold standard is a nice buzzword, but the facts are, almost all of the studies named in e.g. Kleijnen's review do not meet the gold standard (not even remotely), as most of them were (contrary to the above claim) not even placebo controlled (or properly randomized, or in some cases both). In addition to that, the larger number of studies had population sizes under 25 which make a meaningful interpretation of data... let's say, a challenge.
Oh heck, I guess I'm with the "sceptics" campaign.

I am personally quite impressed how the reviewers in those 5 systematic reviews still decided to be so forgiving as to allege a possible effect, or some effect with a weak level of evidence when they more or less uniformly stated that significant results were only to be found in the poorest of all studies. Cucherat even calculated (in my opinion doubtful) combined significance levels, followed by explicitly stating that P rapidly goes towards 0.08 if you eliminate the trials with the lowest quality. Then again, however, followed by the puzzling statement "there is some evidence (... of low strength)" when he basically just demonstrated the opposite.

Be that as it may, the claim made by the Guardian article is, from a certain point of view, perfectly correct. Four of these exact five reviews indeed attribute some effect.
Only just, the article, uh... omits some (probably unimportant!) information that doesn't fit so well into the story.