78

This question already has an answer here:

In this article in the Guardian, Rachel Roberts claims that:

"Five major systematic reviews have also been carried out to analyse the balance of evidence from RCTs [randomized controlled trials] of homeopathy – four were positive (Kleijnen, J, et al; Linde, K, et al; Linde, K, et al; Cucherat, M, et al) and one was negative (Shang, A et al)."

Is this claim true? And if so, is it representative?

I am aware of the risk of cherry-picking trials to support one's hypothesis, but I would expect that systematic reviews would remove this selection bias.

To be clear, I'm not interested in whether there is a scientific basis for homeopathy (since the author of the article does not argue that there is one, apart from anything else).

marked as duplicate by Jamiec Dec 11 '17 at 10:14

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 41
    Reading the cited studies they basically say that he trials were crap and don't prove anything beyond that there was some improvement, which may be due to placebo or even just having a doctor listen to the patients. So they don't really support the claim that homeopathy is effective. – dont_shog_me_bro Nov 30 '17 at 11:26
  • 39
    This is a very common tactic: linking to a reputable journal article which happens to be about that topic, claiming that it supports one's views, even if it doesn't, and hoping most readers don't actually read those papers and are satisfied instead with a vague "scientists proved it!" feeling. One of the greatest examples was a Nature article proving that vitamin C slightly alleviated one of the many harmful side-effects of chemotherapy used in cancer treatment, and this article was picked up by the media as "What they don't want you to know: vitamin C instantly cures all forms of cancer!" – vsz Nov 30 '17 at 13:47
  • @Philipp definitely a duplicate. Moreover my answer to the other question has more recent results than the ones referred to in the question. – matt_black Nov 30 '17 at 16:51
  • 34
    Only four in five million reviews were favourable but after dilution it became four our of five. – TheMathemagician Dec 1 '17 at 13:03
  • 3
    Homeopathy is IMHO one of the best proof that to some extent people who see plots everywhere may have a point: It has absolutely no logic to supports itself on (unless you believe in "water memory") but still manages to be in medicines shop and have doctors specialized in it... – Caterpillaraoz Dec 1 '17 at 15:03
135

This statement is only true if you heavily distort the meaning of the word "positive". The following are quotes from the abstract of all four meta-analyses the article claimed are positive:

At the moment the evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias. This indicates that there is a legitimate case for further evaluation of homoeopathy, but only by means of well performed trials.

 

We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results.

 

There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results.

 

the results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition. Further research on homeopathy is warranted provided it is rigorous and systematic.

One of these didn't actually address the question whether homeopathy works or not, they examined the influence of study design and came to the conclusion that better studies tend to show more negative results on homeopathy. All other three essentially said that their results are not conclusive and that more research is necessary.

They all state that the quality of the studies they're based on is often low, and that this affects their conclusion. It's very misleading at least to claim that these meta-analyses support homeopathy.

  • 72
    "At the moment the evidence of clinical trials is positive" I read that as "there is evidence that trials have taken place" :D – Baldrickk Nov 30 '17 at 15:26
  • 26
    Indeed... "evidence of clinical trials" and "evidence from clinical trials" are two very different statements. – T.J.L. Nov 30 '17 at 15:44
  • 14
    @Baldrickk No, it's clearly intended to say "the clinical trials have evidence which is positive". But as we'd all expect, it goes on to say (very politely!) that all the trials showing a positive outcome were garbage, and properly-run trials tend towards only placebo effect. And as good scientists, the authors correctly say that they don't have enough sigma to firmly establish it as only placebo effect. – Graham Nov 30 '17 at 17:36
  • 15
    In medical jargon, as seen in medical journals, a "positive outcome" is one where the patient didn't get worse. That's it. I learned this to my dismay when reading articles before my wife underwent a certain eye surgery. – davidbak Nov 30 '17 at 19:33
  • 19
    It's very hard to get a mediocre study that shows that homeopathy doesn't work published as high-quality journals don't publish mediocre studies and low-quality journals like positive results. There's not a lot of demand for high-quality studies of homeopathy because it's, bluntly, silly. As a result, the majority of published studies will show that homeopathy works. – David Schwartz Nov 30 '17 at 23:25
26

The claim is loosely worded, but a reasonable interpretation is that there are only five systematic reviews (otherwise, the whole point of avoiding cherry-picking a study is negated). This meta-meta review found eleven independent reviews, and found "Collectively they implied that the overall positive result of this meta-analysis is not supported by a critical analysis of the data. " https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1874503/

And another article looked at 57, and found "Though that body was mixed in size and quality, no clear signal of effectiveness emerged from the higher quality studies." http://blogs.bmj.com/bmj/2016/02/16/paul-glasziou-still-no-evidence-for-homeopathy/

Also, the JREF prize can be considered a review, and no one has claimed it.

Finally, there is an idea that we should approach claims with an "open mind", and entertain all hypotheses, but there is a point at which a hypothesis is so absurd that it cannot be meaningfully evaluated. Even if there were studies that showed that homeopathy works, that would do nothing to prove homeopathy; homeopathy basically requires that there be some supernatural being keeping track of what water has been in contact with what substance, and altering people's biology accordingly. And once you entertain that hypothesis, you might as well entertain the hypothesis that the studies don't actually support homeopathy, but aliens are mind-controlling scientists into thinking they do.

  • 10
    @Accumulation "homeopathy basically requires that there be some supernatural being keeping track of what water has been in contact with what substance" - and to quote Tim Minchin, "it somehow forgets all the poo it's had in it" – Herr Pink Dec 1 '17 at 9:56
  • 6
    +1 I forgot who said it but the suggestion is that we should stop evaluating studies for the effectiveness of homeopathy altogether: doing so fundamentally misunderstands how empirical evidence works in science (which is, simplified, through a Bayesian process); since we know that homeopathy cannot work (it contradicts other, vastly better established scientific principles), no amount of limited studies will ever shift the overall evidence unless it also explains seemingly-contradictory principles of science. – Konrad Rudolph Dec 1 '17 at 15:39
  • @HerrPink - I had to join this stack just to upvote your comment & point you to my profile picture... which is that precise moment from the Tim Minchin video ;) – Tetsujin Dec 4 '17 at 17:56
18

Long comment - Wrote this before I had commenting privileges.

Homeopathy most certainly does not work, except as a placebo.

You linked to an article in The Guardian, written by Rachel Roberts. Click her bio on The Guardian, and what'd you know, she's a 'professional homeopath':

Rachel Roberts is a professional homeopath, who qualified in 1997, and a registered member of the Society of Homeopaths. Holding a degree in biological sciences, Rachel lectures in both homeopathy and medical sciences at various colleges in the UK and overseas. She currently holds the posts of research consultant for the society of homeopaths and executive co-ordinator of the Homeopathy Research Institute

Her article is from 2010, and that's the only thing she wrote for The Guardian. Purely a fluff piece used to prop up her quack pseudoscience.

Nearly eight years later, nothing has changed. Homeopathy still doesn't work.

That's on top of the criticisms of the actual studies put forth by user Mad Scientist.

Rest assured, Homeopathy has not been vindicated in the slightest by this.

Edit: I see some criticism that I didn't address the truth or falsehood of the studies. That's because someone else already did that, and I referred to that person - Mad Scientist - for the answers to that. I addressed the potential conflict of interest that further casts shadow on her credibility in reporting on these studies.

  • 27
    The question asked about the truth of a certain cited statistic; I don't think this answer really addresses that at all. – HDE 226868 Nov 30 '17 at 17:09
  • 13
    @Joshua That assertion requires you to already know the "correct" answer regarding homeopathy's effectiveness. Since this site is about evaluating evidence, dismissing evidence because you "know" it is wrong would be a bad idea. Similarly, this answer rejects the claim based on conflict of interest, but doesn't address the actual claim made; again, it is not based on actual evidence. That is not to say that the claims are true, just that these are poor methodologies for dismissing it. – IMSoP Nov 30 '17 at 18:20
  • 11
    @Joshua You're preaching to the choir. I am in no way defending homeopathy. What I am defending is the very purpose of this site, which is to address claims from a skeptical and evidence-based approach. Your comment, and this answer, do not do that. You may be right, but you are not providing any useful evidence that you are right. – IMSoP Nov 30 '17 at 20:41
  • 3
    @Joshua Put it another way, should we discount the claims of a doctor because they practice modern, western medicine? No! Even if they have the same arguable conflict of interest: they are a professional in the field in question. That's all that's being pointed out. – Nate Diamond Nov 30 '17 at 20:55
  • 14
    Unfortunately, your argument merely relies on emotive words, e.g. "most certainly", "fluff piece", "quack", "rest assured", and "not... in the slightest". This site is about getting beyond that, and basing arguments on scientific investigation, quality journalism, etc. – Craig McQueen Nov 30 '17 at 23:25
4

The article is formally correct in its statements (four of these five reviews indeed say that), but it is either deliberately misleading or abysmally researched, and written by someone lacking professional knowledge (or, both).

It took me 20 seconds on Medline to find another four systematic reviews (although in the Guardian author's defense, three of them published after the Guardian article), and a reference to a publication of around the same time as the Guardian article which did a more profound research on the Cochrane database of SRs, coming to a much different conclusion:

The findings of currently available Cochrane reviews of studies of homeopathy do not show that homeopathic medicines have effects beyond placebo.

The Guardian article's author is very obviously in favor of homeopathics and writes in an heavily argumentative, anecdotical-emotional, and if I'm allowed to say almost polemic way (some emphasis added by me):

(An anecdote fo how I ruined a dinner party)
[...]
I know homeopathy works
[...]
The facts, it seems, are being ignored. By the end of 2009, 142 randomised control trials (the gold standard in medical research) comparing homeopathy with placebo or conventional treatment had been published in peer-reviewed journals.
[...]
The "sceptics" campaign had a breakthrough

The facts. The facts are that the quoted reviews state that the overwhelming majority of trials, if they had any interpretable results (apparently there were others too, no information), was of sheer embarrassing quality. Gold standard is a nice buzzword, but the facts are, almost all of the studies named in e.g. Kleijnen's review do not meet the gold standard (not even remotely), as most of them were (contrary to the above claim) not even placebo controlled (or properly randomized, or in some cases both). In addition to that, the larger number of studies had population sizes under 25 which make a meaningful interpretation of data... let's say, a challenge.
Oh heck, I guess I'm with the "sceptics" campaign.

I am personally quite impressed how the reviewers in those 5 systematic reviews still decided to be so forgiving as to allege a possible effect, or some effect with a weak level of evidence when they more or less uniformly stated that significant results were only to be found in the poorest of all studies. Cucherat even calculated (in my opinion doubtful) combined significance levels, followed by explicitly stating that P rapidly goes towards 0.08 if you eliminate the trials with the lowest quality. Then again, however, followed by the puzzling statement "there is some evidence (... of low strength)" when he basically just demonstrated the opposite.

Be that as it may, the claim made by the Guardian article is, from a certain point of view, perfectly correct. Four of these exact five reviews indeed attribute some effect.
Only just, the article, uh... omits some (probably unimportant!) information that doesn't fit so well into the story.

Not the answer you're looking for? Browse other questions tagged .