13

It's often claimed, for example, in this Spectrum News report that a drug combination of hydroxychloroquine and zinc and antibiotics is an effective early treatment for COVID-19.

Researchers at NYU's Grossman School of Medicine found patients given the antimalarial drug hydroxychloroquine along with zinc sulphate and the antibiotic azithromycin were 44 percent less likely to die from the coronavirus.

[...]

Dr. Rahimian says patients in the more critical stages of infection did not fare as well.

And he cautioned that more research is needed - in particular a randomized controlled trial - to prove how and how well the drug combination works.

Yale epidemiologist, Professor Harvey Risch, in his opinion piece in Newsweek, The Key to Defeating COVID-19 Already Exists. We Need to Start Using It argues that the evidence is sufficient to start using it.

enter image description here

I am referring, of course, to the medication hydroxychloroquine. When this inexpensive oral medication is given very early in the course of illness, before the virus has had time to multiply beyond control, it has shown to be highly effective, especially when given in combination with the antibiotics azithromycin or doxycycline and the nutritional supplement zinc.

What evidence is there for or against it?

21
  • Comments are not for extended discussion; this conversation has been moved to chat. – Oddthinking Jul 30 '20 at 9:49
  • 1
    You should not ask evidence against a claim, but evaluate the validity of the evidence that supports it. For example, the NYU study looks weird, I would assume randomized controlled trials are common practice in scientific method. – bradbury9 Jul 30 '20 at 11:13
  • 1
    @bradbury9 RCTs are rare because they are more expensive and take longer (according to Wikipedia) – user17561 Jul 30 '20 at 20:43
  • 1
    @maxb While i'm sure France Soir and Twitter are totally reliable sources for clinical information, I read the actual study report and those claimed issues are clearly not what the study did. Check the sources not the newspapers. – matt_black Aug 1 '20 at 13:37
  • 1
    Does it count as "not dying of COVID-19" if you die of something else first? – Mark Aug 6 '20 at 1:36
37

There is no perfect study yet, but the best evidence we have says HCQ doesn't work

There are well-known rules for judging the quality of evidence in clinical studies. Since discussion this highly charged question seems to frequently consist of selective citation of studies producing the results some advocated of HCQ want to see, it is worth reviewing those rules.

Randomised controlled trials (RCTs) are the gold standard for demonstrating the effectiveness of a treatment. AS this review says (my emphasis):

Randomized controlled trials (RCT) are prospective studies that measure the effectiveness of a new intervention or treatment. Although no study is likely on its own to prove causality, randomization reduces bias and provides a rigorous tool to examine cause-effect relationships between an intervention and outcome. This is because the act of randomization balances participant characteristics (both observed and unobserved) between the groups allowing attribution of any differences in outcome to the study intervention. This is not possible with any other study design.

But RCTs are harder to do than other sorts of study. and they are sometimes not necessary to determine causality. We have no RCTs on whether parachutes lead to lower mortality when falling from great heights. But other sorts of study are subject to many sources of bias and confounding, and not necessarily conscious bias (the easiest person to fool is yourself).

Sometimes there are multiple studies and those studies have conflicting results (especially when the effect is small). In those cases it is sometimes useful to rely on meta-analysis. But a key to good meta analysis is recognising the quality of the studies being analysed. Even RCTs can be badly designed and run. But non-RCTs can also be assessed for bias and quality. See the good advice offered on how to read trial reports here, and how to search for quality here.

We should not be using different standards for judging whether HCQ works. And no serious scientist who wants to know whether it works should be selectively quoting the studies or, worse, the opinion pieces in the popular media claiming it does. Or, even worse, claiming there is a conspiracy to suppress the evidence.

While the evidence is not yet perfect (is it ever in science?) there have been several reviews of the collective evidence. And none consider the evidence for the effectiveness of HCQ to be strong.

A review of the evidence from RCTs

This BMJ survey of RCTs published in july 2020 points out the problem with the current situation (my emphasis):

Faced with the pressures of a global pandemic, healthcare workers around the world are prescribing drugs off-label for which there is only very low quality evidence. The result—and this certainly seems to be the case for the well publicised example of hydroxychloroquine—might be of no benefit but of appreciable harm. Timely evidence summaries and associated guidelines could ameliorate the problem. Clinicians, patients, guideline bodies, and government agencies are also facing the challenges of interpreting the results from trials that are being published at a rate never encountered previously. This environment makes it necessary to produce well developed summaries that distinguish more trustworthy evidence from less trustworthy evidence.

The trials include studies of a variety of possible interventions including both HCQ and HCQ plus other drugs. The summary reported:

32 unique randomised controlled trials were identified that evaluated drug treatments as of 20 July 2020

But concludes:

Glucocorticoids were the only intervention with evidence for a reduction in death compared with standard care

And further:

Fifteen randomised controlled trials including 8654 participants reported mortality. The treatment nodes included in the network meta-analysis were glucocorticoids, hydroxychloroquine, lopinavir-ritonavir, remdesivir, umifenovir, and standard care. The network estimates did not reveal a convincing reduction for any of these interventions compared with standard care.

But the review also admitted that the standard of evidence was often low. This implies that better, bigger studies might give more certainly in the future. But the best evidence we currently have says HCQ doesn't work (alone or in combinations with other treatments). It also convincingly rebuts the idea that we can't do RCTs on covid: the review covers 32 of them.

A review of a broader range of studies

What about the other studies covering more patients than the RCTs reviewed by the BMJ?

This review on treatment regimens containing HCQ in studies involving over 100,000 participants doesn't come to a different conclusion:

This systematic review and meta-analysis showed no clinical benefits regarding HCQ treatment with/without azithromycin for COVID-19 patients. Although mortality rate was not significantly different between cases and controls, frequency of adverse effects was substantially higher in HCQ regimen group

Also worth noting, given the repeated claims that HCQ is "safe" because it is widely used for other diseases:

The meta-analysis of comparative randomized and non-randomized studies showed that the odds of adverse effects occurrence in patients who received the HCQ regimen was approximately 3.5 times higher than the control group without HCQ regimen

But also:

However, due to that most of the studies were nonrandomized and results were not homogenous, selection bias was unavoidable and further large randomized clinical trials following comprehensive meta-analysis should be taken into account in order to achieve more reliable findings

The study is fairly clear, however, that its results do not suggest that its conclusions on HCQ would differ if better RCTs were done.

In their discussion, the study notes both the problem of using observational studies and a key reason why they can be deceptive (the high spontaneous recovery rate can lead to a statistics that give a false sense of success):

More than five months after the closure of the Hubei region, there is still very little high-quality data on every treatment regimen, which raises questions about gaps in scientific works. In this context, one can only be surprised to see observational data without a control group covering a large number of patients when there is an essential need for randomized comparative data. In fact, Analysis of current data is difficult because at least 90% of spontaneous recovery is observed at the base. Thus, it takes well over 90% of recovery to consider that any treatment really brings something.

It is worth reading their analysis of the quality and bias in existing studies.

Conclusion

So reviews of a wide range of both RCTs and observational studies conclude that treatment regimens using HCQ have not been shown to work. Not that some studies haven't shown positive results, but that the collective analysis of many studies (and taking into account quality and bias) do not show any positive signs that HCQ works. What they do show fairly clearly is that HCQ is not harmless and that the side effects are notable (or at least statistically far more notable than the benefits.)

It could–and has–been argued that studies so far have not tested the exact regimen believed to be effective by some medics (eg HCQ+Az+Zn) or have not tested the intervention at the right stage of covid infection. The second part is clearly not true and the first is only partially true but irrelevant. If the believers in early intervention ran a properly conducted high-quality RCT to thoroughly test their hypothesis then skeptics would trust their argument. They have not yet done so. And the excuse that RCTs are expensive is ridiculous when we already have dozens and when so many poorly-controlled observational studies (which also cost something) have been done.

And, as for the argument that there is a conspiracy to prevent the evidence getting out that HCQ is effective because the pharmaceutical industry wants only highly-profitable drugs to be used, why has the best RCT so far recommended dexamethasone, another cheap off-patent drug? I might as well propose the (better evidenced) conspiracy that the believers in HCQ+Az+Zn given early don't want to see a definitive RCT on that idea in case it doesn't prove their hypothesis. I don't believe that, but it is at least as credible as the anti-HCQ conspiracy.

The best evidence we have so far is that HCQ doesn't work. If you want to argue otherwise, provide better evidence.

18
  • 2
    "The best evidence we have so far is that HCQ doesn't work." The best evidence is RCTs, and we don't have any large RCTs according to your own quote, so how can it be showing that HCQ doesn't work for outpatients? – user17561 Aug 4 '20 at 18:03
  • 2
    @MaxB There are plenty of ways academics and other researchers are incentivised to publish bad work to promote their company or their academic position. This is bad, but not a sign a of some general conspiracy to promote a particular point of view (unless you have some particular evidence that people are conspiring to pollute the literature with a specific opinion). There is plenty of bad science on both sides of this specific debate which isn't exactly evidence of a single conspiracy. – matt_black Aug 4 '20 at 18:06
  • 12
    @maxb I think your logic is "the best evidence isn't very good, therefore it must work". This is not how good science or good skepticism works. If better evidence emerges with a different answer, good skeptics will change their mind. You seem to be so convinced HCQ works the evidence does not influence you at all. – matt_black Aug 4 '20 at 18:12
  • 1
    @Tgr some trials have tested early intervention, so the second claim is untrue. The first part is only partially true as some trials have looked at multi-drug interventions. But the relevance of the claim that the exact regimen has not been tested is partially a consequence of the goalposts being moved after studies prove negative and a result of the HCQ advocates not setting up reliable tests of their proposals. – matt_black Aug 10 '20 at 10:44
  • 2
    @MaxB it is always worth reading the things you cite. According to your source non-RCTs account for <10% of approvals and "The common factor in each of these designs is that the study in isolation is insufficient to demonstrate the effectiveness of the treatment". And the paper called for much more thorough monitoring of drugs initially approved this way to check they actually do work. – matt_black Aug 17 '20 at 13:24
17

A double-blind, randomized, controlled trial was just published today in NEJM: A Randomized Trial of Hydroxychloroquine as Postexposure Prophylaxis for Covid-19. This is a trial where Hydroxychloroquine was given very early, just as Risch demands in the article you quote. The results were negative:

After high-risk or moderate-risk exposure to Covid-19, hydroxychloroquine did not prevent illness compatible with Covid-19 or confirmed infection when used as postexposure prophylaxis within 4 days after exposure.

There are plenty of other controlled trials that show that HCQ does not work in clinical settings, but I'm not listing them here as you restricted the scope to a non-clinical setting.

6
  • 3
    Actually, the Boulware et al study showed that HCQ reduced cases by 49%, if taken within 2 days: c19study.com/boulware.html See my updated answer also. – user17561 Aug 7 '20 at 21:15
  • 15
    @MaxB You can't just dice up the results of a clinical trial and examine them in isolation unless the trial was designed to do those analyses. You're cherry-picking results and are constantly moving goalposts here. – Mad Scientist Aug 7 '20 at 21:23
  • 3
    The delay of treatment was always postulated to be relevant. – user17561 Aug 11 '20 at 6:17
  • 11
    @MaxB That's not the point, you can't analyze smaller groups out of a trial unless the trial was designed for those specific subgroups. You get results that look statistically significant by pure chance there (see also P-Hacking), and there are probably confounding factors that make the distribution of patients in those groups not equal. The site you get this from also posted a completely bogus and dishonest study, it's very far from a reliable source. – Mad Scientist Aug 11 '20 at 6:23
  • 2
    But it is the point: " just as Risch demands" One of the demands was little delay. Further, this study looked at prevention of infection rather than prevention of hospitalization/ICU/death. – user17561 Aug 11 '20 at 6:45

You must log in to answer this question.