There is no perfect study yet, but the best evidence we have says HCQ doesn't work
There are well-known rules for judging the quality of evidence in clinical studies. Since discussion this highly charged question seems to frequently consist of selective citation of studies producing the results some advocated of HCQ want to see, it is worth reviewing those rules.
Randomised controlled trials (RCTs) are the gold standard for demonstrating the effectiveness of a treatment. AS this review says (my emphasis):
Randomized controlled trials (RCT) are prospective studies that measure the effectiveness of a new intervention or treatment. Although no study is likely on its own to prove causality, randomization reduces bias and provides a rigorous tool to examine cause-effect relationships between an intervention and outcome. This is because the act of randomization balances participant characteristics (both observed and unobserved) between the groups allowing attribution of any differences in outcome to the study intervention. This is not possible with any other study design.
But RCTs are harder to do than other sorts of study. and they are sometimes not necessary to determine causality. We have no RCTs on whether parachutes lead to lower mortality when falling from great heights. But other sorts of study are subject to many sources of bias and confounding, and not necessarily conscious bias (the easiest person to fool is yourself).
Sometimes there are multiple studies and those studies have conflicting results (especially when the effect is small). In those cases it is sometimes useful to rely on meta-analysis. But a key to good meta analysis is recognising the quality of the studies being analysed. Even RCTs can be badly designed and run. But non-RCTs can also be assessed for bias and quality. See the good advice offered on how to read trial reports here, and how to search for quality here.
We should not be using different standards for judging whether HCQ works. And no serious scientist who wants to know whether it works should be selectively quoting the studies or, worse, the opinion pieces in the popular media claiming it does. Or, even worse, claiming there is a conspiracy to suppress the evidence.
While the evidence is not yet perfect (is it ever in science?) there have been several reviews of the collective evidence. And none consider the evidence for the effectiveness of HCQ to be strong.
A review of the evidence from RCTs
This BMJ survey of RCTs published in july 2020 points out the problem with the current situation (my emphasis):
Faced with the pressures of a global pandemic, healthcare workers around the world are prescribing drugs off-label for which there is only very low quality evidence. The result—and this certainly seems to be the case for the well publicised example of hydroxychloroquine—might be of no benefit but of appreciable harm. Timely evidence summaries and associated guidelines could ameliorate the problem. Clinicians, patients, guideline bodies, and government agencies are also facing the challenges of interpreting the results from trials that are being published at a rate never encountered previously. This environment makes it necessary to produce well developed summaries that distinguish more trustworthy evidence from less trustworthy evidence.
The trials include studies of a variety of possible interventions including both HCQ and HCQ plus other drugs. The summary reported:
32 unique randomised controlled trials were identified that evaluated drug treatments as of 20 July 2020
But concludes:
Glucocorticoids were the only intervention with evidence for a reduction in death compared with standard care
And further:
Fifteen randomised controlled trials including 8654 participants reported mortality. The treatment nodes included in the network meta-analysis were glucocorticoids, hydroxychloroquine, lopinavir-ritonavir, remdesivir, umifenovir, and standard care. The network estimates did not reveal a convincing reduction for any of these interventions compared with standard care.
But the review also admitted that the standard of evidence was often low. This implies that better, bigger studies might give more certainly in the future. But the best evidence we currently have says HCQ doesn't work (alone or in combinations with other treatments). It also convincingly rebuts the idea that we can't do RCTs on covid: the review covers 32 of them.
A review of a broader range of studies
What about the other studies covering more patients than the RCTs reviewed by the BMJ?
This review on treatment regimens containing HCQ in studies involving over 100,000 participants doesn't come to a different conclusion:
This systematic review and meta-analysis showed no clinical benefits regarding
HCQ treatment with/without azithromycin for COVID-19 patients. Although mortality rate
was not significantly different between cases and controls, frequency of adverse effects was substantially higher in HCQ regimen group
Also worth noting, given the repeated claims that HCQ is "safe" because it is widely used for other diseases:
The meta-analysis of comparative randomized and non-randomized studies showed that the
odds of adverse effects occurrence in patients who received the HCQ regimen was
approximately 3.5 times higher than the control group without HCQ regimen
But also:
However, due to that most of the studies were nonrandomized and results were not homogenous, selection bias was unavoidable and further large randomized clinical trials following comprehensive meta-analysis should be taken into account in order to achieve more reliable findings
The study is fairly clear, however, that its results do not suggest that its conclusions on HCQ would differ if better RCTs were done.
In their discussion, the study notes both the problem of using observational studies and a key reason why they can be deceptive (the high spontaneous recovery rate can lead to a statistics that give a false sense of success):
More than five months after the closure of the Hubei region, there is still very little high-quality data on every treatment regimen, which raises questions about gaps in scientific works. In this context, one can only be surprised to see observational data without a control group covering a large number of patients when there is an essential need for randomized comparative data. In fact, Analysis of current data is difficult because at least 90% of spontaneous recovery is observed at the base. Thus, it takes well over 90% of recovery to consider that any treatment really brings something.
It is worth reading their analysis of the quality and bias in existing studies.
Conclusion
So reviews of a wide range of both RCTs and observational studies conclude that treatment regimens using HCQ have not been shown to work. Not that some studies haven't shown positive results, but that the collective analysis of many studies (and taking into account quality and bias) do not show any positive signs that HCQ works. What they do show fairly clearly is that HCQ is not harmless and that the side effects are notable (or at least statistically far more notable than the benefits.)
It could–and has–been argued that studies so far have not tested the exact regimen believed to be effective by some medics (eg HCQ+Az+Zn) or have not tested the intervention at the right stage of covid infection. The second part is clearly not true and the first is only partially true but irrelevant. If the believers in early intervention ran a properly conducted high-quality RCT to thoroughly test their hypothesis then skeptics would trust their argument. They have not yet done so. And the excuse that RCTs are expensive is ridiculous when we already have dozens and when so many poorly-controlled observational studies (which also cost something) have been done.
And, as for the argument that there is a conspiracy to prevent the evidence getting out that HCQ is effective because the pharmaceutical industry wants only highly-profitable drugs to be used, why has the best RCT so far recommended dexamethasone, another cheap off-patent drug? I might as well propose the (better evidenced) conspiracy that the believers in HCQ+Az+Zn given early don't want to see a definitive RCT on that idea in case it doesn't prove their hypothesis. I don't believe that, but it is at least as credible as the anti-HCQ conspiracy.
The best evidence we have so far is that HCQ doesn't work. If you want to argue otherwise, provide better evidence.
