33

This popular tweet (over 10000 retweets) claims that:

This study shows that after three months the vaccine effectiveness of Pfizer & Moderna against Omicron is actually negative. Pfizer customers are 76.5% more likely and Moderna customers are 39.3% more likely to be infected than unvaxxed people.

It offers the following image as proof: Table

It also links this study: Vaccine effectiveness against SARS-CoV-2 infection with the Omicron or Delta variants following a two-dose or booster BNT162b2 or mRNA-1273 vaccination series: A Danish cohort study

I want to verify two things:

  1. Is this assessment of the study accurate?
  2. Is the study itself trustworthy and accurate?
Improve this question
1
  • Old question, but note that vaccine effectiveness is a technical term and not the same as vaccine efficacy (which is more what we'd intuitively think it means). More about this at a recent, similar question skeptics.stackexchange.com/a/54220/8865
    – cbeleites
    Commented Jan 2, 2023 at 22:59

3 Answers 3

Reset to default
68

No it doesn't. That is directly mentioned in the study:

The negative estimates in the final period arguably suggest different behaviour and/or exposure patterns in the vaccinated and unvaccinated cohorts causing underestimation of the VE. This was likely the result of Omicron spreading rapidly initially through single (super-spreading) events causing many infections among young, vaccinated individuals.

This study essentially compared vaccinated and unvaccinated people directly, it didn't try to adjust for many confounding factors. This is a really simple study, it's only 6 pages in total. It's more like looking at raw data, there are a ton of potential confounders here that simply aren't part of this study. You can see from the quoted paragraph that the authors clearly think that the negative values are an artifact of the study design, and not any real effect.

Improve this answer
15
  • 27
    Pardon my ignorance but I don't get this answer. What you say seems to be just one interpretation. Yes, the data absolutely can be interpreted to mean that the effectivenes of said vaccines vs omicron is negative after said amount of time. In fact, that is literally what the table says. Where is the comparison with unvaccinated people? What do they have to do with it? The certainty with which you deny what the table evidently indicates seems scientifically questionable to my humble little brain. Even if slight underestimation of VE was true, a whole -76% is just an artifact? Really?
    – csstudent1418
    Commented Dec 25, 2021 at 10:25
  • 59
    @csstudent1418 They compared vaccinated with unvaccinated people to estimate vaccine effectiveness. The problem is that you have to make really sure you're comparing similar populations, or the results will not actually measure vaccine effectiveness. For example, if only vaccinated people are allowed to attend restaurants or parties, this will likely lead to many more vaccinated people being infected by a variant that can circumvent the vaccine protection, because the vaccinated people have more exposure than the unvaccinated that can't go to restaurants.
    – Mad Scientist ♦
    Commented Dec 25, 2021 at 11:20
  • 54
    @csstudent1418 It's the typical "correlation != causation" problem. If lots of companies require their workers to be vaccinated, unvaccinated people won't be at the worklace, so they'll have fewer workplace accidents. But that doesn't mean that the vaccine causes workplace accidents.
    – Barmar
    Commented Dec 25, 2021 at 13:12
  • 27
    @Haukinger I wouldn't trust any of the values in that paper on their own. The interesting part is the decline over time, and that is likely to be equally affected by these kinds of biases, so the trend is more reliable than the individual values. I wouldn't really trust this particular paper very far, but the conclusion that the protection wanes over time is also supported by a lot of other observations.
    – Mad Scientist ♦
    Commented Dec 25, 2021 at 13:41
  • 11
    I don't really accept this as an answer since the comment in the study is just a hypothesis on what could, in theory, produce this result. The result could have also been produced, or contributed to, by a vaccine that causes one to be more susceptible to this variant. It's fair to say that it's unknown at this point. There is no clear data proving either.
    – Ruslan Oblov
    Commented Dec 25, 2021 at 21:00
17

This is a very good illustration of one of the most fundamental lessons of statistics: statistics can show you that an effect exists but not what the cause or nature of the effect is.

Suppose that a certain country introduces a new road rule that cyclists cannot use public roads unless they wear a helmet. Six months later, someone does a study and shows, statistically, that cyclists wearing helmets are more likely to have a fatal crash than those who aren't. Should we then conclude that the helmets make cycle crashes somehow more dangerous? Should we suggest that helmets may physically work but the psychological effect is to make the cyclists take more risks? Or should we note instead that cyclists travelling in the most dangerous setting amidst traffic are selected to be the helmet wearers, whereas cyclists without helmets are restricted to the park? Actually the answer is necessarily unclear and the statistics alone cannot clarify it.

The situation is the same here. Yes, it is theoretically possible that the vaccines are biochemically opening the door to omicron, but that is not proven and would be similarly surprising to helmets accentuating head injury. There is prior research into the economic theory of moral hazard, suggesting that vaccinated people may take more risks because they think that they are safer than they are. Denmark uses the coronapas system to filter access to public transport, service and hospitality industries, and other large or close packed groups, which means vaccinated people are more frequently selected for the dangerous settings a bit like our cyclists on the highway.

As such, it is from the statistics alone intrinsically unclear whether the vaccine is unfit for purpose, or whether the rule that vaccinated people are assumed safe enough to be allowed into dangerous places is unfit. Denmark believes that other evidence favours the latter explanation, and as such the covidpas validity will change to require a vaccine not more than 7 months ago.

Improve this answer
4
  • 1
    The keyword you might mention here is risk compensation or risk homeostasis.
    – gerrit
    Commented Dec 28, 2021 at 9:55
  • Indeed, short of some natural experiment, it will be hard to convince people. I would think however, that the unvaccinated camp skew younger, and tend to be less/not worried about the virus, and thus take more exposure risks. With this in mind, I would propose we could get a reasonably confident read on efficacy vs omicron if we also had an age control. Age is the big factor to absolutely control for however. We know the vaccines don't work particularly well in general, and that age is the dominating protective factor. If unvaccinated are sufficiently younger, that can drive this effect.
    – JPErwin
    Commented Dec 29, 2021 at 16:10
  • 1
    @JPErwin I don't think we can know the relative importance of confounders ahead of time. One that I'm concerned about, because it is particularly hard to untangle, is variation in testing /reporting compliance. If unvaccinated people test less and report their tests less than vaccinated people (and we have no data on how much less; they tend to distrust researchers) it could just be that many unvaccinated people got infected but their infections aren't in the database. I don't think an age control or any other statistical method could even theoretically fill in systematically missing data.
    – Josiah
    Commented Dec 30, 2021 at 0:24
  • On further investigation, it seems that even the test asymmetry can be addressed. I will try to update the answer later.
    – Josiah
    Commented Dec 30, 2021 at 11:22
1

We should be careful not to confound two things:

  • Protection against getting infected, and
  • Protection against getting sick with the subsets of hospitalised, dead, and long covid.

The study measures protection against getting infected:

Vaccine effectiveness (VE) was estimated in a time-to-event analysis of Danish residents ≥12 years comparing the rate of infections in unvaccinated and vaccinated individuals with a two-dose BNT162b2 or mRNA-1273 vaccination series.

It doesn't seem to consider whether those infections are symptomatic or not. This is not the authors' fault, but in science communication, people may see the headline "negative effectiveness", and think they're safer without a vaccine. This conclusion is not supported by the available evidence in the paper. In fact, they themselves quote another study:

A recent study from England (in preprint) found higher effectiveness against symptomatic Omicron initially after BNT162b2 vaccination followed by a rapid decline in protection, and that VE increased to 75.5% (56.1 to 86.3%) two weeks after booster vaccination using unvaccinated individuals as comparison.

Unfortunately, they don't actually cite the preprint, so we cannot tell the magnitude of the declining efficiency against symptomatic infection for people who have received two doses of the vaccines under consideration.

All quotes from the original paper.

(On another note: the manuscript is very short and has not yet been peer-reviewed. It is possible that even the conclusion as presented in the manuscript does not follow from the available data.)

Improve this answer
1
  • Why the downvote?
    – gerrit
    Commented Dec 28, 2021 at 15:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .