# Is there an unusual distribution of adverse events by lot number for the mRNA COVID-19 vaccines?

This article which has been making rounds on social media for some time, insinuates that something nefarious is going on with the COVID-19 vaccines. It tries to prove this with the VAERS database by showing that some lots (of Spikevax and Comirnaty) have much more death events reported than others:

Their argument goes that a constant production quality should yield an equal distribution among all lots.

Citing their conclusion:

But the investigation of VAERS has also identified the specific batches of Pfizer and Moderna vaccine that have caused the most harm across the USA, which leads to other extremely serious questions requiring urgent answers.

Why is it that certain batches of the vaccine have proven to be more harmful than others?

Why is it that certain batches of Covid-19 vaccine have proven to be deadlier than others?

Are these huge variations between (reported events for) lots unusual or are there benign explanations?

• Deleted a whole lot of "I don't need to answer properly, because I can do a full, unreferenced, statistical analysis in a comment" comments. Deleted a whole lot of "I don't need to answer properly because I can simply apply ad hominem arguments against the source comments. Jan 27 at 1:31
• @Nimloth The chart isn't deaths by lot number. It's lots per death rate. Confusing, I know. Jan 27 at 4:36
• So the chart tells us that there were 4322 lots of vaccine with 0 deaths per lot, 136 lots with 1-10 deaths per lot, .... and 2 lots with 100 or more deaths per lot. Took a while for me to figure it out too... Jan 27 at 13:45
• This seems like an odd Skeptics question. For one, the question and claim seems unclear to me. Are you asking if there are possible benign explanations for an odd distribution? If it's an odd distribution? What, precisely, is the claim? Is that "there is something nefarious going on?" Jan 28 at 4:12

The graph gives the impression that there is a bulge over to the right, indicated by a red arrow.

This is because the range of columns 2-3 is a band of "10 deaths" but columns 4-7 have a band of "20 deaths", so their counts are about twice what they should be.

So I split each of those columns into two bands of "10 deaths" (with half the death count in each), and the graph becomes this:

and the edited logarithm graphic is

There is no longer any "bulge" in the data. The logarithmic scale further exaggerates what are quite small numbers. Note too, that the "0" band is huge in comparison to any other band.

The '5' counts can't be exactly halved, so the graph isn't quite even (1 count isn't a "bulge" anyway). As mentioned by Shadur this is similar to the Prussian cavalry problem.

So there is nothing alarming about the graph at all: it has been distorted either by accident or design, and

• does not represent the truth.

So no, there isn't an unusual distribution.

• I think an assertion that "it shows a Poisson distribution" ought to be based on an appropriate statistical test, not just eyeballed. Jan 26 at 14:45
• A statistical test would tell us that, too. Jan 26 at 14:49
• This doesn't answer the question. The "bulge" you're talking about appears to be completely unrelated to claim the question is asking about. The claim was that certain lots are producing hundreds of reported deaths in VAERS, while the majority of lots produce 0 such reports. The claim suggests that if the Pfizer vaccines were of uniform quality across different lots, reported deaths would be uniformly distributed across all lots, not concentrated in a few dozen. Is the graph accurate? If so, why are the bars on the right not all 0? Is it explainable by random chance? That's the question. Jan 26 at 21:48
• We've been vaccinating people for well over a year by now, and each lot gets used up quickly - the expected number of adverse events will depend heavily on how long ago the vaccines in a lot were administered. So we wouldn't expect the numbers to follow a Poisson distribution anyway; a better model would be that each lot is individually a Poisson distribution whose parameter is another random variable based on the rate of vaccines administered over time. Then we should expect higher variance (or a "fatter tail") than we would if all lots followed the same Poisson distribution. Jan 26 at 22:38
• @NateEldredge I can in fact eyeball that this is definitely not a Poisson distribution. The huge peak at 0 means λ is at most 0.5 or something, which means you wouldn't expect any lots with more deaths than about 5. The fact that there are 28 lots with more than 40 deaths would be astronomically improbable if we could assume independent probabilities of death (which is when a Poisson distribution would be expected to appear). I don't believe there is anything nefarious going on, but none of the current explanations are at all convincing. I think kaya3 might be on to something though. Jan 27 at 10:09

This is an excellent example of a loaded question. This question implies that many people have died as a result of being administered mRNA vaccines for COVID, and asks why the distribution of those deaths is not as expected. In reality, there are very, very few deaths linked to the mRNA vaccines, which have an outstanding safety profile (which has been established through extensive clinical trials, and has been confirmed by comprehensive surveillance).

VAERS does not establish causality. The reporting standard for the COVID vaccines is very broad. What you are seeing in that data is people who die of unrelated things after being vaccinated.

Very, very few people have died as a result of an adverse effect from the Pfizer vaccine. It's hard to know for sure the total number worldwide, but after almost a billion doses I can count the number of people who died because of a reaction to the vaccine on one hand.

142 occurred after receipt of the BNT162b2 vaccine; of these cases, 136 diagnoses were definitive or probable. The clinical presentation was judged to be mild in 129 recipients (95%); one fulminant case was fatal.

Even this death is not causally linked to the vaccine. A certain number of people will get myocarditis every month.

This graph does not show that certain lots of the Pfizer vaccine are "worse", and in fact does not show that any deaths have occurred as a result of being administered the vaccine.

Several comments have suggested that this answer is dodging the question. This is a dishonest rhetorical device used to attack reasonable responses to loaded questions. Those comments are irrelevant to this answer.

• I agree with your premise that VAERS data is unvetted. It is just a proxy to give researchers an idea of where to look and what to study. But that means if the VAERS data does give us some unexpected correlations, like some lots [allegedly] getting far more deaths than expected, then we can't dismiss it out of hand. It needs investigating and verifying. This answer just dismisses the data as worthless which is unwarranted. Jan 28 at 5:24
• it sounds like the arguments in the past that no deaths are directly caused by smoking or drinking or taking drugs or whatever that in present times is taken for granted. Just saying. Jan 28 at 11:49
• Those deaths are linked to the rare blood adverse event that has been causally linked to the AstraZeneca vaccine. Not the Pfizer vaccine. As discussed in the comments. That you deleted.
– CJR
Jan 28 at 13:15
• @Oddthinking VAERS data are not even hypothesis generating, and they don't represent a useful dataset for statistical analysis of adverse events. They are reports, which are investigated. That produces the dataset for analysis. You can, in fact, report a made up adverse event right now, and it will stay in VAERS. Jan 28 at 15:43
• Can you please clarify and cite the part about the total deaths here? I suspect there is considerable confusion here as well as to which vaccines people are referring, the question is specifically about the mRNA vaccines, but it is not very obvious that this is the case. The Link by @Oddthinking cites deaths linke to the AstraZeneca vaccine, not to the mRNA vaccines. Jan 28 at 17:06

There is no evidence here for the claim that an association between mortality and vaccine lot would be evidence of "something nefarious with regard to vaccine distribution".

Vaccines from within a lot are not randomized across the population.

Early distribution was intentionally associated with expected mortality. See, e.g, The MMWR initial and follow up report on ACIPs recommended Vaccination Distribution Strategy, and the HHS statement on distribution strategy. Phase 1a included health care workers and long term care facility residents. Health care workers were prioritized to maintain hospital capacity, long term care facility residents were prioritized because they have a high mortality risk. Long term care facility residents have both a higher risk of COVID-19 associated mortality, and all cause mortality.

Different states reached different phases of distribution at different times, but priority was given to people at a higher risk of death.

Because some lots went to sub-populations with a higher risk of all cause mortality, we should expect some lots to be associated with higher all-cause mortality.

Is there an unusual distribution of adverse events by lot number for the mRNA COVID-19 vaccines?

# The data cannot support this nor other claims made in the article.

No conclusions about actual harm can be made from the VAERS data alone.

## What is VAERS?

The article consistently conflates unverified reports of harm with actual harm. For example.

This data alone shows that there have been 118 times as many adverse reactions, 174 times as many deaths, and 140 times as many hospitalisations due to the Moderna Covid-19 jab than there have been due to all other influenza vaccines combined.

"This data" show there have been unverified reports of adverse reactions. "This data" is VAERS, the Vaccine Adverse Event Reporting System run by the CDC. Misrepresenting VAERS is a favorite of anti-vaxxers.

VAERS is a data collection system for reports of harm which may be connected with a vaccine. Anyone can make a VAERS report. The intent of VAERS is to collect reports and follow up on them. It cannot be used alone to determine harm.

The CDC says this about VAERS

• VAERS reports are submitted by anyone and sometimes lack details or contain errors.
• VAERS data alone cannot determine if the vaccine caused the reported adverse event.

This specific limitation has caused confusion about the publicly available data, specifically regarding the number of reported deaths. In the past there have been instances where people misinterpreted reports of death following vaccination as death caused by the vaccines; that is a mistake.

VAERS accepts all reports of adverse events following vaccination without judging whether the vaccine caused the adverse health event. Some reports to VAERS might represent true vaccine reactions, and others might be coincidental adverse health events not related to vaccination at all.

Generally, a causal relationship cannot be established using information from VAERS reports alone.

• The number of reports submitted to VAERS may increase in response to media attention and increased public awareness.
• It is not possible to use VAERS data to calculate how often an adverse event occurs in a population.

VAERS can, at best, show anomalies in what is being reported as an adverse reaction to the COVID-19 vaccine. Given the charged nature of COVID-19 vaccination we would expect a lot of noise in the reports as people attribute any post-vaccination reaction to the vaccine.

The next step would be to verify these reports, the article makes no attempt to so.

## Flu vaccines are a questionable control

The article uses VAERS reports about the flu vaccine as a control. And if VAERS represented actual deaths due to vaccination that might be fine. But VAERS represents unverified reports about deaths which could be attributed to vaccination.

Any differences in their VAERS reports could be because of variations in how people feel about the flu vaccines vs COVID-19 vaccines. Because of this variable, using VAERS reports about the flu as a control can only show differences in the VAERS reports. No conclusions about actual harm can be made from the VAERS data alone.

# Their analysis of the data has flaws.

Putting that aside, does the article show there is an unusual distribution of adverse events in the VAERS reports? Let's look at problems with their analysis.

## Lot sizes are not equal

The whole article is based on reports per lot and that makes the whole analysis moot.

For their analysis to be significant they need reports per dose. Otherwise differences in lot sizes could explain the variations. You'd expect a lot of 10,000 doses to have more reports than a lot of 100 doses.

They could get reports per dose by controlling for each lot's size.

They didn't, instead they just assume an average size and apply that.

The above chart shows the number of adverse event reports made to VAERS against the Pfizer Covid-19 vaccine sorted by the lot number of vaccine that was administered prior to the adverse event. We do not have reliable information about standard lot size, but news articles indicate an average lot size of 1000 vials (approx. 6000 doses).

Without controlling for the size of each lot, no conclusions can be drawn. All their conclusions can be explained by different lot sizes.

## Lot groups are not equally sized

When they group by lot number, it is unclear what size their groups are. Eyeballing one graph we see lot groups of 261203, 276532, 276544, 276560, 276571, 279734, 279796. The differences between lot numbers are 15329, 12, 16, 11, 3163, 62.

Are these the group sizes? Are there gaps in the lot numbers? If so, are the gaps real? No explanation is given.

Comparing different sized groups of different sized lots would certainly explain the variations.

## Lot numbers are not random

They do, a few times, group by lot number and sort them alphabetically. This might be meaningful if lot numbers were random, but the article does not claim that.

Lot numbers are likely tied to several variables such as when, where, and who produced the vaccine. Grouping by lot number and alphabetizing is likely producing a jumble of these variables.

This can have the effect of making it look like what is really the effect of many variables look like it comes from the lot numbers.

## <12 states vs 13-50 states: broad, asymmetrical, multi-variable

Similarly, the article splits the data into two groups.

• Lots which were distributed to 12 states or fewer.
• Lots which were distributed to 13 to 50 states.

And then finds more reports were in the 13-50 than in the 1-12 state group. Is this significant?

Why 12? Because 12 is the maximum number of states a single lot of their control flu vaccine was sent to. I'll have more to say on that control later, but they lack a control for the 13-50 set. Are the anomalies due to the vaccine? Or due to being shipped widely across the US? Without a control we can't say.

The sizes of the two sets vary wildly. In the Pfizer case the 1-12 group is 4,289 lots or 97% of the data, while the 13-50 group is just 130 or 3%. Large data sets dampen anomalies, small data sets amplify them.

However, the absolute numbers appear compelling. 99 death reports for the 1-12 group vs 2,799 death reports for the much smaller 13-50 group. The largest number of reports from a single lot is 114 which rules out any small group of very anomalous lots. If true, this is interesting. But can we conclude this is because of the 1-12 vs 13-50 distribution? No.

The problem is lumping all the lots into two broad groups has made a jumble of all the variables. For example, we can imagine that if you're distributing a lot to 13-50 states it has to travel further and longer; is it a proxy for distance and time in transit? Is it a proxy for how they're stored and transported? Why were the lots split? Is it because they were distributed to states with lower populations, lower population densities, and smaller hospitals? Is it a proxy for geographical location? Is it something to do with the lot splitting process?

What if early in the pandemic the lots were distributed widely across the US? Then they're just graphing reporting over time.

Distributing to 13 states vs 50 states is a huge geographical distance that's been lumped together. What happens if you group in finer increments? 50 states includes Hawaii and Alaska, non-contiguous parts of the US requiring transport by ship or plane; what happens if you separate them?

If there was, for example, one state which usually got split lots and had anomalies in their VAERS system that single state could throw the whole 13-50 category off.

The 1-12 vs 13-50 data does bear some further investigation, but no conclusions can be drawn from that data long. With two broad and asymmetrical groups the individual variables must be teased apart. Instead of doing that investigation, the article instead stokes a conspiracy theory about distributing the safe vaccine to only certain states.

## Low sample numbers

According to the charts, of 4519 lots...

• 96% (4322) reported no deaths.
• 3% (136) reported 1-10 deaths.
• 1% (61) reported more than 10 deaths.

The 96% case is no deaths reported. 99% is <= 10 deaths reported. The claim is about that 1%, just 61 lots. Conclusions about the 1% cannot be applied to the other 99%. All that can be concluded is there is something different about that 1%.

They are in such a low number of samples, no band with more than 10 deaths represents more than 15 lots, that we expect high variation.

The article does this elsewhere. In "Finding 2: Pfizer Lots sent to 13-50 states Have Unusually High AE Reports and Deaths" they compare 4289 lots distributed to 12 states with 130 lots distributed to more than 12. 97% of the lots compared to 3% of the lots. Again comparing a high number of lots with a low number of lots. We expect more variability in small samples.

## Exaggerating small variations

Are these huge variations between (reported events for) lots unusual or are there benign explanations?

These are not huge variations, they're only made to look that way. The article exaggerates small sample sizes and small variations through various tricks.

The article cherry picks anomalous data by in most cases discarding the 96 to 99% of data which has no or very few reported deaths and focusing in on the remaining 1 to 4% and using that to draw conclusions about the whole complete data set, or making them seem of equivalent size to the whole data set.

The article uses logarithmic charts. These artificially exaggerate small variations and dampen large ones. In the OP's logarithmic chart 1% of the data (11-20 and to the right) looks equivalent to 96% of the data (the 0 bar).

The remaining sample sizes are limited to about 100 or sometimes much less. When split up into groups, those groups can contain as few as 5 lots. When the numbers are that low, a variation of a single lot is as 20% difference.

Weather Vane covered one trick; the "Range of reported bands by lot" are not of equal width, some cover 10, some cover 20. This has the effect of grouping small numbers together making some groups look larger than they are.

# They grouped the anomalies together and found they are anomalous.

The headline of the article is:

100% of Covid-19 Vaccine Deaths were caused by just 5% of the batches produced according to official Government data.

To support this claim they show lots of charts where they group the lots by number of reported deaths. They grouped the lots by reported deaths and found the lots are grouped by reported deaths. This is a tautology. It's like being concerned that 50% of people are below average.

What the data in the article does show is that 96% of lots had no reported deaths, 99% of lots had 10 or fewer reported deaths, and only 1% of lots had more. That seems fine.

In other places they find individual lot numbers which are "the most harmful" (has the most reports). This is expected when they are looking for the lot which has the most reports.

To discover a concerning anomaly one would have to group by a variable which should be independent. For example, the last digit of the lot number, or the day of week or time of day or month the lot was produced, or the facility which produced the lot. These groupings should be independent of the number of deaths.

If they found otherwise, that would be of concern. But only of concern, it wouldn't prove anything because they're using the wrong data.

# Conclusion

We can conclude with the article's conclusion.

But the investigation of VAERS has also identified the specific batches of Pfizer and Moderna vaccine that have caused the most harm across the USA, which leads to other extremely serious questions requiring urgent answers.

• If you go looking for anomalies, you will find them. They fail to show they are significant.
• VAERS cannot be used to determine actual harm.
• At best this requires further investigation of the VAERS data by the author of the article to rule out other variables.

Why is it that certain batches of the vaccine have proven to be more harmful than others?

• Because the article grouped the batches like that.

Why is it that certain batches of Covid-19 vaccine have proven to be deadlier than others?

• Since this is all based on unverified reports, nothing in the article shows actual deaths.
• With such a large, unverified data set across a large population with such a charged topic, we'd expect such variations in reports.
• The variations represent 1%-3% of the data.

Why is it that the most harmful and deadly Covid-19 vaccines were distributed across the entire USA [13 or more states], whilst the least harmful and deadly were only ever distributed to a few states [12 or fewer]? Was this done on purpose?

• Since this is all based on unverified reports, nothing in the article shows actual harm nor deaths.
• The 13 or more set represents 3% of the data.
• The two sets may be a proxy for any number of other variables such as geographical location, political leanings, time during the pandemic, method of transport, age of the vaccine, method of storage, etc.

A Pfizer whistleblower from a Kansas manufacturing facility did after all reveal that “People are being made to sign off on things that normally they wouldn’t, and then they wonder why their own employees won’t take it”.

• Out of context non sequitur.

There is a very simple, and to me beautiful, explanation. This is original research by me - although I am basing it on facts for which I have sources, though only secondary ones. My main sources are in fact articles which promote the claims made in this article.

Fact: At some point around January 24 2022, the US reported around 534M vaccine shots given. Fact: The batch size for Pfizer is 1.5M doses. Fact: A researcher trying to prove wrongdoing obtained 440 batch numbers from the CDC.349 of these appear in VAERS. It is not clear whether these were all batch issued in the US.

Conjecture: It seems very likely that the 440 batch numbers are ALL or almost all batch numbers issued in the US until that time, since the numbers are consistent with this (calculations in the source)

Fact: 4,522 batch numbers are included in the analysis of the Pfizer vaccine. 5,510 for Moderna.

Fact: The VAERS data includes entries which are not in the correct format of the Pfizer batch numbers, including "Pfizer", "Unknown" as well as entries starting with numbers rather than letters, two batch numbers in the field, putting "#" or "Pfizer"in front of the number etc.

Source

Conjecture: Of the 4,522 different entries for "Batch number" in the Pfizer data, all except a few hundred are in fact typos, misreadings, missing data, or alternative ways of writing the same data.

Conjecture: If data not matching the list of batch numbers actually issued are removed the remaining data will show a completely different graph, which would not be able to support the conclusions of the article.

To prove all of this, one would need to obtain the list of batch numbers from the CDC. I have written them requesting this. It is possible that a US citizen could do so using the FOIA, if I am not succesful.

Note: The APF fact checking site has a page dealing with this article. While they refute what they see as the main claim of the article - that the number of deaths and other adverse effects for some batch numbers are HIGHER than expected - they do not attempt to explain why so many batch numbers have LOW numbers. source

• That website simultaneously claims it's an accident and "a crime against humanity". LOL. It's just hodge-podge designed to be long enough to confuse the issue further. Kinda like Russian state propaganda these days. "firehose of nonsense"
– Fizz
Mar 13 at 22:17
• Welcome to Skeptics! Please provide some references to support your claims. In particular: who says this explains the results, and how did they reach their conclusion? (Arguing that someone could refute this with evidence is begging the question.) Also, who are these fact checking organisations? Who says the distribution is strange? [Doesn't look strange to me, but I haven't done a proper analysis.] Mar 14 at 0:42