-1

As I have not much skill in reading medical research, I wanted to verify my calculation of this research results.

A big peer-reviewed research titled Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting measured occurance of adverse effects among vaccinated people, up to 21 days after each shot, vs the unvaccinated control group, and generated these results:

BNT162b2 mRNA Covid-19 Vaccine Adverse Events

The sum of excess adverse effects in the vaccinated group is 139.3 per 100,000 persons.

Does this mean an adverse effects rate of ~1 to 718? Can the accuracy of this calulation be improved?

5
  • 1
    Also, is this the best place to ask this question? Wondered whether it better belongs to Cross Validated..
    – Roy
    Oct 25 '21 at 6:23
  • 2
    There may be multiple effects in the same person. Also, I'm not sure how helpful is to sum different types... or how likely it is that e.g. "herpes zoster infection" was actually caused by the vaccine.
    – Fizz
    Oct 26 '21 at 10:56
  • 2
    Further, that table also lists events which are rarer in the vaccinated group than in the (ad-hoc) "controls". And things for which there is no statistically significant difference (95% CI for risk diff. contains the zero). And doesn't list common side effect like pain at the injection site.
    – Fizz
    Oct 26 '21 at 11:18
  • Note that there are 25 classes of adverse events listed. With a 95% confidence interval, we'd expect between one and two of the correlations to be spurious -- that is, due to random chance causing something to fall outside the confidence interval, not due to an actual effect.
    – Mark
    Oct 27 '21 at 21:05
  • 1
    Downvoters: thank you for your feedback. I made an effort to write a serious, neutral question (which is my first in this site). Please let me know what should be improved.
    – Roy
    Oct 28 '21 at 17:11
2
+50

I am neither a medical professional nor a statistician, so caveat emptor and all that

Short answer is no.

This is because, in the article the authors state:

A total of 1,736,832 persons were eligible for inclusion in the vaccination cohort...

...The vaccination cohorts included a mean of 884,828 vaccinated persons, with a median age of 38 years

If you compare these to the numbers of persons in the adverse events cohort column recorded for each category, then you can see that a sum total of persons (22,080,670, if my maths is correct) would be well in excess of the enrolled cohort. This means that each reported event is not independent and it is likely that a person who had one adverse event also had at least one other reported event.

Thus, a simple sum of the excess events is not something that can be performed. Indeed, if this were the case, the authors would likely have done so in the paper, but they did not, leading to the conclusion that it is not a valid technique.

Now we must consider that adverse event risk ratios are calculated for vaccines and other medicines, so there must be some methodology to do so, but what that methodology is, I don't know though I suspect an odds-ratio calculation of some sort.

Note that the authors only considered some of these events to be a significant difference, even if the risk ratio or risk difference were higher than expected, so they excluded parathesia (tingling/prickling sensation) as well as vertigo (dizziness), but it looks like the excluded ones are all minor symptoms not associated with a general morbidity/mortality:

The risk was substantially higher on either the multiplicative (risk ratio) or additive (risk difference) scales in the vaccinated group than in the unvaccinated group for myocarditis (risk ratio, 3.24; 95% confidence interval [CI], 1.55 to 12.44; risk difference, 2.7 events per 100,000 persons; 95% CI, 1.0 to 4.6), lymphadenopathy (risk ratio, 2.43; 95% CI, 2.05 to 2.78; risk difference, 78.4 events per 100,000 persons; 95% CI, 64.1 to 89.3), appendicitis (risk ratio, 1.40; 95% CI, 1.02 to 2.01; risk difference, 5.0 events per 100,000 persons; 95% CI, 0.3 to 9.9), and herpes zoster infection (risk ratio, 1.43; 95% CI, 1.20 to 1.73; risk difference, 15.8 events per 100,000 persons; 95% CI, 8.2 to 24.2).

They also note the risks associated with SARS-CoV-2 infection in general and in figures 3 and 4 of the paper comparing to the risk with vaccination:

Infection substantially increased the risk of many different adverse events, including myocarditis (risk ratio, 18.28; 95% CI, 3.95 to 25.12; risk difference, 11.0 events per 100,000 persons; 95% CI, 5.6 to 15.8), acute kidney injury (risk ratio, 14.83; 95% CI, 9.24 to 28.75; risk difference, 125.4 events per 100,000 persons; 95% CI, 107.0 to 142.6), pulmonary embolism (risk ratio, 12.14; 95% CI, 6.89 to 29.20; risk difference, 61.7 events per 100,000 persons; 95% CI, 48.5 to 75.4), intracranial hemorrhage (risk ratio, 6.89; 95% CI, 1.90 to 19.16; risk difference, 7.6 events per 100,000 persons; 95% CI, 2.7 to 12.6), pericarditis (risk ratio, 5.39; 95% CI, 2.22 to 23.58; risk difference, 10.9 events per 100,000 persons; 95% CI, 4.9 to 16.9), myocardial infarction (risk ratio, 4.47; 95% CI, 2.47 to 9.95; risk difference, 25.1 events per 100,000 persons; 95% CI, 16.2 to 33.9), deep-vein thrombosis (risk ratio, 3.78; 95% CI, 2.50 to 6.59; risk difference, 43.0 events per 100,000 persons; 95% CI, 29.9 to 56.6), and arrhythmia (risk ratio, 3.83; 95% CI, 3.07 to 4.95; risk difference, 166.1 events per 100,000 persons; 95% CI, 139.6 to 193.2).

Figure 3 Figure 3 Figure 4 Figure 4

9
  • Thanks, makes sense! If so, what would be a better way to calculate the risk for an adverse effect after vaccination?
    – Roy
    Oct 28 '21 at 17:03
  • BTW the comparison to Covid related adverse effects is problematic given that the chances to get infected without a vaccine are not 100%. Seems like the comparison should be Vaccine RR : Covid RR * Covid risk of infection without a vaccine. But that's unrelated to the question.
    – Roy
    Oct 28 '21 at 17:08
  • @roy - I don't know how to calculate the overall risk, your best bet would be to ask a statistician who specializes in clinical trials data. As for your second comment - that argument doesn't make sense at all. The data presented are for people (unvaccinated) infected with SARS-CoV-2, not the general population of infected and uninfected.
    – bob1
    Oct 28 '21 at 20:13
  • Thanks, I might do so.
    – Roy
    Oct 30 '21 at 5:08
  • Regarding covid vs vaccine AEs comparison - I meant the comparison should be - vaccinated (any covid status) vs unvaccinated (any covid status) - because this represents the actual risks you're considering before deciding on vaccination. The comparison in the research works when 100% of unvaccinated gets covid (and 0% of the vaccinated)
    – Roy
    Oct 30 '21 at 5:11
1

I think there's some missing the forest for the trees here, in the sense that adverse events (aka adverse effects) generally do not have a strict definition beyond something like:

untoward medical occurrences after exposure. These endpoints, which are not necessarily causally related [to the treatment], are called adverse events (or sometimes adverse effects) (AEs)

AEs are often enough protocol-defined. In this Israeli study:

Adverse events were defined according to diagnostic codes and short free-text phrases that accompany diagnoses in the CHS database. A complete list of the study outcomes (adverse events) and their definitions is provided in Table S2.

with the stated intent to capture "a broad set of potential short- and medium-term adverse events among vaccinated persons".

You'll note that the most common AE in table 2 (that you've included in your question) is lymphadenopathy (inflammation of the lymph nodes) which is a common but temporary (and most often benign) side effect of vaccines, but almost certainly not the most common side effect spontaneusly reported... as shown e.g. in this Canadian database, where lymphadenopathy is somewhere in the middle of the pack of the effects reported (and even below myocarditis in the Canadian database, even though that's much, much rarer in the Israeli study):

enter image description here

So the "broad set" was perhaps not so broad in the Israeli study, as if clearly excluded the commonly reported vaccine side effects like headache etc. While you can definitely sum up AEs according to one study's particular definition(s) and even express that total as a ratio to the population under study... it's an not incredibly useful sum/conclusion beyond that study. Whereas specific AEs are in a sense "portable" between studies, assuming they used some reasonably common definition for the individual AEs. (Myocarditis is somewhat complicated as there are [Brighton Collaboration] levels for it. Also the Canadians reported it up to 92 days after vaccination, which is a longer time window than in the Israeli study.) This is almost certainly why the Israeli paper's abstract doesn't present some sum, but the individual AEs that they found statistically significant in the vaccine's direction:

Vaccination was most strongly associated with an elevated risk of myocarditis (risk ratio, 3.24; 95% confidence interval [CI], 1.55 to 12.44; risk difference, 2.7 events per 100,000 persons; 95% CI, 1.0 to 4.6), lymphadenopathy (risk ratio, 2.43; 95% CI, 2.05 to 2.78; risk difference, 78.4 events per 100,000 persons; 95% CI, 64.1 to 89.3), appendicitis (risk ratio, 1.40; 95% CI, 1.02 to 2.01; risk difference, 5.0 events per 100,000 persons; 95% CI, 0.3 to 9.9), and herpes zoster infection (risk ratio, 1.43; 95% CI, 1.20 to 1.73; risk difference, 15.8 events per 100,000 persons; 95% CI, 8.2 to 24.2).

One has to wonder how herpes or appendicitis might be causally related to the vaccine, by the way. It could be simply behavioral like more kissing and going out to restaurants more often, although for appendicitis some connection via a higher likelihood of general inflammatory reaction can't be excluded. (For herpes zoster at least, reactivation from lymph nodes has been posited.) The Israeli paper notes that the clinical trials have noted a trend for appendicitis too, but not statistically significant, perhaps due to undepowered sample to detect a small effect. So, it's somewhat clear that they tried do "double check" some such trends they noticed/suspected, useing their larger database, while excluding AEs they probably considered trivial or "boring".

5
  • Excellent answer.
    – bob1
    Oct 29 '21 at 9:11
  • Thanks, I see your point. My porpose is to find a research based risk rate for serious health issues due to vaccination. There are a lot of testimonies, and a lot of research, but I'm struggling to find a 1:X answer, based on actual medical data
    – Roy
    Oct 30 '21 at 5:07
  • @Roy that's a semantics question - what do you define as serious? Would the person across the street also think those are serious? The data for complications is presented in the paper you linked and the one from Fizz. For instance lymphadenopathy is normally considered non-serious, so you could exclude that, but a myocardial infarction would meet most peoples' criteria of serious. If you exclude lymphadenopathy from the data the "risk" drops significantly, if you also exclude parathesia, neutropenia and lyphopenia (other causes common) then you have almost nothing left.
    – bob1
    Nov 1 '21 at 20:10
  • @bob1 Thanks, for this matter I'll just use the researchers' own definitions. Some effects are mild, some more serious suspected effects are not even looked at.. So as a start I just want to understand the rates from the available data.
    – Roy
    Nov 2 '21 at 5:07
  • @bob1 Correcting the last comment: Thanks. I find the understanding of the rates that are available from this data is needed. Of course differentiating between different effects by severity, as well as checking about serious effects that were not measured here, is also helpful.
    – Roy
    Nov 2 '21 at 5:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.