Recently, some researchers changed the way they describe vaccine effectiveness. It used to be the Pfizer vaccine is 90% effective at preventing (first it was cases, then it was severe cases, finally it was) deaths. Now, they are saying the vaccine cuts (cases, severe cases, deaths) by 11 times. (See, for example, this recent report on the booster shots in Israel.)

Cutting some bad metric down by 11 times sounds extremely impressive. Is 11 times better than 90%?

The answer: they are saying exactly the same thing. If the rate for unvaccinated is 11x, and that for vaccinated is x, then the vaccine effectiveness is 1 - (x/11x) = 91%.

Cutting a number by 11 times is great but hard to achieve. Now, if we have a vaccine that cuts infection rate in half, that is still quite good, right?

That vaccine would have a VE of 50%. CDC doesn't even want to talk about such a vaccine.

***

I will say it again. A 90% vaccine effectiveness does not mean that 9 out of 10 people who got vaccinated are protected. As explained here, VE is a ratio of two infection rates. Both of these infection rates are very small, typically below 10%, and sometimes below 1%, depending on which clinical trial the data came from. In other words, over 90% of the people in the placebo arm did not get sick during the study period. It's logically impossible for 90 out of 100 people to be protected when fewer than 10 unvaccinated people would have gotten sick.

Statements such as "99% of vaccinated people are not hospitalized with Covid-19" are not very informative, unless we are also told what proportion of unvaccinated people are not hospitalized. If we have both, and flip the focus from not hospitalized to hospitalized, then we are looking at a ratio of rates, which is the vaccine effectiveness measure.

Those alternative descriptions are distracting and harmful when misinterpreted.

***

One last thing. A 90% effectiveness against cases and a 90% effectiveness against deaths are not at all similar things.

That's because the rate of death is a fraction of the rate of cases. When talking about cases, a 90% VE may mean pushing case rates from 10% to 1%. When talking about deaths, a 90% VE may mean pushing death rates from 0.5% to 0.05%. It is ludicrous to say that 99.95% of vaccinated people are protected from deaths when 99.5% of unvaccinated people would not have died.

A big drop in death rates is impressive on its own. Less effective is not the same as not effective. I don't like these alternative descriptions of effectiveness because the math is incorrect.

You're arguing here for using absolute risk, rather than risk ratio, which in many cases is a better guide. It isn't for vaccines because the risk is quite variable, depending on country. Risk of having had covid ranges from less than 1 in 10,000 up to almost 1. For a country where everyone has covid the absolute risk reduction is 60-80% from being vaccinated. Without vaccination everyone would have covid.

Posted by: Ken | 10/29/2021 at 12:06 AM

Ken: Nothing in this post re-defines VE though. I'm just using the definition as used in the vaccine trials.

You raise two interesting issues about VE definitions.

1. "Without vaccination everyone would have covid"

This would imply we don't need a control/placebo group. But the statement is true only if we let time -> infinity, and not true within any study period.

Also think about VE for deaths. The analogy would be "without vaccination everyone would die from covid"

2. Relative risk ratio and absolute risk (difference) are both useful metrics, depending on the situation. With vaccines, the community has decided on risk ratios. With ratios, we are assuming that if the vaccine cuts infection by 50% on a base rate of 10% in one place, then the vaccine would cut infection rate from a base rate of 1% to 0.5% in another place. Yes, this allows us to apply the result to any regions and it's easy to forget that VE being the same everywhere regardless of base rate is an assumption. I don't think this assumption is obviously true.

Posted by: Kaiser | 10/29/2021 at 03:19 PM