Big Data, Plainly Spoken (aka Numbers Rule Your World)

Funny math out of LA

In my previous post, I walked through how vaccine effectiveness is being divined from real-world data, and why this currently popular method of analysis results in substantial overstating of VE.

Today, the LA county health department issued a press release that performed this flawed analysis by the book. I'm adding this supplement since the Public Health England study from which I obtained the data used them for a different calculation (also flawed but that's a different post).

The pivotal paragraphs are these (my italics and bolding):

As more people have gotten vaccinated, the proportion of total cases that are among those vaccinated has also increased. This is to be expected because as more people are vaccinated, the number of fully vaccinated people becoming infected will increase. In June, fully vaccinated people represented 20% of all cases diagnosed among L.A. County residents, while unvaccinated and partially vaccinated people accounted for 80% of cases.

Public Health estimates if the 52% of County residents that are fully vaccinated were not vaccinated, the amount of new cases would perhaps be double because everyone would instead have the same risk of infection as unvaccinated people do. While County numbers have been going up, they would be much higher if there weren’t as many people fully vaccinated.

What was done in this analysis?

It starts with cases happening in a given window (June here, corresponding to April 12-Jun 4 in the UK dataset). It then divides these cases into two groups, "fully vaccinated" and "others". Fully vaccinated typically means cases occurring at least 14 days after the second shot (or first J&J shot). We have no idea if they also remove mild cases from the fully vaccinated case count - other wording in the press release implies they did not.

Separately, they tallied the number of people who are fully vaccinated as of the end of the study period, and then called everyone else "unvaccinated". The Californian population is split roughly half-half at this time.

Using those values, they computed case rates for the "fully vaccinated" and "unvaccinated" (others). They then claimed that unvaccinated and vaccinated people are the same in all ways except their vaccination status, and therefore imputed the counterfactual case rate for vaccinated people (assuming that they had not been vaccinated) as equal to that of unvaccinated people.

With that methodology, they said the number of cases would have doubled. That implies that the vaccination cut the case rate by half, which is a real-world VE of 50%. So, buried in this press release is a huge backtrack from earlier claims of 90-100% effectiveness.

***

The reality may be worse because the methodology has clear flaws.

First, the at-risk population for the vaccinated group is over-estimated because of immortal time bias. The only people who could contribute cases to the June tally are those who got their second shot before the middle of May. Those who took their second shots in the last few weeks are added to the denominator but cannot possibly contribute cases to the numerator of the case rate, thereby under-estimating it.

Second, the number of cases attributed to "unvaccinated" is over-stated because this analysis counts (a) infected people who are in between shots (b) infected people who have taken two shots but not yet reached 14 days, all as "unvaccinated cases". In addition, the 2D+14 case-counting window cannot be applied to unvaccinated people since they didn't receive any shots.

Third, the at-risk population for the unvaccinated group is under-counted. Consider someone who got fully vaccinated on June 3 but tested positive on June 5 (someone of type b from the previous paragraph). This person is classified as "fully vaccinated" as of the end of the study period. However, the positive case is counted as "partially vaccinated" which then falls under "unvaccinated" because the infection happened prior to 2D+14 days. This is equivalent to giving the case to the unvaccinated group but counting the person in the vaccinated group, simultaneously upping the case rate for the unvaccinated and reducing the case rate for the vaccinated. The more vaccinations are given out during the study period, the larger is this bias.

The above biases arise from the methodology. The biggest issue in real world data is selection bias: people who decide to get vaccinated are different from people who decide not to. Recent studies no longer offer any analysis of selection biases. The earlier studies indicate that the unvaccinated group may skew more minorities, and people living in areas with higher case prevalence.

 

 

Real-world vaccine studies consistently overstate vaccine effectiveness

The recent surge in Covid-19 cases in Israel and the U.K. has seriously dented the conventional wisdom that vaccines, and only mRNA vaccines, spell the end of the pandemic. If you've been following my blogs on the real-world studies, you should not be surprised that vaccine effectiveness (VE) is not close to 90% claimed by those people. I outlined many reasons why these real-world estimates contain numerous pro-vaccine biases. In this post, I present estimates of the magnitude of several of these biases. (And they are huge.)

I was able to quantify these biases because Public Health England released some real-world data in a recent study (link), which I mentioned in this previous post (link). (Note that I am not replicating the PHE analysis, but using their data to perform the typical VE analysis.)

Here is the relevant table from the study's supplement:

The column labelled "Total" are reported cases of Covid-19 during the study period. I assume that no cases were excluded. Also of interest is the column labelled "Admitted". These are cases resulting in hospitalizations, which serve as a proxy for severe cases.

The table breaks up cases into two age groups (16-49 and 50 and above) and by variants. For my analysis, I combine the variants to get total counts of reported cases of Covid-19.

The following table contains the data used in my analysis.

In addition to the numbers lifted from the UK study, I also estimated the number of people who fall into each vaccination cohort. A vaccination cohort consists of people who received their shots on the same day. The OpenSafely.org project (link) provides estimates of the progress of vaccinations in England, broken up by age group. Starting with those statistics, I derived the number of people in each vaccination cohort. The population estimates are transformed to an index with the unvaccinated cohort set to 100. For example, for every 100 unvaccinated people 50 and older, there are 70 vaccinated people whose dates of vaccination mean they will fall into the (1D to 1D+20) group if they should fall sick.

***

Data analysis should be as simple as possible but not too simple.

The baseline, "first order" analysis is to count all cases that are reported after someone gets the first shot and divide that by the number of people who have taken one or more shots. For the unvaccinated group, divide the cases reported during the study period by the number of people who were unvaccinated.

For the 50+ age group in that UK study, this formula yields a vaccine effectiveness of 57%, which means the case rate of vaccinated people is 57% lower than that of unvaccinated people. This shows the vaccines are definitely effective, but one should not expect they will end most infections.

57% isn't the VE number you've read about. One of the reasons is the "case counting window". Some experts opined that vaccines are not supposed to help us until we reach 14 days after the second shot, and so all headline VE values are computed after nullifying cases outside the case-counting window. In the UK 50+ dataset, the number of cases in the vaccinated group plunged from 3,647 to 577. That's right, 85% of the cases got stripped from the numerator. This single act upped the VE from 57% to 93%, a value with which you're more familiar.

Nothing changes for the unvaccinated group. That's because the unvaccinated people do not get placebo shots, so that the 2D+14 time point loses meaning. The case-counting window is thus asymmetrically applied to remove cases from the vaccinated group but not the unvaccinated group. This remarkable difference between real-world studies and clinical trials has not been addressed.

***

The least complicated way to correct this pro-vaccine bias is to cancel the case-counting window. Doing so results in the 57% vaccine effectiveness calculation which I mentioned already.

A more complicated (and not superior) method is to shrink the denominator in computing the case rate for vaccinated. We thus subtract cases from the numerator as well as people from the denominator. This adjustment changes the 2D+14 VE from 93% to 74%.

The drastic change results from removing 75% of the at-risk population. It turns out that three out of four vaccinated people are not really at risk of getting infected. Let me explain this counter-intuitive observation. It goes back to the case counting window, which introduces a type of "immortal time bias". Any vaccinated person is immortal after taking the first shot until 14 days after taking the second shot. If this person catches Covid-19 during this time, the analyst removes the case when computing the case rate for vaccinated. In effect, this person is immortal until 2D+14. Bias arises because the same counting rule cannot be applied to the unvaccinated group.

The UK dataset contains only cases between 4/12 and 6/4/2021. Anyone who took a first shot after 3/4/2021 arrives at 2D+14 after June 4th and so is immortal in the UK analysis. By my estimation, only a quarter of the vaccinated English people aged 50 and over can contribute to the case count in the 2D+14 case counting window.

The case rate for any group of people should be the number of reported cases divided by the number of people who can become sick. Someone who hasn't reached the start of the case-counting window has zero chance of becoming a case so a principled way to remedy this problem is to exclude the "immortals". (I, however, prefer the simple approach of canceling the case-counting window.)

***

Let's turn attention to the at-risk population for the unvaccinated group. In computing the case rate for the unvaccinated, analysts often use the number of people who are still unvaccinated by the end of the study period. This calculation underestimates who can contribute to the case count, artificially increasing the case rate for the unvaccinated group.

Some of the cases attributed to the unvaccinated group came from vaccinated people before they got inoculated. For example, someone was infected in early May and received the vaccine in June. This case is counted as unvaccinated because the date of infection precedes the date of vaccination but at the end of the study period, this person is vaccinated.

To deal with this bias, I added 37% more people to the denominator of the case rate for the unvaccinated in the 50+ age group. This shifts the VE from 74% down to 64%.

***

For the 16-49 age group, the baseline, simple VE counting all cases is 82%. The headline VE using 2D+14 case window without debiasing is 98%. This number drops to 69% with the aforementioned adjustments to the at-risk populations.

(Remember the VE number is a difference expressed as a ratio. An increase from 10 to 15 and an increase from 100K to 150K are both 50% increases.)

***

In the U.S., the CDC decided to discount mild and asymptomatic cases from the "fully vaccinated" population. Mild cases occurring at any time are nullified for the vaccinated group but not for the unvaccinated group.

The UK dataset contains data on hospitalizations so I can estimate the impact of this change in definition - on top of the case counting window.

The change pushes the 2D+14 VE from 93% to effectively 100%. It also elevates the debiased 2D+14 estimate from 64% to 99% for the 50-plus age group. That's because the number of cases counted for the vaccinated group falls from 577 to 12, a drop of 98%, while those for the unvaccinated group did not change.

***

To summarize, the data from the UK study offer a glimpse at how sensitive the real-world vaccine effectiveness metric is to certain case-counting rules. Specifically, the case counting window and the change in definition of "breakthrough infection" both, each by itself, cause VE to jump above 90%.

Those are by no means the only biases embedded in the real-world VE formula (we have ignored selection bias for example). A realistic estimate of VE is probably closer to 60%, which is an excellent number.

***

P.S. For those interested in the arithmetic, a key concept in this analysis is the vaccination cohort - people who reeceived the first shots on the same day.

For example, someone who got the first jab on 1/1/2021 on average received the second shot 80 days later, which is on 53/22/2021. If this person gets infected prior to 1/22/2021, the case is counted in the 1D to 1D+20 group. However, this case will not matter to the UK study because only cases reported between April 12 and June 4 are included.

If the case shows up between 1/22/2001 and 4/5/2021, this infection is counted in the 1D+14 to 2D+13 group but it happened before the start of the study window. If someone vaccinated on 1/1/2021 becomes sick after 4/12/2021, this case falls within the study period, as April 12th is the 21st day after this person's second shot.

 

Keeping up with the two books at CDC

In May, the CDC started to practice a form of "off the books accounting" (Here's a page describing creative accounting principles). In the U.S., we now have two sets of books when it comes to counting Covid-19 cases.

On May 1, the CDC introduced a new definition for "breakthrough cases." Breakthrough cases are cases that are found among the vaccinated people. Prior to May 1, all PCR-positive cases, including mild and asymptomatic cases, were counted as breakthrough if they occurred at least 14 days after the single J&J shot or the second mRNA shot. After May 1, only PCR-positive cases that result in hospitalizations and deaths within that case-counting window are counted as "breakthrough cases".

Because of the revised definition, the U.S. now has two tallies of cases among the vaccinated: one count includes mild and asymptomatic cases while the other count excludes them. (No such distinction is made for unvaccinated Americans.)

So if you ask how many Covid-19 cases there are in the U.S., there are two possible answers. The reality is even messier because each state may be reporting one or both counts to the national database.

That statement from the CDC implies that one of the two counts (the one including mild and asymptomatic cases for vaccinated Americans) is incomplete.

***

Consider media statements such as "95% of Covid-19 cases in the U.S. are now in the unvaccinated."

This statement is meaningless unless you're told which set of books was consulted. Were mild and asympotomatic cases in fully vaccinated Americans included in the total, or only CDC-approved breakthrough cases? (A separate but still pertinent issue is whether sufficient fully vaccinated people are tested to measure mild and asymptomatic infections.)

It appears likely that a data analyst computes the number of cases in the unvaccinated population by subtracting the number of breakthrough cases from the total number of reported cases. The total number of reported cases is typically the number of positive PCR test results, and therefore it should include all cases, including mild and asymptomatic, and regardless of vaccination status.

Now, pay attention to the following types of cases:
a) Mild and asymptomatic cases among the vaccinated that were reported at least 2 weeks after the last prescribed vaccine shot
b) Mild and asymptomatic cases among the vaccinated that were reported prior to 2 weeks after the last shot
c) Hospitalizations and death among the vaccinated that were reported prior to 2 weeks after the last shot.

CDC does not qualify these cases as "breakthrough cases" but all of these produced PCR-positive test results. The first statement means these cases are not counted as vaccinated cases while the second statement says they are counted as unvaccinated cases - if the analyst subtracts the breakthrough cases from the total cases.

Imagine that! Someone who is hospitalized two days after receiving the second Pfizer shot is counted as an unvaccinated case in this math. Someone who has mild disease a month after receiving the second Pfizer shot is also counted as an unvaccinated case.

What happens if the data analyst recognizes this problem, and wants to fix the math? You'd have to create a third bucket to hold all those cases - which are neither cases in the fully vaccinated nor cases in the unvaccinated. In the business world, this is known as "off the books accounting". You simply disappear these cases into the nether world.

In a follow-up post, I will estimate the scale of this off-the-books accounting using real-world data. Stay tuned.

Every headline produced by real world studies has been retracted and rewritten

Back in March, I reviewed several early "real-world studies" of vaccine effectiveness (here, here, here, here), including the Dagan et. al. study from Israel that eventually became one of the most frequently cited studies on the unreasonable effectiveness of mRNA vaccines.

The researchers of these early studies acknowledged the difference between observational datasets, and data coming from randomized clinical trials; specifically, the problem of selection bias inherent in the former but obviated by the design of the latter. Matching methods were deployed to reduce the effect of selection before generating a vaccine effectiveness number that mirrors vaccine efficacy in an RCT.

I also later reviewed several studies (Denmark, CDC, CDC 2) that utilized regression adjustments. Regression adjustment is an exercise in wishful thinking: for example, an age-adjusted estimate of vaccine effectiveness is the value of VE after removing the effect of age. Mathematically, the regression adjustment imposes a common age distribution on the vaccinated and unvaccinated subgroups, so age becomes a non-factor.

However, the age-adjusted regression estimate is relevant only if
(a) the estimate of VE for any given age range is reliable, and
(b) the specific age distribution imposed on both subgroups is realistic.

In the present situation, it is hard to believe either of those assumptions. For vaccination campaigns everywhere prioritize high-risk subgroups, particularly older people with or without comorbidities.

The headlines coming out of those early real-world studies were loud and clear: real-world evidence confirmed that the vaccines performed as well as and maybe even better than the clinical trials indicated. The media highlighted VE estimates of over 90% on all reported cases of Covid-19.

As a reminder, this was a CNN headline from February 2021.

The first sentence of the article cites the 90% number: "Pfizer-BioNTech's Covid-19 vaccine appears to reduce symptomatic coronavirus infections by more than 90% in the real world, Israeli researchers said Sunday."

If you want the other view, Fox News said the same thing.

***

By end of May, amidst reports of breakthrough infections, the story started to evolve. The new headlines now focused on VE estimates of close to 100% on reported severe cases, hospitalizations, or deaths. Readers were admonished for ever believing that vaccines should prevent disease, even though the primary endpoints for all clinical trials and early real-world studies were prevention of symptomatic, PCR-positive Covid-19 cases in the selected case-counting window.

Since June, the U.K., despite having one of the world's fastest vaccination programmes, began to suffer another surge in cases. The new headlines now drew attention to hospitalizations rather than cases.

The report cites a Public Health England study, saying "although the Delta variant reduces the effectiveness of vaccines against symptomatic infection, two doses of COVID-19 vaccine still protect against severe disease."

Israel, the early star in vaccinations, faced a similar scenario, and their reporters followed the same path, delivering headlines such as this:

It's not about infections anymore; only hospitalizations matter.

It's not just the headlines that changed. This new batch of studies employs different methods of analysis. Matching and regression adjustments were both sent to the scrap pile. The new analyses now wantonly ignore selection and other kinds of biases, and treat the observational data as if they were collected from randomized clinical trials. They assume the vaccinated and unvaccinated groups are balanced on all variables, an assumption not remotely plausible.

Here is an example of such a calculation, lifted from Matan Holzer's twitter account (see title of bottom chart):

In these type of studies, vaccine "efficiency" for hospitalizations is based on the ratio of the odds of hospitalizations for vaccinated vs unvaccinated to the odds of vaccination. The same ratio is obtained by dividing the hospitalization rate of the vaccinated by the hospitalization rate of the unvaccinated. This is the analysis that would have been performed to evaluate VE in a clinical trial - although those trials did not contain enough participants to attain statistical significance on the hospitalization metric.

Such analyses gave rise to headlines that proclaim that only unvaccinated people are sick enough to go to hospitals.

What are the data supporting the above headline (link)? It's the share of hospitalizations that are attributed to the unvaccinated people.

In the meantime, the following chart from BBC shows how the new wave of infections is  growing even faster than in the last surge, which happened before vaccinations.

***

By mid July, hospitals in the U.K. and Israel are admitting a rising number of fully vaccinated patients. So, the story is shifting again.

The previous metric comparing vaccinated and unvaccinated groups as if they were randomly assigned is abandoned. Experts now pretend that they did not introduce this analysis to the public, and criticize those looking at this number as a probability illiterate - failing to grasp Bayes' Rule. (here) Nate Silver chimed in:

I agree that the efficiency metric is poorly conceived, a fact that was as obvious in May as it is in July. I don't like the "story-first" mentality adopted by many scientists, which led to all those headlines about almost no vaccinated people getting hospitalized.

As the hospitalization story falls apart, the media is lining up behind the VE metric on deaths. We are being told that the vaccines have broken the link between infections and deaths, which reminds me of when they said the vaccines severed the link between infections and hospitalizations.

The math has been clear on this from the start: hospitalizations lag cases, and deaths lag both. We are going to find out in the coming weeks whether this latest narrative sticks

We need a post-mortem on the real-world studies, a honest evaluation on why these findings have low predictive value.

 

 

P.S. [7/23/2021] The same story is unfolding in Israel. Weeks ago, they happily cited naive estimates of vaccine effectiveness - first on infections, later on severe disease. Now that those metrics don't show what they used to show, suddenly those metrics are worthless. This Times of Israel article is highly informative.

Real-world vaccine studies through the lens of vaccine trials

Real-world studies are hard because observational data are packed with known and unknown biases. One thing is for sure: any analysis that doesn't correct for known biases is clearly flawed.

When discussing real-world studies of Covid-19 vaccine effectiveness, I have often lamented the ill effect of "case-counting windows". Recall that none of the vaccine studies count all cases from the day the participants enter the trial and get their first shot. In the case of Pfizer, all Covid-19 cases that occurred prior to the 7th day after the second dose were removed, in effect, nullified.

This procedure is feasible in the vaccine trial, when the same case-counting window can be applied to both vaccine and placebo arms. In real-world studies, the unvaccinated people do not take placebo (saline) shots, and therefore, it is impossible to locate 7 days after the second shot. As a result, the case-counting window removes cases only from the vaccinated group and not from the unvaccinated group.

tldr; Scroll to the end of the post

***

I'll show you the effect visually. We start with the cumulative case curve that was displayed in the Pfizer FDA Briefing document.

This chart is straightforward. As time passes, the number of cases grew on both vaccine and placebo arms. At the time of the interim analysis (which led to the emergency use authorization), the cumulative case rate for the vaccine arm was 0.25%, or 25 out of 10,000 was found to be sick while that for the placebo arm was 1.4% or 140 out of 10,000. Because of the trial setup, we can assume all else being equal, and so the vaccine is said to have cut the case rate by just over 80%. The vaccine efficacy is 80%.

But Pfizer said VE was 95%, not 80%. This is because of the case counting window. All cases on either arm that occurred prior to 2D+7 days are nullified. Like this:

This case-counting strategy knocks down the case rates for both placebo and vaccine arms. The placebo group now has a cumulative case rate of 0.8% at interim analysis, while the vaccine group has a cumulative case rate of 0.04%. The vaccine is now said to have cut the case rate by 95%. (But this number is not comparable to the 80% number above. The 95% efficacy applies only to people who did not get sick prior to 2D+7 while the 80% number applies to everyone in the vaccine trial.)

What happened is purely arithmetic. The second calculation is based on a subset of the data used in the first calculation.

***

Next, we do a thought experiment. We pretend that the data came from a real-world study instead of a clinical trial. We will therefore relabel the placebo group as "unvaccinated" and the vaccine arm as "vaccinated". We will now compute a vaccine effectiveness using the methods from real-world vaccine studies.

Here is the picture:

The cumulative case rate for the unvaccinated group is 1.4% while that for the vaccinated group is 0.04%. Thus, the vaccine apparently cut the case rate by 97% - almost perfect protection, even better than what was seen in the vaccine trial.

What just happened is that I took the orange line (for unvaccinated) from the first chart and the blue line (for vaccinated) from the second chart. The blue line is uncontroversial: given the dates of vaccinations of the vaccinated people, the analyst can compute how many days after the first shot someone got sick with Covid-19, and therefore apply the case-counting window, removing all cases occurring prior to 2D+7. This mimics the formula used in the vaccine trial.

When it comes to the unvaccinated group, the analyst could not apply a case-counting window. That's because the unvaccinated group did not take any injections - unlike the placebo participants in the vaccine trial who took saline shots. As a result, the analyst just counts all the cases in the unvaccinated group during the entire study period. This case rate is then compared to the case rate of the vaccinated - but restricted to various case-counting windows.

The beauty of this thought experiment is that it allows us to estimate how much pro-vaccine bias has been introduced by this case-counting window. On the second chart, we know that if case-counting could be applied to the orange line, the case rate would have been 0.8%. On the third chart, we know that the case rate was treated as 1.4% when case-counting became infeasible. Thus, the baseline case rate is inflated by 70% due to this asymmetric application of case counting.

Without a doubt, these real-world studies produce vastly inflated estimates of the true vaccine efficacy. (And here, I have only covered one of many biases in these studies.)

 

TLDR;

Real-world data are messy because unlike a clinical trial, there is no design to constrain biases. Real-world studies are evalulated on whether they have sufficiently corrected for known biases. One useful trick is to run the real-world study methodology through an unbiased dataset (e.g. data coming from a clinical trial or randomized experiment.) One hopes that the RW methodology yields the same answer as when one analyzes the clinical trial using techniques for analyzing trials.

The analysis here shows that the most popular real-world study methodology over-estimates the effectiveness of vaccines. The reason is that the case-counting window is applied symmetrically in the trial analysis but only to the vaccinated subgroup in a real-world study. This asymmetry causes the baseline case rate to be inflated by 70% (in the case of Pfizer), and vaccine effectiveness is the relative improvement over that baseline.

The case-counting window for Pfizer starts at 7 days after the second dose, meaning that all reported infections happening before that day are nullified when analyzing the clinical trial. In the real-world studies, these earlier infections are nullified only for the vaccinated group but not for the control group. This is a pro-vaccine bias built into real-world data. The only remedy is a correction performed at analysis. A first-order correction is to divide the unvaccinated case rate by a factor of 1.7 before computing vaccine effectiveness. (This correction however does not adjust for many other biases found in real-world data.)

 

Story time: vaccinated people cannot spread Covid19

If you ask me for advice, I say get the vaccine, but do not let your guard down afterwards. Sure, go out more, and re-connect with friends but continue to wear masks and maintain distance in crowded places.

But some of you say, the CDC tells us if one is fully vaccinated, one can treat the pandemic as over. Throw all (pre)caution to the wind.

Let's dissect that recommendation. If you are fully vaccinated, you can still get Covid-19 but they claim you will not get severely sick. They encourage you to return to pre-pandemic living. So they must believe that fully vaccinated people with mild disease cannot spread the virus. They must also believe that fully vaccinated people with asymptomatic infections cannot spread the virus either.

These are important points because half the U.S. population is now fully vaccinated, and if fully vaccinated cases can spread the virus, then potentially lots of people are spreading the virus, invisibly. While we don't get severely ill, the more sick people roam around, the more the virus replicates, and the higher the chance that a rare mutation surfaces that causes another lethal wave of Covid-19.

Starting around April, self-described experts began making claims that "fully vaccinated people cannot spread Covid-19". For example, in this Johns Hopkins piece, an expert said "I think the preponderance of the evidence supports the fact that vaccinated individuals are not able to spread the virus."

***

A month or two ago, I gave a talk to University of Denver students, with the key message being "don't judge a research study by its abstract; learn what research methodology it used to draw its conclusions." See this post for more.

At this stage of the pandemic, three research methods dominate Covid-19 research: clinical trials, real-world studies, and lab experiments.

If we want to prove that vaccinated individuals are unable to spread the virus, what kind of study do we need?

We can immediately rule out the lab experiment. Testing blood samples for antibodies will never yield data on clinical outcomes such as symptoms, infections, transmission or spread.

Spread is a very hard outcome to assess. You need to first identify infected people, and then do contact tracing to figure out the transmission chains.

Can clinical trials yield such data? Absolute not, if we use the vaccine trials as they were designed. Those trials only tested participants for infection once every month or two, and most cases were self-reported, meaning that they only identified a fraction of the infections. Moreover, no contact tracing was attempted so we have no idea who transmitted the virus to whom. Therefore, these trials yielded no data about spread.

What about real-world studies? While we have big sample sizes for these studies, they also contain nothing about transmission chains. The aforementioned Johns Hopkins article cites this CDC study. I fail to see how the data proved that vaccinated people cannot spread the virus. The paper shows that vaccinated people are less likely to be infected (on this, I have concerns about inadequate debiasing of the data), and then they stretch that result to claim that the vaccine stops transmission.

Nevertheless, the study made no attempt to figure out who transmitted the virus to whom. For example, how do we know that one of the vaccinated and Covid-positive participants did not infect a dozen unvaccinated co-workers? I don't know the answer, nor do I think that is the most likely scenario but I fail to find any evidence supporting the claim that vaccinated individuals do not spread Covid19 in that study.

So what we have is "story time". Readers of the study are lulled to sleep by the nice dataset that was collected to measure overall vaccine effectiveness, and as they doze off, the researchers feed them the story of preventing spread, which has little if anything to do with the data.

 

P.S. [7/14/2021] In addition to story time, we have contradictionary evidence that gets ignored or dismissed. For example, the Las Vegas Review Journal just reported that 10 vaccinated health-care workers got Covid-19, possibly all during a pool party. Eight of these are fully vaccinated. Only 1 unvaccinated attendee got sick. So 10 out of 11 cases from that party were vaccinated. 

They only know about these cases because the health care workers decided to get tested. There are probably more cases because many attendees may not have gotten tested. Note, however, that fully vaccinated people are more likely to ignore mild symptoms and not get tested.

If fully vaccinated, PCR-positive people cannot spread Covid19, each of these 10 healthcare workers were infected by an unvaccinated, infectious person or persons. Even in this small example, we don't know who spread the virus to whom.

Regrettably, the Review-Journal printed a fallacy that I have pointed out from day the first clinical trial result was published. It said "In laboratory tests, the Pfizer and Moderna vaccines were about 95 percent effective in preventing symptomatic cases of COVID-19. That means that about 1 in 20 people could become sick after being vaccinated, though their infections would be less likely to be severe." In these trials, the case rate in the placebo group - people who got saline shots - was only around 1 percent. If their interpretation were correct, then the vaccine bumped the chance of getting infected from 1 percent to 5 percent - 5 times worse! See this post for how to interpret vaccine efficacy.

Forced consent because they can

Professors write recommendation letters for their former students. We do so voluntarily and without compensation. We also do so with few complaints - even when the student contacts you days before the application deadline.

If I am writing a serious recommendation letter - one that, if the admissions officers read it, helps the student, rather than a generic one that reads like every other, it takes hours of time. Any long-serving professor has had thousands of former students. Remembering who the reference requester is is the first challenge. Understanding what program the student is applying to, and what s/he intends to do with the degree is the next challenge.

Recently, I was getting ready to upload a letter I wrote for a former student. I clicked on the link sent to me by the school, and landed on a third-party website. This company has decided to turn reference letters into a data business and I am asked to accept the following conditions:

This is the typical forced-consent tactic we're all familiar with. It's the type of notice you get from Facebook or Gmail - which they justify by offering you a "free" service. That was eons ago. Forced consent is now being imposed by companies that charge you to use their services: Amazon, Uber, AirBnB, etc. are not free services, and yet they are monetizing our personal data, regardless. Companies that turn off the ads if you subscribe do not turn off the data collection, either.

Which brings me back to recommendation writing. I'm offering free help to a former student, and for doing that, I am forced to become the subject of data collection. I give them credit for being upfront about this - not just linking to an inscrutable Privacy Policy but also highlighting some of the terms I'm agreeing to. They directly admit that my personal information will be harvested and stored. They say they will delete or anonymize the data after they deem it unnecessary, except as stated, the data can be considered necessary indefinitely.

They explicitly disclose that they will scrape data about me from other sources (there is no such thing as "receiving" data, as data don't randomly present themselves to one's door) and merge the datasets. This might happen if they place a cookie (or cookie-like object) on my browser when I upload the reference letter. If they then collaborate with other websites (think Facebook Like button, which is everywhere), then they can track my web browsing thereafter.