Big Data, Plainly Spoken (aka Numbers Rule Your World)

Lessons from the Tiktok saga

In a few days, we’ll find out if Tiktok is really going to be pounced out of the U.S. market. [This was written in early January. Answer appears to still be yes when I pushed the publish button on this post as the app was removed from the app stores.] In the meantime, the Tiktok saga has already revealed many things about the data collection industry. (In this post, I shall assume that the allegations by the U.S. government hold up even though there aren’t any actual evidence of wrongdoing.)

What technologies can do > what technologies are doing

With the legislative action against Tiktok, the lawmakers are applying a doctrine of preemptive strike. I’ve always believed in early action to mitigate potential harms from new technologies. There are endless examples of such predictable harms. Just to name a few:

• Self-checkout tech will induce more stealing
• Tolling via reading license plates will lead to more nonpayment and evasion
• Car telematics will raise people’s insurance rates, and the number of parking and speeding tickets issued
• Deep fakes using deep learning technologies will supercharge all kinds of scams

Big Tech has always opposed early action. I don’t know how many times I’ve been told off on these worries. All technology can be misappropriated, Big Tech supporters say, and so the possibility of harm cannot justify restricting any technology. A knife can be used to kill, would you ban knives? Meanwhile, let's focus on the great utility knives bring.

Not banning something, however, does not mean turning a blind eye. (And that is what the industry has been doing regarding deep fakes. Just look at how even the term “deep fake” is fading away according to Google Trends.)

In this chart, blue is for “deepfake” while red is for “deep fake”. Both lines are moving downwards.

By now, we know:

• many retailers have had to dial back on self checkout (link)
• within one week of pushing out congestion pricing in Manhattan, NYPD had to take action on toll evasion (link)
• Texas just sued AllState for compiling telematics data and then raising premiums, dropping coverage, etc. (link)
• Let’s see who’s right on deep-fake scams!

The basis of this Tiktok action is the possibility that the collected data would be exploited for nefarious use. If we apply the Big Tech viewpoint, this possibility of misuse is of no concern. Nevertheless, Big Tech is silent about this legislative action against Tiktok.

What I learned from the Tiktok saga is now we’re supposed to worry about how technologies could be put to nefarious use before someone uses them nefariously!

 

Social media apps collect highly sensitive data that can be damaging to users.

The government alleges that the data collected by Tiktok could be damaging to its users. By extension, the data collected by any social media app, indeed any app, could harm people.

Since all apps available on major phone brands have to follow guidelines set by Apple’s iPhone or Google’s Android, it’s hard to imagine that Tiktok engages in data collection that goes beyond the capabilities of other social media apps, indeed any mobile app. If you’ve ever opened one of those terms and conditions, you’ve probably been advised that there are hundreds or thousands of data collectors on every website. These are businesses whose raison d’etre is monetizing user data.

Just for giggles, I saved this popup from a UK-based dictionary website (typically, U.S. sites don't even show these popups):

Let's review this quickly, shall we? "We care about your privacy" [therefore] "We and our 777 partners store and access personal data..." Further down, "[We and our 777 partners] use precise geolocation data." Pray tell why a dictionary website needs to know the user's precise location. I commend them for being honest, and speaking in plain English; and if you're alarmed, don't be mad at Collins, they are not the world champion of data collection by any measure. I picked on them just because I was on their site recently.

To break this down further, there are data elements that all apps collect, and then, there are data elements that are proprietary to individual app developers.

If you’ve ever seen a pop-up allowing you to “customize” the amount of tracking on a website, you’ll have heard of such terms as “legitimate use.” Most of the data that have been alleged to be massive invasion of Americans’ privacy – such as location, device ID, device type, IP address, battery usage, etc. – are routinely collected by any IP device. They are a form of “data exhaust”: logging these data elements have been standard even before the Web got commercialized. Everyone classifies such data as “legitimate use” (e.g. necessary for the normal functioning of the app).

Any data element can be “legitimized” under the above framework. If we were to exclude location data from normal logging, then some developer will add a feature on their app that requires location data, thus legitimizing the data element! For example, weather apps are big business because most users are looking for local weather, which requires them to disclose their current location. For weather apps to function, users must authorize the collection of location data, and the app developers to sell them.

Proprietary data are also collected by all apps. If only the app owner (but not others) can see such data, then it must concern something users do inside the app. Some examples might be how much time a user spends on Tiktok, draft versions of videos that never went public, and what kinds of editing occurred on a video. It’s actually hard to come up with many data elements in this category. Let’s take the social network of a particular user, i.e. which other Tiktok user is this person connected to. In most social-media apps, this is public information (by design) so it’s not really proprietary to Tiktok. Further, the businesses that make money from data collection frequently pays app developers to install their scripts inside their apps, in which case even “proprietary” data have been made available to third parties, putting them in the other category.

Location data is sensitive, as it allows bad people to find us. But the idea that banning Tiktok will stop a foreign government from getting location data on Americans is ludicrous, when there are thousands of data collection scripts on every phone. If we don’t want our location data out there, we can stop using apps that require location data. (Even that might not be effective – see point #1 of this post.)

The lesson of this Tiktok saga is that the government has reason to believe that Tiktok collects sensitive, potentially harmful data from its users, an allegation which generalizes to all apps.

 

Our data can be dangerous even if we’ve done nothing wrong.

One of Big Tech’s favorite slogans about data collection is "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place." Eric Schmidt famously made that comment in 2009 when he was CEO of Google, at the very beginning of this data collection tidal wave. The implication is that if a driver was speeding on the highway, and Google detected it (e.g. via tracing location data on an Android phone), and sold the data to a third party on contract with the police to issue speeding tickets, then it’s the driver’s fault for producing the data in the first place.

The Tiktok saga wholly usurps such logic because the legislators are claiming that most people, who have done nothing wrong, could become victims if their data fall into the wrong hands!

 

Social-media apps provide the platforms for indoctrination.

The U.S. government alleges that foreign states can exploit Tiktok to spread propaganda. But the real lesson here is the social-media platforms plant ideas into their audiences via exposure and filtering, which makes them not different from traditional mainstream media. One difference is that traditional media are broadcast networks so the same messages are sent to everyone while on social media, different messages (even opposite messages) can be sent to different people, without individuals realizing they got different messages from others.

It beggars belief that these foreign states can’t figure out how to use Facebook, Instagram, Twitter/X, Youtube, etc. to influence those audiences. It’s possible – although unproven – that these U.S.-originated platforms do the government’s bidding in filtering speech, and therefore may be ineffective at spreading opinions unfriendly to said government.

This feature of social media has long been known and exploited - by the marketing and advertising industries. The entire social-media influencer industry is built around the idea that these platforms efficiently deliver marketing messages, including many spams and scams, to users. Savvy politicians have also mastered using social media to “mobilize” constituents; some governments have also censored social media. This Tiktok action confirms the efficacy of social media.

Know your data 41: collateral damage of data exhaust

In a previous post, I referred to a report that revealed Manhattan's congestion pricing program faced a toll evasion crisis right after launch - the non-compliant drivers didn't wait even one week.

What happened after the first week? New York Post (link) disclosed that law enforcement has already started issuing tickets. Toll evasion must have been quite rampant!

The journalist revealed that NYPD wrote over 100 tickets during a six-hour crackdown on toll evasion.

***

That's not the point of interest for this post. To me, the most informative line is this one:

In addition to the summonses, 10 vehicles were seized during the operation — eight of which had suspended registrations, and two that were impounded for other reasons. One individual was arrested at the scene, though authorities clarified the bust was for a warrant unrelated to congestion pricing.

I call this the collateral damage of data exhaust.

Because of this congestion pricing scheme, every single car entering lower Manhattan is identified. They have installed 1,400 cameras. These cameras can identify also every driver and passenger in every car - although I haven't heard whether that is being done.

This situation on the roads is wild wild West compared to what we face on the Web or on our phones. At least there we have some laws, albeit limited, concerning PII (Personally Identificable Information). This whole congestion pricing setup clearly involves PII but there doesn't seem to be any protection against abuse.

What those sentences tell us is that the PII is not just used for the tolling application for which data are being collected; the PII is then taken elsewhere, used to look people up in databases unrelated to congestion pricing, which leads to arrests and seizures of property. In the world of data science, such appropriation of data collected for one purpose to another is confusingly called "data exhaust".

In effect, every single camera is a cop who can identify someone without even asking for ID, and who has 24/7 access to all kinds of databases.

Know your data 40: omnipresent listening

Apple markets itself as the tech giant that is most protective of user privacy. One reason to believe this claim is that unlike all the other tech companies, Apple charges premium prices for its products and is not as reliant on advertising as its revenue source. Nothing in this post disputes this assertion although we will wonder how low the bar is to quality as "most".

Apple recently settled out of court a lawsuit that alleges that Apple's Siri secretly records people's conversations, then extracts "insights" which are sold to advertising customers. It's a scenario that most of us recognize is possible, and perhaps many of us thought couldn't possibly be happening, not on an Apple device.

The allegation is that some users were delivered "eerily accurate targeted ads that appeared after they had just been talking about specific items like Air Jordans or brands like Olive Garden".

***

Of course, any interactive technology requires the tech to capture the user's instructions or reactions and thus we expect the tech to be recording, whether it's audio or video. This is true of Siri, other voice assistants, any home devices you can interact with, gaming and TV monitors, etc.

So, if one chooses to use these features, one should expect omnipresent listening.

Just think about the idea that "Hey, Siri" is used to activate Siri. If the phone is not constantly listening to you, how does it pick up "Hey, Siri"?

The concern is really about the lack of user control, or the disrespect of user intention (what has been euphemistically called "unintentional" here). I laugh every time the word "unintentional" is used. There is almost nothing that is unintentional in tech. The recording has to be triggered by some instruction and that instruction is written up in the code, a deliberate act. It's possible that the recording is set to be universal regardless of user intention or controls but that's not what is being admitted.

The allegation is that Siri was supposed to record conversations only if the user utters the words "Hey Siri" but in fact, conversations were (sometimes) recorded even when those words were not spoken.

Further, one doesn't typically hand over entire recordings to advertisers. The recordings must be turned into transcripts, then insights are extracted (e.g. person A mentioned brand X), the person A has to be identified so that the advertiser can send ads to that person's devices, etc.

The amount of code needed to execute the entire targeted advertising program, starting with recording the conversations and ending with delivering the ads, is substantial and involved. In my view, if such code exists, it is intentional.

***

It just so happens that while I was writing this up, a friend called me up and told me that his Instagram (that's a Facebook property) started flooding him with ads from a particular VPN brand after he recently asked me for VPN recommendations. He was quite alarmed and annoyed. He said he only wrote down the recommended names on a notepad so the only way Instagram knows we talked about that vendor would be through listening to our phone call.

That scenario is certainly possible but hard to prove since there is so little transparency when it comes to technology. In fact, any number of trackers may be collecting the data, which eventually find their way to Instagram.

Without whistleblowers or discovery, it's challenging to prove the case against Apple in court. But since Apple is settling, we might not hear about this again.

 

 

On the second day of congestion pricing...

Here comes the most predictable story of the AI revolution in toll collection. Drivers are evading tolls, stupid! (link)

Since Sunday, New York City has imposed tolls for driving into midtown and downtown Manhattan, which can potentially reduce the volume of traffic, and thus pollution in the city. They call it “congestion pricing,” which is a misnomer, since the price is not zero when there is zero congestion (the toll is $2.25 for passenger cars after 9 pm).

In any case, "congestion pricing" is powered by omnipresent surveillance, as AI image recognition technology is used to recognize car license plates, and then via a database lookup to establish their identities, the car owners are sent bills. You need lots of cameras all over the city to make this possible. According to this report (link), they installed 1,400 cameras in 110 “detection points”.

It used to be that cars lined up to pass by toll booths where each driver must hand over cash to the human toll collector before the car was allowed to pass through. This means drivers paid first, then used the roads. Then came AI image recognition technology, which snaps photos of cars, extracting the license plate numbers, and via a database lookup, sends bills to people’s homes. As a result, drivers get to use the toll roads first, and pay later.

It doesn’t take a genius to figure out that one can’t get charged if the AI failed to discover one’s license plate number. One can use any number of ways to obscure the license plate, or not even show it. One can fake the plate numbers, or simply cover it up. It appears that the toll evasion problem is so serious that on the second day of congestion pricing, NYC officials have proposed new rules to crack down on such tactics.

The next thing they will discover is more revenue leakage – what proportion of plate numbers are incorrectly read, what proportion of bills are sent to the wrong addresses, and what proportion of people don’t pay the bills.

***

On a related matter, my own experience is that banks are trusting the handwriting recognition software used by ATMs to read checks less and less; I feel like more and more, I'm asked to enter the amount manually, which suggests that the software is not sure about the number, or that the banks are extra cautious. 

Was Debrina Kawam homeless?

The New York Times published a short entry on the life of one Debrina Kawam, the woman burned alive while sleeping in a New York subway car recently (link).

In the initial reporting, the victim was described as "homeless" but eventually, when she was identified, the police used language that suggests otherwise: they said she had a "brief stint" in the New York homeless shelter system. This CNN article even said:

The victim was initially believed to have been homeless, which complicated efforts to identify her, law enforcement sources previously told CNN.

The NYT reporter added details, saying:

In the fall of last year, Kawam was homeless. After an outreach team encountered her at Grand Central Terminal, she entered the New York City shelter system and was assigned to a facility in the Bronx. But she never showed up.

***

Since I have been digging into how the U.S. HUD (Department of Housing and Urban Development) counts the homeless population (here, here), Kawan's case presents an opportunity to put our knowledge to practice.

If she had been alive in late January, she would have been found by the homeless counting team to have been sleeping in the subway system, which means she is part of the "unsheltered" homeless. If she had registered with the assigned Bronx shelter, she would have been counted as "sheltered homeless."

Thus, the reporting that suggests she was not homeless at the time of her death appears inaccurate.

 

P.S. Another tidbit from the NYT article was her futile attempt to find her mother in New Jersey. If her mother had allowed her to stay even temporarily, Kawam would not be included in the homeless count (neither in New York nor in NJ) as this situation fits one of the exclusion criteria set out in the HUD Guide.

The science of counting more people using fewer people

In the previous post, I explored how the U.S. government counts the homeless population. We learn that they don’t try to count everyone year-round but estimate the count on a single day of the year. But they don’t really count every homeless person everywhere on one day, either. In this post, I look into how they handle the sampling dimension of geography.

Each jurisdiction (dubbed CoC in the HUD Guide) divides its territory into sub-regions. The sub-regions are labeled high- or low-density based on the anticipated number of homeless. The sub-regions are “sampled”: not all sub-regions are counted; those sub-regions without actual counts are estimated. All high-density sub-regions are chosen, plus a selection of low-density regions. This sampling scheme makes a lot of sense, as it expends the most effort in places where most homeless reside.

Elsewhere, the HUD Guide discloses that some regions are completely excluded, e.g. suburban sub-regions that have not reported any homeless people for a few years might be removed in subsequent years. Thus, there are really three sampling “strata”: high-density sub-regions are 100% sampled, zero-density sub-regions are 0% sampled, and the low-density sub-regions are partially sampled (the counts in those low-density areas that are not sampled will be subsequently estimated).

Unfortunately, the authors of the HUD Guide call the above sampling scheme a “Random Sample of Areas”. For example, on page 12, the guide misrepresents the sampling scheme as:

An example of a random geographic sample is a CoC [what I’ve been calling a jurisdiction] that selects some geographic areas (e.g. according to Census tracts or, city blocks) to represent the entire CoC geography. The data collected from the homeless people counted and interviewed in these regions would then be adjusted to represent the overall homeless population.

Without reading the details of the sampling scheme, one is misled to believe that every sub-region has equal chance of being selected, which isn’t an appropriate way of sampling these sub-regions, nor is this the recommended sampling scheme described later in the Guide.

Now, your numbersense might have detected something I slipped past you a few paragraphs ago... how is it that they know which sub-regions have high or low numbers of homeless before they start counting? Seems like there is a chicken-and-egg issue. This is indeed the case: in order to reduce the workload, we need to send more workers to high-density areas but if we know nothing about the homeless population, how can we designate some sub-regions as high-density?

The truth is any statistical procedure involves some guesswork. In this case, because HUD isn't dealing with homeless for the time time ever in 2024, they can use historical data as a guide; in addition, experts working at organizations that work daily with homeless people have valuable insights. Imagine a sub-region that has experienced a recent surge in homeless. The historical data would be misleading in this case but the expert knowledge would likely be informative.

***

Now, consider what happens inside a sub-region. We ask the same question as before: do we just walk around the entire area and count (and survey) every homeless person we encounter? In the HUD Guide, this method of counting is called “complete coverage count”.

Most large jurisdictions probably practice something called “known locations count” method. This involves compiling a list of all locations that homeless people tend to stay at overnight, and visiting those locations. This labor-saving tactic, just like the other one we’ve seen, carries the risk of under-counting. They also must work with experts to figure out all the spaces where the homeless reside.

The Guide also allows something called “service-based” count. This is another fascinating concept: the staff visits social service centers, such as food banks, that homeless people might visit during the day. This count is materially different from the other counts in two ways: the counting happens during the day (not at night); the counting does not happen on the night of the counting day but during a seven-day window that starts after the night of counting.

This "service-based" count is a remedial count; it is used to correct the initial night count. The assumption is that some homeless will be missed on the night of counting but they might show up at social service centers during the day. Deduplication is really important here. They must make sure the same person isn't counted twice. This is the reason why the service-based count doesn't commence until the main counting is finished.

***

If you still think counting people is easy, I'll leave you with one further complication.

Since the unsheltered homeless usually aren't found at their sleeping locations during the day, the counting takes place during the designated night, but the window of counting spans midnight. So, if left to our own devices, some of us will write in one date while the rest of us will write in the other date as the day of the count.

The Guide explicitly stipulates that every worker should use the date of the start of the counting window as the date of the count.

***

I was expecting the whole exercise of counting the homeless to be a mess but what I found instead was a well-written, quite comprehensive documentation of the methodology used by HUD workers to measure the homeless problem.

 

 

How hard can counting be?

Since the pandemic, the homeless problem has become visible in some major cities. The U.S. government just reported an 18% increase in the homeless population in 2024. This news brings forth an intriguing statistical question: how do they count the homeless?

Do they conduct a poll with some kind of random digit dialing, and compute the proportion of respondents who admit they are homeless? (Just kidding…)

HUD (Department of Housing and Urban Development) reports the homeless statistic as a count: 772K people are flagged as homeless, which is 18% higher than the count in 2023. How do they know this? Do they go around and find every homeless?

A moment’s thought quickly dispels that as impractical, well nigh impossible. To start with, the number of homeless people changes every moment in time. Is the 772K a snapshot or some kind of average? According to the Bloomberg article, it’s an “annual point-in-time estimate.” So, the bean-counters conduct a homeless tally once a year, on some specific date.

Even that should be impractical. Do they run around every neighborhood in all 50 states to find every homeless person hanging out on the streets that day? To learn more, I found my way to this HUD guide (link).

Reading this guide is like peeling an onion. Each answer leads to more questions; to their credit, the guide covers most of what we need in order to understand the count.

First, they provide a clear definition of homeless. Only people who meet the inclusion criteria are counted.

A homeless person can be “sheltered” or “unsheltered”. Sheltered homeless people are those who checked into shelters temporarily while unsheltered are those sleeping outside, at train stations, etc.

Notably, if some community offers permanent housing to the homeless, they are no longer regarded as homeless. So hypothetically, the homeless problem could be “solved” by giving away free housing, i.e. by affluence. (Before you jump all over this, I put “solved” in quotes for a reason.)

Sheltered homeless is easier to count as most organizations that offer temporary shelters to homeless receive public funding, and thus must satisfy reporting requirements. To obtain the sizes of these populations, it is possible to just extract data from a database. There are still potential problems, such as incomplete or inaccurate reporting.

Unsheltered homeless is counted in a more labor-intensive way by having workers approach them to conduct a survey. The survey isn’t strictly necessary if all we want is a count but since the government also wants to study different subgroups of homeless (e.g. homeless youth), the workers attempt to interview the homeless people they encounter. The guide also discloses that each interviewee receives “incentives” to participate – not shocking as many businesses also pay people to fill out questionnaires.

***

Having reduced the problem to counting the homeless who are sleeping outside, we’re still stuck with the prospect of running around whole regions to find every such person! This is still impractical for any sizable jurisdiction. We therefore need statistics. Counting everyone is called a “census”. Instead of counting everyone, we use statistical sampling to arrive at an estimate of the census tally.

Sampling means we don’t try to count everyone. Instead, we count some of the population, and then use math to extrapolate. How does HUD decide what to count?

This topic is unexpectedly rather complicated. The first sampling dimension is time. We want to describe the extent of homelessness for the whole year of 2024 but we certainly won’t be counting people every day of the year. As mentioned above, HUD only counts unsheltered homeless on a single day of 2024. Naturally, we should wonder why that single day (in late January) is selected as representative.

The HUD guide answers this question directly. It turns out they make no claims that the snapshot tally is “representative”. Even if it’s not the elusive number we’re chasing after, this estimate is still useful – imagine that the homeless condition has worsened generally across the U.S., however you want to define it, we’d expect that deterioration to be reflected in the single-day tally as well.

Further, HUD makes this amusing claim: they picked the last week of January because they expect to find the smallest unsheltered homeless population during that time of the year. Is this a brazen attempt to under-report the total? Here’s how they explain it: In many regions, the cold winter weather drives homeless to check into shelters, rather than sleeping outside. That’s why they expect the relative proportion of unsheltered homeless to reach a seasonal low. Because HUD staff believe that counting sheltered homeless is easier, and less prone to inaccuracy, compared to counting unsheltered homeless, they conduct the counting exercise in January each year.

The other benefit is labor saving. Counting the unsheltered homeless is labor intensive.

By restricting the counting to a single day, we have reduced the workload by several hundred times. But any labor saving comes with a cost – if someone wants to extrapolate this snapshot to the entire year, much more work is involved.

[A further nuance: it’s too hard to coordinate all jurisdictions to count on a single night, or for larger communities, to finish counting on one night. In practice, there is a week during which the counting takes place. Moving from one night to multiple nights creates the risk of duplicate counting as the homeless population is fluid. HUD spends quite a bit of energy to make sure the same person isn’t counted multiple times.]

The task at hand is still too daunting. Just imagine having to run around every corner of Manhattan to count all homeless people on a single night. Thus, the task is further condensed by sampling along a second dimension: geography. I’ll discuss this topic in the next post.

Contaminated science

 

 

In one sense, the E.coli outbreak attributed to McDonald's has ended, as reported cases have dwindled to a drip. In another sense, it hasn't ended, because the number of cases is still growing.

This is the challenge of foodborne disease outbreak investigations. It is still possible that the theory that onions caused the outbreak is a false lead. In the most successful investigations, scientists were able to isolate samples of contaminated foods which contain the specific strain of the bacteria causing illnesses. An example of a very successful case is the spinach investigation featured in Numbers Rule Your World (link).

***

Late last week, CDC put out a new press release (link), updating the public about the current investigation. But it manages to unnerve me with terrible messaging.

The communication on November 13, 2024 carries the following conclusion:

"Epidemiologic and traceback data show that fresh, slivered onions served at McDonald’s were the likely source of this outbreak. Affected onions served at McDonald's locations were recalled and are past shelf life."

The sentences point to two types of evidence: epidemiological data come from questionnaires sent to reported cases asking for information such as what foods were consumed prior to getting sick from E.coli; and traceback data require salvaging potentially contaminated ingredients at source and testing them in labs for the specific strain of E.coli associated with the current outbreak.

We already knew about the epi data. What's new in this press release is the traceback data. Epidemiological data always come first, and were used to point the finger at McDonald's, then Quarter Pounders at McDonald's, then fresh, slivered onions in Quarter Pounders at McDonald's.

Scroll to the bottom of the press release, and this is the entire paragraph published about traceback:

FDA tested recalled onions and one sample tested positive for non-O157:H7 Shiga toxin-producing E. coli. The E.coli found in onions was not the outbreak strain and no human illnesses have been linked to this strain. Additional testing results will be released when available by FDA.

It's a word salad worthy of U.S. presidential candidates.

Translation: the investigators did not find the specific strain of E.coli associated with this outbreak in the onion samples they collected from the farms. We don't know how many samples they tested but only one of them tested positive for any kind of E.coli, and unfortunately for the investigation, it wasn't the right strain. Thus, this positive result does not strengthen the evidence against onions from those farms, and their customer, McDonald's.

And yet, whoever wrote the summary paragraph inserted the false claim that epi and traceback data were consistent, and complementary.

As a scientist, I find that this type of communciations undermines my confidence in our public health organizations. One can say this is an instance of "story time". They lull us in with some data, and then spin a story ostensibly supported by the data but not really.

Indeed, most media outlets have swept the uncertainty under the rug, referring to this outbreak as the McDonald's outbreak, or the onions outbreak. The CDC press release actually only used the term the "likely" source.

***

For giggles, I asked ChatGPT whether the lab test results added to the evidence that onions were the cause of the outbreak, and here is its answer:

No, the lab testing did not directly add to the evidence that the onions were the cause of the E. coli outbreak. The epidemiological evidence was already strong, with 99% of the interviewed individuals reporting eating at McDonald's, and 84% specifically eating menu items containing fresh slivered onions. The lab testing of recalled onions found a non-O157:H7 strain of E. coli, which was not linked to human illnesses and did not match the outbreak strain. Therefore, while the testing confirmed that the onions were contaminated with E. coli, it did not provide direct confirmation that the specific strain responsible for the outbreak was linked to the onions. The epidemiological data (such as the high proportion of sick individuals who ate the onions) was the primary evidence supporting the conclusion that the onions were the source.

I like this answer!

***

Note: It's possible that the lab testers knew something else that links those tested onions to the outbreak but nothing about the lab test results as disclosed in the press release supports its summary paragraph.

Note 2: In the meantime, the economic damage of food recalls is real. McDonald's is reportedly spending $100 million to recover from the fallout (link).

 

The election could have been called much earlier

(Source: St Stev, Flickr)

The talking heads on TV talk shows like MSNBC and Fox News are notorious for their innumeracy. But on election night last week, they impressed me by demonstrating numbersense.

Viewers have been warned not to expect a result on the day of the election, as all the reputable forecasters such as Nate Silver, FiveThirtyEight, and the Economist, had proclaimed the 2024 edition to be the most tightly contested ever. Silver famously announced that he ran 80,000 simulations to discover that the 2024 presidential election would be a toss-up (50/50 with Kamala getting the edge by the tiniest of decimal points). Close races will require counting of all votes, including early and mail-in votes, and some states accept mailed votes that arrive after election day if they were postmarked prior. It sounded like we were predestined for another cliffhanger, like in 2020.

Various channels stayed with that narrative faithfully throughout the broadcasts, as the hosts (including some data scientists) dove into selective counties to report how Trump was performing against his 2016 or 2020 margins, and whether Kamala was over- or under-performing Biden's 2020 numbers.

Meanwhile, a red wave was rolling over the whole map, as we eventually learned. The early taking of Florida was a harbinger. Then, Trump started leading in all seven so-called "swing" states. Even now, the talking heads told viewers the election was "too close to call". It was not until the next morning that most of these networks eventually confirmed Trump the victor.

Nevertheless, some time before midnight, the tone of the election broadcasts changed. The hosts ceased pretending that Kamala had any chance left, even as they perfunctorily stated that their networks had yet to call the overall winner.

Those were the telling moments at which the talking heads gave in to their numbersense. Their data science colleagues were telling them to shush for the sake of good "science"; the business leaders didn't want them to ruin the watch parties. And yet, the numbersense in their heads insisted the race was over.

The hosts tip-toed into post-mortem analyses, but checked themselves, acting like unruly drivers who cut over multiple lanes to make a turn, all the time apologizing to other drivers while doing the naughty. Later, they threw caution to the wind, and earnestly examined the bruised blue corpse of the election.

***

The mood turned on the concept of "too close to call". What does it mean?

In the U.S., the news outlets follow the presidential election as a collection of state elections because of the electoral college. The focus is on the "swing states", of which there are seven in 2024. These are the (only) states for which the outcomes are somewhat uncertain.

The media outlets have been endowed the responsibility of "calling" the election. For some reason, each state election is reported as a horse race, with vote counts continuously updated after the polls close. As a result, while people in the Western states are still voting, the results in some other states have already been publicized. Furthermore, early results don't necessarily forecast final results since the sequence of vote counting is far from random. The non-random order of counting affects close races even more.

All the swing states were tagged "too close to call" on election night. What calculations happen to draw this conclusion?

Let's take a look at the situation in North Carolina at 10:47 pm ET, as shown in this CNN screenshot:

Trump was leading Kamala 51% to 48%, a gap of 135,000 votes. The race was labeled "too close to call".

The key number is displayed at the bottom: the current result reflected 84% of the total votes. That means 16% of the votes were still to be counted. Since 4.8 million votes have been counted, they were expecting roughly 910,000 votes. (I'm assuming no third-party votes, which is roughly correct as the Kamala and Trump percentages added up to 98.6%.)

The gap of 135,000 is 15% of 910,000. Thus, in order for Harris to get even, she would have needed to grab 15%+48% = 63% of the remaining votes while Trump gets 51%-15% = 36%. If she were to overtake Trump, she would have needed to win over 63% of the remaining votes.

In the initial 84% of votes, she claimed 48% to Trump's 51%; in the last 16%, she had to win 63% to 36%. In my view, that was an impossible feat so the race could have been called at that point (or even earlier). While the last 16% of the votes could have been skewed Democratic, it's not likely because the vast majority had already been counted, and counting happened simultaneously across all counties.

The data scientists behind the scenes would have access to even more data, at the county-level. The remaining votes were going to be found in a subset of the counties. They can repeat the same analysis obtaining a county-level required vote split for Kamala to get even or overtake Trump. On election night, the hosts worked around the election maps and highlighted some of the counties that have higher populations, especially Democratic strongholds where Kamala could have found additional votes. It didn't convince me that Kamala could cover the gap.

***

Now, the networks would defend their non-call by a criterion of "mathematical impossibility". That essentially means that they wouldn't call the race unless and until Trump's vote tally exceeded half the total votes, making it "mathematically impossible" for Kamala to win, regardless of uncounted votes.

This defense rings hollow because the same networks were extremely quick to call states like California and Wyoming - typically within a minute of the polls closing. At the time they called those "easy" states, they clearly did not see Trump's tally exceed half. So they were using different criteria for different states.

The whole business of having media outlets "call" the election results boggles the mind. These outlets have an incentive to hold out as long as possible because of viewership and ratings.

 

It's 2016 all over again: resurfacing my post from 2016

In the aftermath of Hillary's shocking defeat in the 2016 U.S. presidential election, I wrote a blog post titled "This election forecasting business."

In the aftermath of last night's news, I point you back to my 2016 blog post (link), as it is as relevant as ever. Most election forecasters, including the pioneer Nate Silver, had predicted that the election would be extremely close, like a coin flip.

***

This is the tldr; from my old post:

tl;dr Citizens shouldn't care about this election forecasting business. It's there for the benefit of the politicians. The forecasters have been over-selling these predictive modeling technologies. This is especially true if they are merely aggregating polls. If you can't validate the accuracy of such models, how much time/money are you willing to spend on them? The prediction markets people are quiet but they too have no clothes. Journalists should spend less time writing code, and get on the road and talk to real voters.