Big Data, Plainly Spoken (aka Numbers Rule Your World)

Self-checkout takes another hit

I recently wrote about a common misconception about self-checkout technology in supermarkets. Many people experience shorter wait times, and attribute the gain to the technology - when in fact the higher efficiency is merely because the supermarket has many self-checkout counters open compared to human workers. See here for more.

Self-checkout technology has another downside, which has surfaced with the tide of the economic slowdown. Target - one of the top retailers in the U.S. - just reported disappointing business results, blamed on "organized retail crime" (stealing from stores). In this Yahoo article, they explained a major reason why retail crime has increased - self-checkout technology:

our stressed social contract may be capping how far we can push this people-light, technology-heavy model. Last month Wegman's ended its scan-and-go shopping app. Why? Shrinkage, of course.

[Note: Wegman's is another major supermarket chain.]

It's telling how scared these journalists are of the tech industry; I guess they are employed by a tech company (Yahoo) so it's understandable. The above sentences were tagged on at the end of a paragraph, enclosed in parentheses, and drenched in technical jargon when the message can be simply stated as:

self-checkout technology has opened the floodgates to cheaters and thieves who steal from major retailers, causing substantial losses to retailers

This "shrinkage" is very damaging to a company's finances. The retailer paid for the cost of the stolen products but received $0 for them. It's a "total loss". This isn't counting any expenditure the retailers spent on, e.g., more technology to prevent or detect stealing, or efforts to catch thieves.

Is this really surprising? Not if you live in a city with a subway system which - maybe 20 years ago - started replacing humans with ticket machines (e.g., New York CIty). Which of these scenarios result in more fare evasion?

(A) each entrance has staffers in front of the turnstiles
(B) most entrances have no staff on duty since passengers buy tickets from machines

No behavioral economist is needed to figure this one out! In fact, there have been quite a few reports on self-checkout theft, e.g. this one.

We can be sure that Target executives are being marketed more technology - for detecting self-checkout theft. One idea might be to use video surveillance to find acts of stealing, face recognition to identify thieves, and cellphone tracking data to locate suspects.

We already know such a solution doesn't work. Think about what happens if one's iPhone got stolen. If Find My Iphone is turned on, one can literally see where the thief is, and track his/her movements. But...Apple isn't going to send someone to retrieve the phone for us, nor, as far as I know, will the police do so.

 

 

 

Other numbers from the colonoscopy trial

You think I'm done with the colonoscopy trial - I'm not.

Previously, I discussed the core controversy, which is when investigators learned that the trial outcomes are not what they expect - in fact, colonoscopy effectiveness was far below what the medical profession has believed - they decided to cancel the primary endpoint and tout an "adjusted per protocol" analysis, which was designed with knowledge of the disappointing primary result.

Any per-protocol analysis is a partial lie. It takes a well-designed randomized clinical trial and dissolves the bedrock of random treatment assignment, effectively turning the RCT into an observational study. I call it a lie because people who do PP analysis describe their result as coming from an RCT. The authors of the colonoscopy study are proud that this is the first ever randomized controlled trial for this treatment. This is exactly what happened with all the Covid-19 vaccine trials too: they only analyzed people who took two doses (but if someone gets sick after the first dose, they are ineligible to get the second dose). If the analysis is PP, the study should be classified as an observational study even if the original design is an RCT.

In the last post, I did a deep dive into the two analyses that were presented in the NEJM paper to show why the adjusted PP analysis allowed the research team to print an outcome of 30% improvement on diagnosing colon cancer rather than 18% in an ITT analysis. I find the underlying mechanics rather implausible but as with any observational study, it's anybody's guess as to whether the statistical adjustments helped or hurt.

***

There are a few numbers reported in the study that haven't gotten as much attention in the media.

The most important is the unproven impact on all-cause mortality. Exactly 11% of the invited-to-screen group died within 10 years, and the same proportion died in the usual-care group from any cause. In other words, even if colon cancer was the cause of death on fewer death certificates in the screening group, the same number of death certificates would have been issued in both groups. This is probably because colon cancer kills at an advanced age, and people die from other (related or unrelated) causes if they didn't die from colon cancer. Thus, colonoscopy did nothing to change the life expectancy of the participants.

The researchers did report a large impact on colon cancer specific mortality in their "adjusted PP analysis" (50% compared to 10% in ITT) but they were silent on all-cause mortality. The mortality result was merely stated, and no further details were offered on compliers vs noncompliers, age groups, countries of origin, etc., thus no one can judge whether it makes sense.

Also, note that all rates are cumulative over 10 years of follow-up. Actually, that is also a partial lie. It's a "median" of 10 years of follow-up, meaning that only half of the participants need to have reached year 10. From Figure 1, I can see that about 66% have reached year 10, 85% have reached year 8, etc. The important thing to realize is that the further out in the case curve you go, the smaller the sample size that underlies the numbers (i.e. less reliable).

Another important number is the absolute risk difference. Thirty percent sounds like a large improvement as a relative ratio but it isn't because the baseline risk is very low. Cumulatively over a median 10-year follow-up period, the baseline risk was only about 1.22% (the risk of the usual-care group). Thus, 30% improvement on that is about 0.4%. That means out of 1000 people invited to screen, we can expect about 4 fewer diagnoses of colon cancer over roughly 10 years.

Flip that ratio around and we have what is called Number Needed to Treat (NNT). To prevent 1 colon cancer case, we need to screen 250. Using the ITT result of 18%, the NNT is 455.

A glaring gap is the lack of a cost-benefit analysis. A colonoscopy apparently costs $3000 in the U.S. So, out of 1,000 invitations, perhaps 500 do the screening, which cost $1.5 million, in order to lower diagnoses of colon cancer by 4. Thus, the cost of each avoided colon cancer case (not death, just case) is $375,000.