Big Data, Plainly Spoken (aka Numbers Rule Your World)

Privacy for me not for you

Two years ago, I described the saga about the college kid who put up twitter feeds that track private jets, thus revealing their movements (link). I said: "The right not to be tracked is poised to become a civil rights issue."

This prediction is one more step closer to becoming true, based on Business Insider's update on this saga. Apparently, hidden inside the FAA reauthorization bill is a provision that lets private jet owners obscure their tracks.

Who owns private jets? Not the average citizen.

By default, citizens will be afforded zero privacy, while the super-rich and well-connected will buy privacy from the politicians. That's where we're heading.

Jack Sweeney's jet tracking stunt is a precision-guided bomb, targeted at the private-jet class. It exposes the hypocrisy of tech leaders.

***

But such laws can't stop engineers from getting what they want.

Famously, Elon Musk banned Jack Sweeney's twitter feeds but he just moved his feeds to a twitter competitor.

Jack already has a response to the new law. "We can still figure out who's who via context clues."

These tech CEOs will discover that they couldn't escape the data-hungry, privacy-busting tech infrastructure they brought to our world. What are these context clues? Their homes, for example, are not easily disguised. The make and model of airplanes. etc. It may take more work but the determined engineer can figure it out.

***

When Eric Schmidt was CEO of Google, he infamously said "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place." (link)

The same Eric Schmidt some years later got into the military drone business, and he sang a different tune: "You're having a dispute with your neighbour. How would you feel if your neighbour went over and bought a commercial observation drone that they can launch from their backyard. It just flies over your house all day. How would you feel about it?" He was calling for a ban on civilian drones (link).

As far as I know, he has not retracted his earlier statement.

 

 

 

 

A CEO says they will spend money to lose money

Oops, I thought this post went live but found it sitting in the draft pile. I referred to it in this week's post. So here it is...

 

Wendy's, a U.S. burger chain, must have thought it was a smart move to jump on the AI bandwagon by saying they would implement "dynamic pricing" next year.

Immediately, journalists rightly associated "dynamic pricing" with "surge pricing," famously implemented by Uber. Economists love surge pricing - the practice of hiking up prices when the demand of Ubers exceed supply, because they believe prices should reflect supply-demand relationships. I hated surge pricing from the start because it implements price discrimination - in the case of "taxis", it has the effect of letting well-to-do people get in front of the line by paying more, instead of jostling on the streets to flag a cab. It's no mystery why people who can afford to pay double, triple, etc. the normal fare are delighted by surge pricing.

Wendy's though is a burger chain, not a Michelin-star restaurant. I don't think its customers are going to appreciate surge pricing. The backlash was so swift that the next day, the company denies that it had ever intended to do "surge pricing". They said:

Wendy’s will not implement surge pricing, which is the practice of raising prices when demand is highest. We didn’t use that phrase, nor do we plan to implement that practice,” a spokesperson said in an email to NBC News. They added there are “no plans” to raise prices at high-demand times.... "Digital menu boards could allow us to change the menu offerings at different times of day and offer discounts and value offers to our customers more easily, particularly in the slower times of day."

Let's pretend we are the team that approved spending $20 million to install these digital menus boards. The minute they are installed, the corporation must recoup $20 million in order to break even. If the project just breaks even, the finance team would never have greenlighted it; we must have claimed that by having these digital menu boards, we would generate some additional sales in excess of recouping the investment.

Since our CEO said we wouldn't raise prices during the busiest time, we obviously wouldn't raise prices during low-traffic periods either. Let's pretend that we would never raise prices. Sounds like we told the CEO that we could recoup $20 million plus generate additional sales by lowering prices!

What would be the theory? It would be that by lowering prices during slower periods, they would generate incremental traffic and sales to recoup $20 million plus a lot more.

What might go wrong? First, is price the only or limiting factor that causes slower traffic? If price isn't the reason, then they won't generate more sales. Second, would they merely shift revenues from busier periods to slower periods as some existing customers would want to take advantage of lower prices? In that case, not only would they not make more money, they would make less! Third, even if nothing changes, they would lose money because the people who show up during slow periods now pay the lower prices.

***
That said, it's smart for Wendy's to switch to digital menu boards, just like many other places have. As consumers, we already know the story: digital boards make it easier to raise prices to everyone! That's been the lesson of the last few years of crazy food inflation here in the States.

Know your data 36: the what and the why

In the unlikely place of a food blog (link), there appeared a feature article about restaurant data science, which provides a rare glimpse of the ins and outs of the data collection business.

The article describes pretty basic elements of data science that have been used by other industries for a while; evidently, restaurants have been reluctant to join this game until now.

***

Right from the start, the featured company is described as "stealthy." This description has a double meaning – it's a "stealth startup" although given the number of important restaurant groups mentioned in the article that already use its services, it's unclear whether this sense of "stealth" was what the reporter intended; the other meaning is that the company collects data "stealthily" and works behind the scenes with restauranteurs. I have never understood the secrecy in the data collection business when the industry leaders insist that they utilize the collected data strictly for the benefit of their customers – recall the recent fallout over Wendy's announcement to adopt adaptive pricing (link).

Perhaps these data collectors won't voice what they know, which is that raw data aren't all that useful. There is a question that lingers in my head as I read through the article: can we have at least one clear statement of the benefits arising from the collected data?

The article begins with what must be the elevator pitch of SevenRooms, which is portrayed as a leader in data collection and analytics in the restaurant space.

"Let's say you're dining at Lure Fishbar in Soho on a weeknight for dinner... kicking things off with a bottle of Champagne. As it turns out, without even going over to your table, owner... and managers know what you and and your tablemates are spending in real time, as you're spending it."

This leading paragraph is a bit anticlimactic. Restaurant managers have always known which table ordered what, and whether the table is a big spender – even without deliberate data collection. And the reporter didn't answer the question: How does SevenRooms utilize such knowledge? How do these actions benefit the diners?

"Based on your previous visits to any restaurant in the Mercer Street Hospitality group... they know your dietary restrictions and your wine preferences; how often you've visited...; how much you spend on average, per night, and per year."

This sentence also focuses on the what, but not the so what? Let's think this through.

What extra service do frequent diners get? Do the waiters come by more often because I'm a frequent patron? Compared to the prior state, does total service effort increase because more service is rendered to frequent patrons? Or is total service level fixed, implying that the waiters visit low-frequency diners less? If total service effort goes up, are waiters paid more? If so, does the manager expect that the higher service effort will induce frequent patrons to spend more money, thus paying for itself?

***

Next, let's consider the nature of data such as "wine preference". What SevenRooms actually possesses is the history of wine orders for each patron at the subset of restaurants that use SevenRooms, from which the diner's preference is deduced. The accuracy of this deduction depends on the individual. Some people may like only a single type of grapes but others may want a particular vintage from a particular winery of one type of grape if they order a particular dish at a particular restaurant but another vintage from another winery of a different grape when eating some other meal, and so on. If all those factors are included in the model of wine preferences, and we desire accurate predictions at the individual level, then we'd need quite a lot of data for each diner, beyond the amount that SevenRooms could have amassed.

From this discussion, you should realize that "wine preference" information is only useful if (a) the restaurant group is relatively large so it can capture most of the diner's wine drinking; and (b) the diner attains some threshold of loyalty to the restaurant group.

We have encountered this data science application before. Your Netflix movie recommendations won't hit the target if (a) you do a good chunk of your movie watching on other platforms; and (b) you've watched (and rated) only a small number of movies on Netflix.

***

Let's pretend that SevenRooms actually has a good model of people's "wine preference".

For a one-grape patron, it's easy to get "wine preference" right, and I'm sure the managers would have gotten it right, with or without data collection.

In the other extreme, a good wine list may have many wines that fit with a diner's history, and should the restauranteur impress the diner by sending the top recommendation - in lieu of the wine list - to the table? "Here's your wine for the evening. (And don't you dare say no)." If the restaurant doesn't send the wine directly, but arms the sommelier with the recommendation as an ice-breaker, is this materially different from the status quo?

***

The reporter gets more and more confused about the benefits of SevenRooms's data as she issued more words. The data could seemingly solve world hunger. The following miracles were mentioned:

"data can be key in the battle for luring diners"

"using customer data to tailor menus"

"should you choose to splurge on that $3,000 bottle... all hands on deck, with every manager getting an alert"

"can parse seating arrangements so that some tables are earmarked for Resy, some for OpenTable, ..."

"steps up service for VIPs"

"using it to plug and play among various reservation platforms quickly and on the fly"

"stitching together historical information on... needs and guidances"

"plotting tables on the floor plan to optimize the flow of service"

"tracking a table's stage in their dining experience"

"quote wait time"

"automated email campaigns"

keep track of which restaurant reservation channels are responsible for sending restaurants what numbers of diners

let restaurants send automated Priority Alerts to their most valuable guests about availability

"helps restaurants reward their most valuable guests to get them through their doors most often"

run a loyalty program "behind the scenes"

"improves customers' experience"

"and much more"

For each of these presumed benefits, one should consider how it changes the revenues and costs of the restaurants, whether the collected data really provide reliable insights to drive these actions, and how will the restauranteurs learn whether they yield the expected incremental gains.

From grand theft to grand retreat

 

Last year, I wrote about the expected downside of deploying self-checkout technology in retail stores (link). Since then, many retail chains have announced cutbacks in self-checkout tech.

The latest to join the exodus is Dollar General (link). DG operates 20,000 stores. They have installed self-checkout registers in 14,000 stores. Because of rampant theft, the retailer is admitting "there is no substitute for an employee presence".

Here is the new policy: In 300 stores, the self-checkout option will be removed entirely. In 9,000 stores (64%), self-checkout will be relegated to an overflow option during rush hours. In the other 5,000 stores, self-checkout will be limited to five items or fewer.

These changes are very telling. The theft is so widespread that the changes affect every store. Limiting usage to five items caps the potential loss.

Target, Walmart, Five Below, Costco, etc. have all announced changes to self-checkout operations in the last few months.

***

This is not the first time removing humans backfires. I see the same problem every week in the NYC subway. Some travelers jump over turnstiles, enter through emergency exits, or use various tricks to evade the fare. Since switching to self-serve ticket machines, most stations have zero staff presence. No one is around to stop fare-evaders, not even to look them in their eyes. (Fare evaders are not limited to the stereotypes; I have seen amongst them families with kids, professionals in suits, and people of all colors and ages.)

I'm guessing that something similar is happening with automated toll collection on highways. Just like self-checkout, there are obvious ways to fool the technology (e.g. remove license plates, cover them, or fake the plate numbers). Like most businesses that deliver a service and collect money later, I'm guessing highway operators face a "non-payment" problem, which requires an expensive collections operation. Contrast this to the past: few drivers ran through toll booths with human toll collectors without paying; tolls were collected before the service was rendered.

For sure, these technologies bring benefits as well. The hidden cost is unfair distribution of costs. The NYC subway is constantly in grave financial trouble. Eventually, they must raise prices. The people paying higher prices are the ones paying the fare! In effect, they are paying for their own tickets, plus the tickets of the fare evaders. For self-checkout, the ones not stealing will eventually pay for those abusing self-checkout. Those opting not to use self-checkout also will pay for the groceries of the self-checkout thieves.

This state of affairs may last for a while. Over time, more and more customers will migrate to the cheating segment, as it is the rational thing to do, given the rules of the game. The more people cheating, the smaller the remaining group of customers bearing the cost, the greater the incentive for them to cross over.

***

There is one other thing that these retail executives are going to discover soon, something that is just as predictable as self-checkout theft. This problem will be particularly acute in those stores that choose to combat theft by deplaying staff as monitors. The reintroduction of humans has transformed their roles from servicing with a smile to policing with a scowl. Customers find themselves between machines and police, not the most inviting environment. (The machines will likely also be equipped with surveillance tools.)

As some of these retail chains have learned, there is still a place for human-rendered services.

On the evidence raised by SAT distractors

After reading my prior post, long-time reader Mark (link) alerted me to a series of posts he wrote 10 years ago on the anti-SAT movement. This is a topic that bubbles up every few years, and it appears that the current generation of college administrators may go where their predecessors didn't. Some colleges have started to drop the SAT requirement for admissions.

Post 1, Post 2, Post 3, Post 4, Post 5

Let me summarize Mark's various posts and add some color commentary.

***

In his first post (link), Mark explores the relationship between SAT scores and college GPAs. This exercise in analyzing correlation is meaningful to both sides of the debate. The distractors detractors argue that "there already is something that accurately mirrors the grades a student gets in school. Namely: the grades a student gets in school" while proponents say SAT scores are the best predictors of college success.

For example, when MIT reinstated the SAT requirement after the Covid-19 pandemic (link), the dean of admissions stated:

Our research has shown that, in most cases, we cannot reliably predict students will do well at MIT unless we consider standardized test results [especially the math component] alongside grades, coursework, and other factors.

Mark points out that the right measure of any predictor is incremental insight – what additional nuance it adds to the collection of predictors. This explains why the SAT looks more like a general intelligence test than a high-school exam. Otherwise, the SAT score would be an unnecessary duplicate of high-school GPA. (Why the previous sentence is stated forwards but not backwards is another sign of weak analytics in this space.)

Mark's point is self-evident: assume that high-school GPAs predict college performance just as well as the SAT, as the distractors detractors suggest, then SAT scores should mirror high-school GPAs, and if they did, the two metrics would result in identical rankings of the students; if so, what's the cause for complaint?

***

The second post (link) examines various reasons SAT critics have given for dumping the test. Mark is unimpressed by the quality of the evidence, and in many cases, its absence.

The weirdest argument is that according to Google's description of its own hiring practices, Google does not consider standardized test scores or college ranking. Mark remarked that "Google doesn't talk much about SATs at least in part because they've largely maxed out the metric."

Let's say we tally up the background data of Google hires; we are likely to find a good number of Ivy League graduates with high SAT scores. Can Google legitimately claim their hiring does not consider test scores or college ranking?

Sure, someone is going to say "Those people weren't hired because they graduated from an Ivy League school, or that they had high SAT scores. They just happen to have those characteristics. Correlation is not causation!"

Well, admitted students to top colleges not only have high SAT scores, but they also have high GPAs, and other great things on their applications so did they get accepted because of the SAT scores, or the high GPAs?

***

The third post presents a series of math questions from actual SAT exams because Mark realized that many critics of the SAT attacked an imaginary scarecrow version of the SAT test, rather than what the test really is. Seeing these real questions helps ground the debate.

He also touched on one virtue of the SAT test, which is its transparency. Past exams with answers are readily available. By contrast, transparency is definitely missing in high-school GPAs. Nevertheless, in real life, transparency frequently backfires. Where are these SAT critics when it comes to high-school GPAs? Let's compile GPAs and split them by race - I'd be surprised if they didn't find a race gap. The reason that the race gap in GPAs isn't in the news is that schools don't release such data.

***

The fourth post is a must-read. Here is Andrew's recap of what Mark wrote.

Mark trashed those critics who grumbled about the point deduction for wrong answers in the SAT. Mark explained why the point deduction scheme in fact simplifies life for test-takers, because the penalty makes it counterproductive to do "random guessing" (what Mark describes as "guessing without first reading the question"). Tests that don't deduct points for wrong answers are more annoying because smart test-takers should guess on certain questions based on their level of confidence in knowing the answer.

The unspoken point behind Mark's post is that the SAT is rigorously designed, and continuously improved, by lots of statisticians, and therefore it is the best instrument we have. In Numbers Rule Your World (link), half of Chapter 3 is devoted to the statistical work behind a niche issue in SAT tests, work that reduces specific biases due to race and other characteristics.

The biggest irony in the anti-SAT movement is that on the major complaints about the SAT, alternative instruments such as high school GPAs and reference letters are far worse.

***

In the last post of this series (link), Mark turned our intuition upside down. He argued that the students most likely to benefit from the SAT admission requirement are those good students who go to public schools – or if we extend this line of reasoning, good students from underprivileged backgrounds. That is to say, the SAT requirement benefits precisely those people whom its distractors detractors say they care the most about, with the caveat that the benefit accrues to the cream of that crop.

Ten years later, the MIT dean of admissions echoed Mark's point when he said that MIT uses the SAT score to identify disadvantaged students who are most likely to succeed at MIT.

unlike some other inequalities — like access to fancy internships or expensive extracurriculars — our empirical research shows the SAT/ACT actually do help us figure out if someone will do well at MIT.

***

This last discussion leads us to some general topics in the area of statistical measures. No metric is perfect, most metrics are correlated with each other, all metrics promote perverse behavior, alternatives often appear better only because of lack of data or analyses. See Chapter 2 of Numbersense (link) for more on measurement, where I dig into the anti-BMI movement. BMI is the widely accepted metric for assessing obesity; like SAT scores, the metric has attracted a small loud group of distractors detractors.

 

After tasting the forbidden fruit, college admissions will...

The war against standards in college education continues to feature unscientific discourse. Consider this PBS piece, which starts with:

Standardized test scores in the U.S. used to all but make or break college applications for high school seniors.

Nope. U.S. colleges have not - for as long as I've lived - determined admission based solely on standardized test scores. In fact, the admission policy of the typical competitive American college differs from those in many other countries, primarily because they purport to evaluate applicants "holistically," incorporating subjective criteria such as high school GPAs, extracurricular activites, and athletics.

The PBS host then went on to describe a student who scored 18 out of 36 on the ACT, was told by Google that she should apply to community colleges instead of Emory, and complained that "[...] didn't consider my grades as an A student". This has multiple fallacies wrapped into an incoherent mess. Why should Google stop any student from applying to Emory? If she did apply, the school would have looked at her high school GPA because of the "holistic" admission policy. In fact, most Emory applicants are likely to have scored above 18 on the ACT, and those who scored above 18 are likely to also be A students at their high schools.

Later PBS informed us that this particular student chose to ignore Google, Emory did use a holistic application policy, and she did get admitted. Instead of writing a complimentary piece about the value of holistic policies in dealing with certain "edge cases" (what Gladwell would have called outliers), this journalist decided to use such edge cases to launch an attack on standardized testing.

***

Nevertheless, during Covid, many colleges made standardized test scores optional in the application process; and many colleges have decided since to remove standardized testing from their "holistic" admission policy. Universities like Berkeley and Chicago stopped requiring SAT scores prior to Covid; other schools like Columbia, and William & Mary have boarded the ship.

According to the PBS article, the reason is that these tests are "biased." This is one of my favorite topics, which I discussed extensively in Chapter 3 of Numbers Rule Your World (link). Bias is a word that is thrown around casually, with little appreciation of what it means. In the PBS article, they used a single example to define it:

White students were doing better on more heavily weighted easier items with common words like golf and canoe, but Black and Latinx students did better on items that were more academic and more difficult, although those were weighted less heavily, so, words like vehemence and sycophant, which I can barely say.

This is a very specific instance of a very convoluted type of bias, exactly the kind that is well-researched by ETS, the entity that runs the SAT. In the book, I cover the statistical discipline called "differential item functioning," which are fascinating experiments embedded into standardized tests that measure precisely this sort of biases of individual test items, a first step toward correcting for them.

The justification of bias elimination is typically the pursuit of "objectivity" or "fairness." What do these test critics propose as replacement for standardized testing? Teacher evaluations is one alternative mentioned by the PBS host.

What about teacher evaluation is "objective"? I have read thousands of applicant reference letters - the vast majority are routine letters that tell me nothing ("Student X is one of my best students who worked diligently and earned an A in my class."). The memorable ones are always personal - personal references are the opposite of "objective" or "fair".

I have read some extremely negative personal references as well. The writers clearly had taken on the mission to warn admission officers about the applicants. I had chosen to ignore them in the past. Why? Because such personal references are too subjective - I knew neither the applicant nor the reference writer, I had been served a sample size of one, for every negative letter there is likely to be another reference who wrote a positive one, so who was I to believe?

Flipped to the other side, why should I believe extremely positive personal references? The same set of circumstances applies.

Does "holistic" help here? Surprisingly no. Of course, I can believe the extremely positive personal references if the student also participated in the Math Olympiad or got perfect SAT scores. But then, the references do not add much new information. The reference letters provide additional information if and only if they are uncorrelated with the rest of the application, i.e. the straight-A student with high test scores in which a reference might disclose as a cheater, or the student with low grades in which a reference might disclose as a victim of bullying. If it's uncorrelated with the rest of the portfolio, then this is just a question of trust. It's far, far from the ideal of "objectivity" that these test critics are demanding.

If eliminating "bias" were their chief concern, then the movement to abolish standardized testing from college admissions will fail. It merely substitutes one form of bias with another.

[Don't get me started on the bias due to the differential writing abilities of the teachers writing these reference letters. Apparently, these test critics are not worried about it as I haven't encountered the movement to ban reference letters yet. Worse, has there been a proper scientific study of biases inherent in personal recommendation letters? Have there been any attempts to correct such biases?]

***

I have to credit the PBS host for hinting at one of the little-known drivers of the anti-standardized testing movement. Many colleges tasted the forbidden fruit during the Covid pandemic when 80 percent of colleges decided to make test scores optional.

Because it became easier to submit applications, many colleges including Emory experienced a surge in the number of applicants. According to PBS, applications to Emory jumped by 20 percent in a year.

What the host didn't mention is what the application surge did to Emory's "selectivity" rate, which is the number of admission offers divided by the number of applicants. Since class size is fixed in the near term, any increase in applicants just lowers the selectivity rate. Quickly looking around the web, I learned that Emory's selectivity rate was 15.6% for Fall 2020. So what if we added 20% more applicants and left the number of offers the same. The new selectivity rate would be 15.6%/1.2 = 13%. And what did Google say Emory's selectivity rate was in Fall 2021? You guessed it: 13%. The entire increase in applicants translated to an apparent rise in selectivity.

Did the quality of the average applicant pool improve? If the incremental applicants consist of those who do not want to take the ACT, or those who have taken the ACT but elected to not disclose the score, I'll let you take a guess.

I noticed and wrote about this more than 10 years ago - you can read all about this tactic and others used by colleges and grad schools in Numbersense (Chapter 1, link).

While the absolute selectivity rate of Emory has gone up, its relative selectivity rank against other competitive colleges might not have. That's because one of the enablers of the applicant surge is the Common App, a brilliant invention if there was ever.

This App, purportedly a time saver for college applicants, is a tide that raises all boats. The Common App increases the total number of applications filed across the entire pool of colleges that utilize it. It's the same concept as Amazon's one-click purchase. All these schools will experience a surge in applications and a corresponding improvement in selectivity, as Emory saw.

Of course, the Common App has been in play for a while. The new battleground is standardized test scores. Removing this requirement lowers the barrier to filing an application. Those schools that made it test optional are now riding the next tide. What will college admissions offices do after tasting the forbidden fruit?

***

One more thing. The PBS host was angry that Google had recommended that the student not apply to Emory due to an ACT score well below the average ACT score of prior admitted students. Ironically, one of the benefits of standardized test scores - and generally, of any statistical metrics - is its signaling value.

Using test scores, students and parents are able to make realistic assessments of their chances of admission at selective colleges, and opt out of applying, saving both time and money. Now, students can apply blissfully, and enter a lottery with very low odds of winning. The chance of admission for someone who scored 18 out of 36 on ACT to Emory is nearly zero. Without these scores, the schools pretend that everyone has a chance.

To learn a lot more tricks used by admission officers, see Chapter 1 of Numbersense (link).

Ethical dilemmas poll: results post

Update on 8-29-2023

Last week, I put up a poll that asks users what they would do if faced with some ethical dilemmas that arise on the business side of data science. I'm presenting the first set of results here. I'm guessing the responses have stabilized but will provide further updates below should the trends shift.

Case Study 1

Is it acceptable for BestHotels.com to present fake statistics to influence users?

A. Yes (8%)

B. No (87%)

C. I don't care (5%)

 

Case Study 2

Has AnyVoice done enough to mitigate such scams?

A. Yes (18%)

B. No (73%)

C. I don't care (9%)

 

Case Study 3

Should ZT&C launch the downgrade campaign?

A. Yes (27%)

B. No (42%)

C. I don't care (31%)

 

Case Study 4

Should banks be allowed to sell such customer data?

A. Yes (8%)

B. No (92%)

C. I don't care (0%)

 

Case Study 5

A. Yes (66%)

B. No (30%)

C. I don't care (2%)

 

***

What do I make of these results?

First the limitations. They reflect the biases of my blog's readers. Also, these issues are worthy of a substantive debate that goes beyond a poll. I chose scenarios that I think are not clear-cut, allowing arguments from either side; the poll format forces readers to pick a side, and it appears on average each question (except one) has a definitely more popular position.

The most lop-sided response came for Scenario #4 in which data scientists purchase bank account data to develop "personalized" pricing. Over 90% of respondents don't want banks to sell their data for this purpose. This is also the only scenario in which almost everyone had an opinion!

People in the business already know that this scenario has already happened, and I am not aware that any American government agency has bothered to ask citizens whether they condone this practice. One agency did intervene to stop a merger between Visa and one of the key vendors of such data but that action isn't a pushback against selling bank account data.

Food for thought: can these data science models be justified because they advance social justice? One could argue that the pricing tool would hike prices on those who can afford them while lowering prices on those who can't.

***

Scenario #3 is also an example of personalized pricing: here, the data scientists create models that determine whether broadband subscribers are paying too little or too much. The responses are roughly split in three: respondents who have no opinion; those who want to notify premium subscribers that they can save money by downgrading their plans; and those who think the company should let the subscribers stay in their plans.

In this case, the business has all the usage data it needs to build the model, unlike in Scenario #4 when the best data for the job must be acquired from third parties. In Scenario #4, the business adjusts pricing for everyone while the overall change leaves the business with more revenues. In Scenario #3, the business definitely takes the action that increases its revenues, and is debating the other action that suppresses revenues.

***

Here is the link to the poll.

 

The hate campaign against negative reviews

A friend sent me this New York Times article about a "hate campaign" orchestrated on the book reviews website, GoodReads (acquired by Amazon a while back, no surprise), that allegedly suppressed sales of a new book.

I'm definitely a biased observer here since I have authored two books, and am affected by online book reviews. I also have emotional responses towards negative reviews, and like other humans, I exercise far less scrutiny of positive reviewers. It's a difficult issue to grapple with, and instead of offering conclusions, I want to suggest some topics for discussion.

(This is an example of a 1-star review on Amazon. When I wrote this post, the GoodReads site was malfunctioning and no reviews would load for me on multiple browsers. In my experience, GoodReads reviewers give fewer stars relative to Amazon reviewers.)

 

(A) The gripe that the negative reviewers published their opinions even before the book's publication date.

By itself, this is an invalid complaint, because publishers routinely send out advanced copies to friendly reviewers who are expected to help generate buzz prior to and leading up to the publication date. The real gripe is that they don't want to hear negative buzz, but if in an alternate universe, it's not a "hate campaign" but a "love campaign", I don't think this gripe would emerge.

Should there be an embargo on all reviews before the publication date?

The novel was published on June 6, 2023. The first review on Amazon was on June 6, 2023. Of the nine reviews posted on the first day, four were 5 stars, three 4 stars, and none below 3 stars. As of this writing, about 25% of the total reviews are below 3 stars. Who are the first-day reviewers? Should there be a reasonable X number of days to allow people to buy the book, read it, and write a review?

 

(B) The gripe that the negative reviewers objected merely to the outline of the book, and have not read the book itself.

According to the journalist, after the friendly reviewers put up a summary of the plot of the book on GoodReads, some unsettled readers wrote bad reviews because they hate the concept. This gripe is more valid. But... realistically, how many of the people who write reviews (including positive ones) have read the book? How much of the book must be read to be considered "qualified" to have an opinion? Did the famous people who put up blurbs on the book covers read the books first page to last?

The same question can be asked of not just early reviewers but any reviewers.

 

(C) The gripe that GoodReads do not ban these allegedly invalid reviews or reviewers.

The problem is how to set the censorship criteria. Should the authors and their publicists have the right to decide which reviewers or reviews are invalid?

Take, for example, the criterion that GoodReads should ban reviewers because they have not read the book. How is GoodReads going to know that? Do the publishers send GoodReads a list of names to whom they have sent advanced copies? What if one of the advanced readers pass the book on to someone else, who then write a review after reading the book?

If someone puts up a positive review after reading just the first 10 pages, is the review valid?

Take the other example of banning all reviews before the publication date. Surely, the publishers didn't mean that because they actively seek out advance praise. Emphasis on the word praise.

My own favorite censorship criterion is to delete all content-empty reviews. You routinely see reviews that have no value at all - "This ia a wonderful book", "This book is terrible. I hate it!" Having read the review, I have gained no appreciation for whether the reviewer read one page of it. But be careful what I reap! There are at least as many content-empty positive reviews as there are negative ones.

Here are two examples from Amazon:

Should online platforms censor content? This is a challenging question.

***

The journalists lamented that the book in question sold only 1,000 copies in the first 10 days, attributing the supposedly poor performance to the "hate campaign". Too bad the readers who purchased the book did not organize a "love campaign" on GoodReads.

 

 

NYT in story-first mode

NYT published a brain-dead article filled with statistical fallacies (link).

Since the Times has people on staff who know statistics, the real explanation for such an article is "story-first thinking". Data mined to support a story line.

***

The headline of the article provokes its least convincing statistic - that only seven black students were offered spots at Stuyvesant, one of the nation's top public high schools famous for STEM, out of 762 offers.

The position of the author is that the above statistic reveals systematic "racial and ethnic disparities". In statistics of education, "disparity" is a descriptive statistic describing a gap, which is different from "discrimination," which suggests foul play. The former is correlation while the latter is causation. This author mischievously used the term disparity while avoiding discrimination, even though the entire article presumes the disparity is unfair.

***

In fact, Stuyvesant (plus seven other similar schools in New York) only considers one factor in admissions, the student's score in the Specialized High Schools Admissions Test (SHSAT), and so the admission decision is not tainted by any judgmental criteria. All test-takers are ranked in order of test score, and then, each student is allocated a spot in their most preferred school - if all slots in the school have been filled, then the next preferred school is considered, and so on.

This allocation method guarantees "fairness" in the sense that if a student isn't allocated to a school higher on his/her preference list, it's because every offer for that school went to students who have scored higher on the SHSAT.

***

If we are interested in the question of whether the allocation of offers is fair across ethnic groups, the right question is: among students of similar ability (for example, score bands), do black or Latino students have a lower offer rate than whites and Asians?

This question of fairness pops in everywhere in education. In Chapter 3 of Numbers Rule Your World (link), I examined it in the context of designing fair SAT questions.

Surprise, surprise. Such data have not been released. Since there are no data, I'll explain conceptually how the numbers work.

***

Let's assume 30% of test-takers are Asians, and 20% each are blacks, latinos, and whites, and 10% others. Assume 50% of offers are given to Asians, 30% to whites, 5% each to blacks and latinos, and 10% to others. Overall, 15% of the test-takers receive an offer.

Given those assumptions, the offer rate for Asians is 25%, for whites it's 23%, for blacks and Latinos, it's 4%. Asians and whites are 1.5-1.7x more likely than the average to get an offer, while the offer rates of blacks and Latinos are a quarter of the average.

The NYT claims this proves that the process unfairly favors Asian and white students at the detriment of blacks and Latinos.

Not so fast. Remember all offers are completely determined by test scores, without any judgmental factors. So the proportion of students who receive offers is the same as the proportion who score above the cutoff score. Think of the cutoff score as the score of the last student taken up by a school. This cutoff score is an outcome of the offer process, not an input.

***

We can now split each subgroup into those who score above the cutoff and those below. Among Asians, the split is 25%-75%; whites, 23%-77%; blacks, 4%-96%; Latinos, 4%-96%; others, 15%-85%.

The reason why Asians and whites have more offers is because they have higher SHSAT scores.

Let's now insist that the offer rate for every ethnic group must equal the overall rate of 15%. For Asians and whites with a higher score distribution, this necessarily means that we cannot accept everyone above the cutoff score. It turns out that we can only accept about 60-67% of those who score above the cutoff score, and 0% of those who score below, in order to get to a weighted average of 15%.

For blacks and Latinos, we'd take 100% of those who score above the cutoff, but that isn't enough to get to 15%. We also must take 12% of those who score below the cutoff.

This is the unspoken math behind the NYT article. They are proposing: rejecting 40% of high-scoring Asians and whites while accepting 100% of high-scoring blacks and Latinos; and accepting 12% of low-scoring blacks and Latinos while rejecting 100% of low-scoring Asians and whites.

Shame.

 

 

Covid vaccines contain PEDs, according to an athlete

Another sport has the doping demons. News arrived that U.S. triathlete Collin Chartier has been banned for three years after testing positive for EPO, a favorite performance-enhancing drug for endurance sports.

Doping is featured in my book Numbers Rule Your World (link), written before Lance Armstrong finally admitted to doping after vehement denials for years. I made several surprising observations. Firstly, if you or I are running an anti-doping lab, we'd calibrate the test to reduce false positives at the expense of false negatives, since a doper who is getting away with it isn't going to expose our error but a falsely accused clean athlete will surely erode our reputation. Secondly, and consequently, we expect most positives to be true positives. Thirdly, a high proportion of dopers are getting away with it.

To Chartier's credit, he did not invent excuses but admitted to the offence. As reported in the article, other triathletes have paraded out the usual unlikely excuses - one even claimed that a "tainted burrito" and a "tainted Covid-19 vaccine" were responsible for the positive tests.

Now, if we believe Chartier (and many other athletes who got caught doping), he only took EPO once and only after the two greatest wins of his career. He also claimed that no one else but him knew about the doping, and his coach, who is related to the world champion of the sport, said he was blindsided by his own athlete.

***

I'm still waiting for the day when an athlete shakes up the establishment by admitting all the negative tests s/he got while doping. That would take courage. Wait - Armstrong frequently said he was the most tested athlete on Earth while he was getting away with it. Does that count?