Big Data, Plainly Spoken (aka Numbers Rule Your World)

Getting serious about AI

Finally, we have a serious report on the potential harms of AI that is recommended reading (link, paywall).

***

Here are five things about AI harms that I think everyone should know.

1. The press is typically focused on the wrong harm

Perhaps misdirected by industry heavyweights, the press frequently waxes about the potential for AI to "kill us all." But it is mundane harms, such as being robbed $1,000 by an anonymous person across the ocean, that most of us will likely be victims of.

2. Don't forget the term "deepfake"

A few years ago, the press warned us about the harms of "deepfakes." These days, not so much. In fact, one has to dig pretty deep into the Financial Times article to find a mention of "deepfake." At least they used the term, which is a pleasant surprise. What the industry is now calling "generative AI" is the same thing as what was labeled "deepfake".

The name change switches our attention from the harmful use cases to the beneficial ones. But a name change does not reduce the potential for AI abuse. Rather than ameliorating it, the potential for harm is scaling up as quickly as the underlying technology is developing.

 

3. It's two sides of the same coin

It's wonderful to imagine a world in which we can deploy AI that gives us benefits without also inducing harms. But it's a mytical world that won't exist.

Take the simple example of 1-click shopping, for which our credit card details must be stored. For sure, these technologies make it much more convenient for online shoppers, and delight retailers as they reduce the friction of buying. But they also open people to hacking, stolen identities and scams.

If our account details weren't saved on the servers of these online retailers, then they could not be stolen. Not saving such data also destroys the value of 1-click shopping.

 

4. Reducing AI harms through more AI is a dead end

I laugh every time someone claims that the solution to AI abuse is more AI. This idea requires AI to predict accurately which outputs came from AI.

An example would be technology that tells professors which students submitted essays written by ChatGPT. The AI harm reduction technology reads an essay and computes the probability that it was written by an AI not a human.

But... we are looking at one side of the coin. What's on the other side? People - e.g. newsrooms, banks - are looking to use the same AI technology as the students to write articles, replacing human writers. The more successful are these use cases, the less predictive is the AI harm reduction technology.

Remember that the ultimate goal of AI is to equal or exceed human ability. Let's pretend we have an AI that is effectively equivalent to humans. How is it possible for another AI to predict which output came from AI and which from humans?

 

5. AI developers can reduce AI harms today (but they don't)

Try subscribing to ChatGPT. You will find that you can't do so without disclosing your identity. They won't accept a Google Voice number, they absolutely want your mobile number, which completely identifies you to them.

The main reasons why ChatGPT wants your personal data are commercial.

However, we know that they know who we are. Now, consider the professor who wants to know if someone used ChatGPT to write an essay.

You don't need any AI to make a prediction. ChatGPT knows for sure because ChatGPT has the personal data of the student. So if ChatGPT is serious about reducing this type of AI harm, it can absolutely do it. This method works even if the AI evolves to the point where it generates essays indistinguishable from those by human writers.

Why won't they pursue this path? I'd let you all ponder this.

On the evidence raised by SAT distractors

After reading my prior post, long-time reader Mark (link) alerted me to a series of posts he wrote 10 years ago on the anti-SAT movement. This is a topic that bubbles up every few years, and it appears that the current generation of college administrators may go where their predecessors didn't. Some colleges have started to drop the SAT requirement for admissions.

Post 1, Post 2, Post 3, Post 4, Post 5

Let me summarize Mark's various posts and add some color commentary.

***

In his first post (link), Mark explores the relationship between SAT scores and college GPAs. This exercise in analyzing correlation is meaningful to both sides of the debate. The distractors detractors argue that "there already is something that accurately mirrors the grades a student gets in school. Namely: the grades a student gets in school" while proponents say SAT scores are the best predictors of college success.

For example, when MIT reinstated the SAT requirement after the Covid-19 pandemic (link), the dean of admissions stated:

Our research has shown that, in most cases, we cannot reliably predict students will do well at MIT unless we consider standardized test results [especially the math component] alongside grades, coursework, and other factors.

Mark points out that the right measure of any predictor is incremental insight – what additional nuance it adds to the collection of predictors. This explains why the SAT looks more like a general intelligence test than a high-school exam. Otherwise, the SAT score would be an unnecessary duplicate of high-school GPA. (Why the previous sentence is stated forwards but not backwards is another sign of weak analytics in this space.)

Mark's point is self-evident: assume that high-school GPAs predict college performance just as well as the SAT, as the distractors detractors suggest, then SAT scores should mirror high-school GPAs, and if they did, the two metrics would result in identical rankings of the students; if so, what's the cause for complaint?

***

The second post (link) examines various reasons SAT critics have given for dumping the test. Mark is unimpressed by the quality of the evidence, and in many cases, its absence.

The weirdest argument is that according to Google's description of its own hiring practices, Google does not consider standardized test scores or college ranking. Mark remarked that "Google doesn't talk much about SATs at least in part because they've largely maxed out the metric."

Let's say we tally up the background data of Google hires; we are likely to find a good number of Ivy League graduates with high SAT scores. Can Google legitimately claim their hiring does not consider test scores or college ranking?

Sure, someone is going to say "Those people weren't hired because they graduated from an Ivy League school, or that they had high SAT scores. They just happen to have those characteristics. Correlation is not causation!"

Well, admitted students to top colleges not only have high SAT scores, but they also have high GPAs, and other great things on their applications so did they get accepted because of the SAT scores, or the high GPAs?

***

The third post presents a series of math questions from actual SAT exams because Mark realized that many critics of the SAT attacked an imaginary scarecrow version of the SAT test, rather than what the test really is. Seeing these real questions helps ground the debate.

He also touched on one virtue of the SAT test, which is its transparency. Past exams with answers are readily available. By contrast, transparency is definitely missing in high-school GPAs. Nevertheless, in real life, transparency frequently backfires. Where are these SAT critics when it comes to high-school GPAs? Let's compile GPAs and split them by race - I'd be surprised if they didn't find a race gap. The reason that the race gap in GPAs isn't in the news is that schools don't release such data.

***

The fourth post is a must-read. Here is Andrew's recap of what Mark wrote.

Mark trashed those critics who grumbled about the point deduction for wrong answers in the SAT. Mark explained why the point deduction scheme in fact simplifies life for test-takers, because the penalty makes it counterproductive to do "random guessing" (what Mark describes as "guessing without first reading the question"). Tests that don't deduct points for wrong answers are more annoying because smart test-takers should guess on certain questions based on their level of confidence in knowing the answer.

The unspoken point behind Mark's post is that the SAT is rigorously designed, and continuously improved, by lots of statisticians, and therefore it is the best instrument we have. In Numbers Rule Your World (link), half of Chapter 3 is devoted to the statistical work behind a niche issue in SAT tests, work that reduces specific biases due to race and other characteristics.

The biggest irony in the anti-SAT movement is that on the major complaints about the SAT, alternative instruments such as high school GPAs and reference letters are far worse.

***

In the last post of this series (link), Mark turned our intuition upside down. He argued that the students most likely to benefit from the SAT admission requirement are those good students who go to public schools – or if we extend this line of reasoning, good students from underprivileged backgrounds. That is to say, the SAT requirement benefits precisely those people whom its distractors detractors say they care the most about, with the caveat that the benefit accrues to the cream of that crop.

Ten years later, the MIT dean of admissions echoed Mark's point when he said that MIT uses the SAT score to identify disadvantaged students who are most likely to succeed at MIT.

unlike some other inequalities — like access to fancy internships or expensive extracurriculars — our empirical research shows the SAT/ACT actually do help us figure out if someone will do well at MIT.

***

This last discussion leads us to some general topics in the area of statistical measures. No metric is perfect, most metrics are correlated with each other, all metrics promote perverse behavior, alternatives often appear better only because of lack of data or analyses. See Chapter 2 of Numbersense (link) for more on measurement, where I dig into the anti-BMI movement. BMI is the widely accepted metric for assessing obesity; like SAT scores, the metric has attracted a small loud group of distractors detractors.

 

Ryanair fights with its retailers

This story (link) about Ryanair's rift with several online travel websites caught my eye. The outline of the story reads like a typical business dispute, the likes of which U.S. cable TV consumers have regularly experienced, such as when Disney channels were taken off the Charter network (link). In Ryanair's case, the back story is fascinating, bringing out the contradictions and tensions in the modern tech industry, and that's what I want to discuss today.

***

Ryanair is a budget airline based out of Ireland, one of the largest airlines in Europe. A few online travel sites (Booking, Kayak, Kiwi) have recently suspended selling Ryanair tickets. This action escalates an ongoing feud between the two companies. Ryanair is unhappy that online travel sites "trick" customers using an impossibly low fare, and later tag on extra fees. In addition, Ryanair won an injunction to stop a third party called Flightbox from scraping pricing data from its website. Flightbox supplies the scraped pricing data to online travel sites. Ryanair is also annoyed that these online vendors hide customer email addresses from the airline, and has countered by launching a "Know Your Passenger" initiative.

The removal of Ryanair's listings may have negatively affected Ryanair's sales during this past holiday season, although only slightly, while the dispute is not expected to negatively impact Ryanair's profit during the quarter.

That's a lot of grievances to unpack.

***

In the air travel industry, airlines are the "manufacturers", online travel websites are the "retailers", and passengers are customers. Online websites collect commissions for selling air tickets from the airlines to the customers. The airlines also sell directly to consumers, setting up competition with online travel websites. Such competition is routine in digital commerce; many manufacturers have direct presence on Amazon, for example.

Online travel websites such as Travelocity, Kayak and Expedia have been around for many years. At first, they pitched themselves as "disintermediating" the industry. Travelers would be better off buying plane tickets online, cutting off the "intermediaries", who were travel agents relying on airline commissions. In the late 1990s, Expedia started out as a do-it-yourself travel planning tool inside Microsoft that refused to serve travel agents.

Eventually, it emerged that these startups did not so much remove intermediaries as take their business. These online travel websites are now the ones earning commissions from airlines. When we purchase these sites, we aren't buying directly from manufacturers. Similarly, Amazon has displaced big-box retailers by becoming the mega-big-box retailer. Square or Apple or Google has not disintermediated payment processors; they just substituted previous players, and worse, in some cases, as when a website processes Visa transactions through Square, the new players inserted another middle layer that collects even more intermediary fees. Many of these websites also now serve their former adversaries: some brick-and-mortar travel agents use these websites to serve their customers. Many real estate agents list their short-term rental properties on AirBnB.

The disintermediation story never made any sense since the intermediate layer fulfills some functions, e.g. comparison shopping, and search tools, that aren't possible when a customer buys directly from an airline's website. Running websites that provide such services costs money.

***

These intermediary businesses go to unsavory lengths in pursuit of revenues. Some years ago, mom-and-pop restaurant owners that use Grubhub were surprised to find "fake" websites and phone numbers created in their names by Grubhub. When potential diners search for their favorite restaurants, Google sends them to these Grubhub websites, so their orders are effectively intercepted and rerouted, allowing Grubhub to charge restauranteurs tolls. (More on this here.)

Typically, these websites justify their fees (as high as 30%) by claiming they generate sales leads for restaurants. They claim credit for every transaction conducted on their websites. Such credit attribution policies confuse correlation with causation. It produces a perverse incentive: the easiest way to make a buck is to redirect traffic from the manufacturers to themselves.

Of course, some of the transactions would not have happened without Grubhub but what proportion?

The Ryanair affair has shown that the influence of such intermediaries is much weaker than claimed. After Ryanair listings disappeared from those travel websites, Ryanair's bookings only dropped by 1 or 2 percent. In other words, almost all of their customers found their way to Ryanair's website and made their purchases there.

Since those diners searched specifically for their favorite restaurants on Google, most of them would have ordered directly had they not been diverted to Grubhub.

***

The other two complaints are salvoes in an ongoing information warfare. I previously wrote about the web scraping business here.

Flightbox is a company that scrapes data on behalf of its clients, such as Booking.com. Since web scraping is in a legal gray zone, large or public companies may attempt to shirk legal liability by outsourcing such work to third parties.

An Achilles heel of any online business is that they expose their pricing data to the entire world. Previously, one has to be physically present in a store with pen and paper or mobile devices to collect pricing data – it was very expensive, even impractical, to get complete data on all stores and all products. That all changed.

Any economist can tell you information has value: in any negotiation, or trade, or power play, the side with the better information has better odds of winning. Businesses know this well, which is why they suck up any bit of information they can find. Imagine the advantages Target can gain against Walmart if Target knows Walmart's pricing on everything.

Almost all online retailers pronounce that scraping data from their websites is against the rules. You can typically find such provisions in the terms and conditions which they know only a select few read. Flightbox and other web scrapers consciously ignore such provisions. Thus, Ryanair went to the court to obtain an injunction. Not surprising because the online travel websites are using the scraped data to compete with Ryanair's direct sales efforts.

I won't take sides in this dispute as I'd be shocked if Ryanair's data scientists aren't scraping data "illegally" as well. As you may have heard, ChatGPT is being sued also for "illegally" obtaining data to train their large language models. Further, unless the courts severely penalize web scraping, the online travel sites will simply hire something else to continue scraping.

The tussle over email addresses is also part of this information warfare. I'll address it in a future post.