Big Data, Plainly Spoken (aka Numbers Rule Your World)

Visible and invisible errors, and the nefarious power of suggestion

In Chapter 4 of Numbers Rule Your World (link), I explain how predictive models make errors. One of the key ideas in the chapter is the visibility of errors. Modelers are much more likely to care about visible errors than invisible errors. Most classic discussion of predictive accuracy ignores this important issue, and assumes that all errors, and all types of errors, are visible. In real life, many errors are invisible, or can be deliberately hidden from users.

Consider the failed child-abuse prediction model used by Chicago and described in a previous post. The errors made by this model are highly visible, and frustrating to case workers. This is why this scam was exposed. Case workers have had to work through thousands of cases of false positives. False negatives are equally visible, in the form of at-risk children who were not flagged for investigation.

Errors by auto-correction software are visible and offensive like mosquito bites. But we mostly react to the false positive errors, when a perfectly fine word has been changed to the wrong word. It's less noticeable if the software misses a wrong word or a typo. Not surprisingly, such software gets a bad rep.

***

By contrast, lots of predictive algorithms generate errors of prediction that are invisible to users.

For example, many drivers use Google's Waze for navigation. Waze gives predictions of how much time the driver will save by following its route. There is no way for the driver to measure whether that prediction is accurate.

I highlighted in my book - anti-doping tests to catch cheating athletes. If the test commits a false negative error, you bet the cheating athlete will not call a press conference to taunt the testing labs for their big mistake. If the test gives a false positive, however, the pleas of innocence would be deafening.

***

Sometimes, it takes expertise to notice the errors. Take Google Translate. If the target language is completely unknown to the user, the user can't tell if the translation is good or not.

Rainfall forecasts usually come as probabilities - 30 percent chance of rain. It takes quite a bit of effort to validate the prediction of 30-percent chance; someone who's not a scientist is unlikely to have to tools to do it.

These errors can be visible but usually stay in the shadows.

***

Errors tend to feel worse if they are visible. Invisible errors present opportunities for mischief.

Take Waze for example. At the start of your journey, Waze will proudly announce "I have chosen the most optimal route for your trip. You will save 8.3 minutes by following this route." Drivers have no way of verifying whether (a) the route suggested is indeed the most optimal or (b) whether this route saved 8.3 minutes relative to some other route or 10.5 minutes, or 2.4 minutes, or any other number. But many drivers are convinced that they have saved time. The uncorroborated perception of time saving demonstrates the power of suggestion. By contrast, the old-school GPS such as TomTom or Garmin does not make claims - to their detriment.

Such suggestions only work if the errors are invisible. When the developer of the child-abuse prediction model for Chicago tried the same trick as the developer of Waze, they ran into trouble. The Chicago Tribune reported that the predictive model generated alerts to social workers of the form: 

Please note that the two youngest children, ages 1 year and 4 years have been assigned a 99% probability by the Eckerd Rapid Safety Feedback metrics of serious harm or death in the next two years.

It's rather easy to disprove this statement because the errors are easy to spot. Eckerd has agreed to tone down the language.

 

Overselling predictive models (Hint: future performance matters)

Those who have read my books or taken my courses would be the least bit surprised by the news that just came out about the naked emperor of Chicago's much ballyhooed predictive algorithm for child abuse. Chicago Tribune has the story (link).

Key points:

• Data scientists (over-)sold an algorithm to predict children "at risk for serious injury or death."
• The director of the government agency that purchased this product concluded after two years that "[the model] isn't predicting much."
• At least $366K were spent i.e. wasted on this program. This number most likely does not account for the costs of support staff and infrastructure and the actual waste described below.
• "More than 4,100 Illinois children were assigned a 90 percent or greater probability of death or injury. And 369 youngsters, all under age 9, got a 100 percent chance of death or serious injury in the next two years" [a false positive problem - not to bring up a sore subject but several infamous outlets proclaimed that Hillary Clinton would win the Presidency with 90% or higher probability!]
• "high-profile child deaths kept cropping up with little warning from the predictive analytics software; Predictive analytics (wasn't) predicting any of the bad cases" [a false negative problem]
• Selling the dream: "If it is possible to use big data to spotlight a child in trouble and intervene before he or she is hurt, then doing so is government's moral obligation, advocates for the technology say."
• The algorithm, despite being paid for by public funds, is hidden from public view but the Tribune report pieces this together: "Eckerd [the vendor] retrospectively analyzes thousands of closed abuse cases and from them draws data points that are highly correlated with serious harm. The parents' ages could be a factor — or their previous criminal records, evidence of substance abuse in the home, or the presence of a new boyfriend or girlfriend."

The story does not invalidate the practice of predictive analytics but points out pervasive problems within the industry. Most importantly, we are over-hyping this technology. I cover the statistics of such models in depth in the marketing chapters of Numbersense (link). Predictive models are impressive in a relative sense (in marketing, we use the term "lift") but not usually in an absolute sense. Every social science based predictive model I have seen misses a sizeable number of targets while falsely labeling lots of "cases".

When a vendor does not report on accuracy metrics, or when such metrics are not interpretable in practice, or when the metrics are generated by third parties, or when metrics are computed purely from historical data, one has to be very careful about separating the good products from the scams.

***

It bears repeating that the "big data" used in this type of analysis has multiple complications which I collectively describe as OCCAM.

Observational - we cannot run an experiment and assign people to have criminal records, or other characteristics at random, so it is tough to get any read on causal mechanisms. Thus, there is a good chance to suffer from spurious correlations.

Seemingly Complete - but not really. Many cases are unreported and not in the training data. One of the excuses used by Eckerd when confronted with inaccuracy is that certain children are not even scored because they do not appear in the data. That is technically correct but does not change the fact that the model did not predict those deaths.

No Controls - many predictive algorithms do not use controls explicitly. Another issue is that the possible controls all come from cases reported to the authorities. Without a doubt, if we take the profiles of the at-risk children as determined by the algorithm, we will find many other children in the general population that do not show up in the database at all.

Adapted - the data collection process was not designed specifically to support predictive modeling.

Merged - various datasets get merged during the analysis, which introduces errors.

More discussion of OCCAM here and here.

***

While I appreciate that the Chicago Tribune wrote and published this article, this is yet another media report on predictive modeling using "big data" that does not contain any data or quantified metric of predictive accuracy.

The two chapters most relevant to this post are Chapter 4 in Numbers Rule Your World (link) and Chapter 5 in Numbersense (link).

 

Primer on the limitation of current AI

Today, I turn attention to another Technology Review article in the same special AI issue, the important article by Rodney Brooks on the limitation of the current AI systems, driven by the "deep learning" "revolution." Brooks was the head of MIT's CSAIL lab, and an expert in AI who has lived through several waves of AI hype.

Brooks frames his content differently, titling the piece "The Seven Deadly Sins of AI Predictions." He opens with the breathless forecasts published by various media outlets, predicting that AI (computers) would wipe out the job market for human beings in a broad range of industries, in some cases, in a matter of 10 or 20 years. He then describes seven fallacies that have led these hype-meisters astray. Read the whole article here. Some of these dire forecasts: Oxford/Yale, McKinsey. [PS. I do not understand why those technology leaders who claim to believe in these forecasts do not immediately stop their AI programs for the public good.]

Brooks calls these predictions "ludicrous." He points out that there have been zero realistic demonstrations of robots that can take over grounds and maintenance work even though the forecasters claim that 90% of such jobs would disappear in 10 to 20 years. For the rest of us, who are not knee-deep in AI research, it's simple to validate Brooks's viewpoint: just make a call to your favorite service provider (your bank, your credit card provider, your health insurer, for example); subject yourself to the AI "chatbot" for even five minutes. 

In debunking the hype, Brooks outlines several key limitations of the current AI systems, built on the deep learning revolution (discussed in the prior post). 

The current AI is task-focused and narrow in scope

Computers can be trained to tag images with labels like "people playing Frisbee in a park" but we can't then query these computers with related questions such as "what is the shape of a frisbee?", "Is today's weather suitable for playing frisbee?", "Roughly how many people play frisbee at once?", etc.

This fallacy is a type of over-generalization. The fact that a computer can perform a specific task is used to imply a general competence. Such generalization makes sense for a human being (say, asking the above questions to someone who's playing in the park) but it fails when applied to a computer.

This is a significant limitation. This is one of the reasons why the chatbots are so annoying. They can only understand a small set of words. If you try to use any word outside that set, the AI computer is stumped. If you explain your problem the way you talk to a human being, it is confused. If you provide use only words from that set of words it knows but in a sequence different from what the chatbot expects, it rejects you. 

The current AI cannot adapt

The much-heralded AI success stories, IBM Watson and AlphaGo, playing chessJeopardy and Go respectively, cannot deal with any rule change while the human player can immediately. Recall the key advance in this AI revolution is the engineering feat of processing huge amounts of training data (40 million pictures of hot dogs to recognize a hot dog). The training data used by Watson and AlphaGo consist of historical game logs. If the rules change, all those logs become useless, at best compromised. It would take a very long time to collect enough data under the new rules. 

The current AI does not "learn" the way humans do

This is one of my pet peeves. The word "learning" as used in "machine learning" does not mean anything like what it means in human learning. Brooks cites Marvin Minsky who coined the term "suitcase words." Learning means so many different things that it is meaningless.

The key quote: "machine learning is very brittle, and it requires lots of preparation by human researchers or engineers, special-purpose coding, special-purpose sets of training data, and a custom learning structure for each new problem domain."

Let me parse this for you. All of the AI gains are built from Olympic amounts of human efforts. For example, IBM's Watson is nothing if human curators had not gone to heroic lengths to structure and organize data in Wikipedia, encyclopedias, etc. Every blog post used in these databases (including this one) represents one or more human beings assisting the computers by organizing and structuring the text. The day that humans retire from updating blogs, encyclopedias, Wikipedia, etc. is the day that these computers with their machine learning grind to a halt. If you love machine learning and AI, please stop for a moment and give thanks to all the human beings who have assembled the training datasets for these computers.

Then, also don't forget about the engineers who wrote the "machine learning" software, and fine-tuned these systems.

The "special-purpose coding" is the little dark secret in the automation business. It turns out that these machines aren't that smart - they make lots of errors. Frequently, these errors are detected (by unhappy customers), and they can be patched - but usually in not-very-elegant ways by "special-purpose" code.

The case of Google's search algorithm is instructive. The core idea of pagerank is clean and mathematically simple and it tackles a lot of searches accurately. But there are still a sizable set of searches that don't return good results. Google has a whole team of engineers who work on search improvement. According to Wired, "the improvement process is a relentless slog, grinding through bad results to determine what isn’t working." They then deploy additional code to solve those specific search problems.

The development model is suitable for only some problems

Much of the AI software is created using the agile development paradigm of "fail often, fail fast" that have driven web development in recent times. As Brooks points out, this paradigm relies on "the marginal cost of newly deploying code being very, very close to zero." In most industries, the cost is not zero. There are lots of legacy systems that are still functioning. Replacing them will surely cause a disruption in the business processes.

On a related note, the fail-often paradigm only works in industries where the cost of failing is very, very close to zero. Many of the proponents of this paradigm give away their software "for free" so that users are not really customers. Typically, the cost of errors is zero when the value of the product is zero. When it comes to health care or self-driving cars, the cost of errors is >>>> zero. 

The jump from recognizing the object in a photo to diagnosing cancer from an image is not straightforward. Brooks cautions against magical thinking and also "exponentialism," the assumption of exponential growth far into the future.

Read the article in full here. This one is also worth your time.

 

 

 

 

A rare article about the AI revolution worth your time

MIT Technology Review has several fantastic articles in a special issue about the AI hype that has consumed the data science world.

I recommend the piece "Is AI Riding a One-Trick Pony?" by James Somers (link). If the question is to be taken literally, the answer is yes. The bigger question is whether this pony is taking us to the promised land, or a holding spot. As Somers stated, "AI today is deep learning, and deep learning is backprop." Backpropagation is a concept embedded in the algorithm of deep learning, and it was invented 30 years ago in Geoff Hinton's lab. The key to its recent ascent is the availability of much larger data sets, and the ability to do more computations faster. This rebirth of AI is a triumph of engineering rather than an intellectual advance.

Here are several other provocative quotes from the article:

• maybe we're not actually at the beginning of a revolution. Maybe we're at the end of one.
• there's a sort of reality distortion field that Hinton creates, an air of certainty and enthusiasm, that gives you the feeling there's nothing that vectors can't do... it's only when you leave the room that you remember: these "deep learning" systems are still pretty dumb
• a deep neural net that recognizes images can be totally stymied when you change a single pixel, or add visual noise that's imperceptible to a human
• to get a deep-learning system to recognize a hot dog, you might have to feed it 40 million pictures of hot dogs. To get [a two-year-old baby] to recognize a hot dog, you show her a hot dog
• the latest sweep of AI has been less science than engineering, even tinkering
• we're still largely in the dark about how those systems work, or whether they could ever add up to something as powerful as the human mind

One of the hot subjects of the moment is the claim that these neural networks can learn "representations." This is really a half-truth, and much more work is needed to prove such an audacious claim. Somers writes the following passage, which is similar to the stories about  representational learning being passed around at meetings: "so-called vector arithmetic makes it possible to... subtract the vector for 'France' from the vector for 'Paris,' add the vector for 'Italy,' and end up in the neighborhood of 'Rome.' It works without anyone telling the network explicitly that Rome is to Italy as Paris is to France."

This vector system is a representation of the raw data in latent dimensions. The general concept of latent variables has been central to statistics and psychometrics for a long time. Think about any IQ or personality tests. You are asked a series of questions. Your answer choices represent the raw data. But the researcher isn't really interested in the responses to specific questions. The questions are designed to measure latent (hidden) dimensions such as your analytical ability, your extroversion, etc. Each question sheds some light on some subset of latent dimensions. Your score on, say, extroversion is some weighted average of your raw answers to various questions. Using your answers, the researcher can place you in the universe defined by those latent dimensions. In the example concerning country capitals, words are being mapped onto a universe defined by latent dimensions.

The interpretation of such dimensions has always involved human beings. So is the case here. The neural network itself has no idea what Rome or Paris or France or Italy is; it's just another word for all it's concerned. It's the human interpreter who imposes the meaning upon the output of the neural network. It's the human who selects those word pairs because he/she knows the correct relationship between them.

Read Somers's article here.