Big Data, Plainly Spoken (aka Numbers Rule Your World)

The absurdity of predictive policing, part 2

This is a follow-up post to Part 1. Read that post first to catch up. In Part 1, I set up a scenario, in which 1,000 people are flagged by a predictive-policing model and sent to jail for their future crime of burglary.

At the end, I asked how many of the 1,000 people were accurately flagged by the predictive model.

***

At the crudest level – and this is the argument made by those who sell predictive policing technologies, the model was 100 percent accurate. Look, it flagged 1,000 people, and subsequently, none of these people committed burglary! Amazing! Genius! Wow!

The absurdity of this calculation is that such perfection could have been obtained with any model – even a random model. If I randomly selected 1,000 people from the list, and convinced the courts to impound them because of their future crimes, those 1,000 would also not commit burglary. Therefore, the fact that the predicted burglars didn’t commit the crime proves nothing about the model’s effectiveness! It just states the obvious fact that if we lock someone up, that person couldn't commit burglary.

For each person flagged by the predictive model, the relevant question is: if we didn’t throw this person in jail (or take other actions that would alter the outcome), would s/he have committed burglary? If the answer is yes, then the model has made one accurate prediction. The trouble is it's counterfactual. Taking action using this model removes our ability to measure its accuracy.

***

In practice, a few strategies can be deployed to assess such a predictive model.

Strategy 1 is back-testing. Apply the model to historical data. Train the model using data before a cut-off date. Use it to make predictions after the cut-off date. Run time forward, and compare the burglary rate between the flagged and unflagged populations.

This is easy to do but fraught with dangers. It only works if the modeler does not cheat by using data after the cut-off date. The most common way of cheating is to tinker your model after learning that it isn't accurate enough.

Since every analyst building models tinkers the model upon learning it isn't accurate enough, every analyst "cheats". (Textbook authors, though, have memberships to the private clubs where analysts only use their test sets once.) Thus, basing any decisions purely on back-testing is foolish. Buying predictive technologies from a vendor who only reports back-tested results is foolish.

Strategy 2 is dry run. Run the predictive model, flag the predicted offenders but take no action. After sufficient time, look at flagged versus not flagged and the burglary rates between the two groups. If the model is any good, the burglary rate amongst those flagged should be substantively higher.

People who don't want you to know about accuracy counter with these objections: (a) it takes too long (depending on the burglary rate of the neighborhood, this could well take 6-12 months to collect enough cases); and/or (b) you're letting crime happen while this experiment is running - this critique is usually delivered with a contempt for science: "you're not being practical, you're being a scientist!".

Reporting results from a dry run builds confidence in the accuracy of the predictive models - on average. It still doesn’t change the fact that for any individual flagged and arrested based on the model’s prediction, we would never know if the model was correct.

***

Further reading:

Chapter 4, Numbers Rule Your World: discussion in the context of predicting terrorists (link)

Chapter 5, Numbersense: discussion in the context of Target's pregnancy prediction model (link)

The absurdity of predictive policing

Big (surveillance) data is changing everything - that is undeniable. But is Big Data changing everything for the better? That is what you have to think about.

As someone who's done data analytics for a long time, I can say confidently that the potential for harm is at least as large, but probably larger, than the potential for good.

A case in point is data as related to crime and policing. This post picks up from the perceptive article in the New York Times, "How Reporting Practices can Skew Crime Statistics" (link).

The article points out something really important - data is as data is defined. Different definitions result in different analytical results, frequently contradictory. No one should interpret analytical results without first learning the data definitions.

The author describes two extremely popular styles of "big data" analytics: top N ranking, and trend analysis. In crime statistics, sample insights are (1) "South Bend, Indiana is the 27th worst city for violent crime rate among all cities with over 100,000 people." and (2) "Violent crime has worsened in South Bend since mayor Pete got elected in 2012."

With the volume of data available today, those two simple analyses still account for the vast majority of "analytics" out there.

South Bend, Indiana is the 27th worst city for violent crime rate among all cities with over 100,000 people.

Here is a list of definitions that affect the interpretation of this statement, most of which are described in the NYT article:

• whether the suburbs are included as part of the "city", or more broadly, how the boundaries of the city are drawn. This matters because the crime rates in the suburbs are different from the crime rates in the city centers.
• whether the error in the count of population is comparable across all cities under consideration
• what is considered a crime
• what proportion of crime is reported
• what crime is classified as "violent"
• are there any exceptions

You might think the population of a city is an objective fact but it isn't. Definitions matter, and they matter a lot when you're comparing different cities.

Violent crime has worsened in South Bend since mayor Pete got elected in 2012.

In addition to the points listed above, we also need to know if any of those definitions changed between 2012 and 2019. Changing how something is defined almost always breaks the continuity of any analysis, thus undermining any trend analysis. I raised this point in my book Numbersense (link) in the chapter on obesity - if the medical community changes how we measure obesity, then all historical data are rendered useless so such proposals must be considered using great caution.

Further, we should find out whether violent crime was increasing or decreasing prior to 2012 in South Bend, whether violent crime was increasing or decreasing in other cities comparable to South Bend since 2012, whether violent crime was increasing or decreasing in the nation as a whole since 2012, etc.

***

Some readers of my book Numbersense (link) might be mystified why I spent so much energy discussing definitions - of grad school admissions statistics, of U.S. News ranking, of obesity measures, of unemployment rates, of inflation rates, of fantasy football metrics, etc., etc.

A key to numbersense - to judging the statistics and data thrown at you - is to understand clearly how the data are collected, and in particular, how any metric is defined.

The discussion of crime statistics by the New York Times makes this case cogently.

***

Now let me get to the absurdity of predictive policing.

A particularly popular and cited application of Big Data "for social good" is predictive policing. The goal of predictive policing is to prevent crime from happening.

The ideal of "Minority Report" is often hoisted on us. Wouldn't it be great if some AI or machine learning or predictive model can figure out who will be burglarizing your home next year, and have the police lock the would-be burglar up before s/he could commit the crime?

I have done a good amount of predictive modeling in my career so I am not saying what I'm saying because I'm a luddite. It's because of that experience that I am leading you down the following rabbit hole:

There are not many burglaries. According to Statista, the worst state (New Mexico) suffered 770 burglaries per 100,000 residents in 2018. This means that if the analyst is given a list of 100,000 New Mexicans chosen at random, a perfect model should label 770 of those as potential burglars. That is to say, we want the model to pick out 0.77 percent of the list.

A perfect model does not exist. Any model generates false positives. Because burglars are rare, any reasonably predictive model would need to red-flag many more than 0.77 percent of the list to have any chance of capturing all 770 potential burglars. Let's say the model points the finger at 1 percent of the list. That would mean 1,000 potential burglars. Since there should be only 770 burglars, the first thing we know is that at least 230 innocent people would be flagged by this model.

Next, think about what happens to the 1,000 flagged people. In the ideal of predictive policing, police would knock on their doors and arrest them for their future crimes. These people who haven't burglarized anyone would be duly jailed by courts who buy into predictive policing. (How would they defend themselves against an allegation of future crime?) 

We could say with certainty that those 1,000 did not subsequently commit burglary - given that they were incarcerated.

So humor me, and try to answer the following question: Of the 1,000 people flagged by this model, how many of these were accurately predicted?

 

 

 

The Facebook data privacy settlement fails to impress

The FTC settled with Facebook on data privacy.

In the last year, news kept coming out about how the social media giant mishandled user data, including taking phone numbers collected for two-factor authentication and using them for marketing, saving passwords unencrypted, and asking users their passwords to other services. Despite the $5 billion fine, the settlement has been widely panned by the technology press as a slap on the wrist.

The size of the fine is not meaningful in the context of Facebook’s massive advertising business that generated $16 billion revenues in the 2^nd quarter of 2019 alone.

Even more frustrating are the so-called new regulations, which fail to address any underlying issues.

Consider the requirement that Facebook ask users to “opt-in” to facial recognition. Requiring opt-in has long lost any meaning because of the forced consent tactic. Facebook can simply put up a notice saying that you will not be served unless you consent to the following activities. This kind of notice is prevalent with mobile apps - to use these apps, you must agree to a list of demands.

Consider the requirement to encrypt passwords. This is standard IT best practice branded as regulatory success.

Consider the requirement to bar third-party apps that “fail to certify that they are in compliance with Facebook’s platform policies”. This is a reiteration of existing policy. The FTC does not stipulate any restriction on types of use. The issue is intractable since once the data moves to third-party servers, Facebook has no ability to monitor them.

As the following articles show, few are impressed by this settlement.

Business Insider: List of regulations

Vox: Facebook FTC settlement

Bloomberg: Privacy settlement won't hinder its ad business

Mashable: How FTC let Mark Zuckerberg off the hook

Techcrunch: 9 reasons the settlement is a joke

DUIW 420: offering up 20 paper ideas pre-approved for prestigious journals

First, you have to read till the end for the 20 paper ideas. 

And if you're wondering about the acronym, it's Driving Under the Influence of Weed on 420 Day, which I learned from Andrew Gelman's blog is a day of celebration of cannabis.

Andrew's blog post is about the exemplary work done by Sam Harper and Adam Palayew, debunking a highly-publicized JAMA study that claimed that 420 Day is responsible for a 12 percent increase in fatal car crashes.

The discussion provides great fodder for examining how to investigate observational data, which is what most of Big Data is about. It is a cautionary tale for what not to do.

***

The blog begins with Harper/Palayew channeling Staples/Redelmeier, the authors of the study: "fatal motor vehicle crashes increase by 12% after 4:20 pm on April 20th (an annual cannabis celebration)."

This short sentence captures the gist of the original study but it omits an important detail: to what is the increase relative?

If we ran an experiment, we would recruit a group of drivers, and select half of them at random to smoke weed on April 20. Then, we would count what proportion of drivers suffered fatal car crashes after 4:20 pm. The analysis would be straightforward: what's the difference in proportions between the two groups? With such an experiment, it is possible to draw a causal conclusion.

Alternatively, we could conduct a case-control study. The cases are the drivers who suffered fatal car crashes on April 20. We collect demographic data on these drivers. Then, we define a set of "controls", drivers who did not suffer car crashes on April 20 but on average, have the same demographic characteristics as the cases. Next, we need data on cannabis consumption, preferably on April 20. We want to show that the level of cannabis consumption is significantly higher for cases than for controls.

(For further discussion of these analysis designs, see Chapter 2 of Numbers Rule Your World (link).)

The actual study was neither experiment nor case-control. It was a piece of pure data analysis, based on "found data". I like to call this "adapted data," the "A" in my OCCAM framework for Big Data - data collected for other purposes that the researcher has adapted for his/her own objectives. In this study, the adapted data come from a database of fatal car crashes.

So how was the adapted data analyzed? Harper/Palayew answer this question in their second description of the research:

Over 25 years from 1992-2016, excess cannabis consumption after 4:20 pm on 4/20 increased fatal traffic crashes by 12% relative to fatal crashes that occurred one week before and one week after.

The cases are the fatal car crashes that occurred after 4:20 pm on 420 Day. The comparison isn't to the drivers who did not suffer crashes on the same day. The reference group consisted of fatal car crashes that occurred after 4:20 pm on 4/13 and 4/27. The difference in the average number of crashes is taken to result from "excess cannabis consumption". 

Notice that such a conclusion requires a strong assumption. We must believe that absent 420 Day, 4/13, 4/20 and 4/27 ought to have the same fatal crash frequencies.  

***

You hopefully recognize that the analysis design for adapted data is on much shakier ground than either an experiment or a case-control study. 

Harper/Palayew's initial debunking focused on one issue: what's so special about April 20? To answer that, they repeated the same analysis on every day of the year. The following pretty chart summarizes their finding:

The red line is the line of no difference (between the analyzed day and the two reference days from the week before/after). Each vertical line is the range of estimate of the difference for a specific day of the year. The range for 4/20 is highlighted, and several other days with elevated fatal crash counts are labeled.

The chart was originally published here, with the following commentary: "There is quite a lot of noise in these daily crash rate ratios, and few that appear reliably above or below the rates +/- one week." Andrew adds: "Nothing so exciting is happening on 20 Apr, which makes sense given that total accident rates are affected by so many things, with cannabis consumption being a very small part."

While the chart looks cool, and sophisticated, the following histogram of the same data helps the reader digest the information. 

I took the daily estimates of the fatal crash ratios from Harper/Palayew's published data. Each ratio presents the crashes on the analysis day relative to the crashes on the two reference days. The histogram shows the day-to-day variability of the crash ratios, which is what we need to answer the question: how special is 4/20?

The histogram is roughly centered at 1.0 meaning no observed difference. The black vertical line shows the ratio for 4/20. It is leaning right - in fact, it is at the 94th-percentile. In classical terms, this is a p-value of 0.06, barely significant. 

The following 21 days have more extreme ratios than 4/20: 

Jul 4 Dec 23 Dec 21 Nov 21 Sep 1 Dec 20 Sep 2 Jul 3 Dec 31 Oct 31 Nov 23 Dec 18 Dec 6 Jul 14 Sep 4 Dec 22 Mar 17 May 25 Apr 1 Mar 7 Dec 19

Will JAMA editors accept one research paper for each of these days? The work is already done - the rest is story time. 

 

P.S. [4/27/2019] Replaced the first chart with a newer version from Harper's site. This version contains the point estimates that the other version did not. Those point estimates are used to generate the histogram.

New Facebook data breach, and the importance of Business Intelligence

 

On Friday, Facebook announced that hackers have gained access to personal data of at least 50 million users (see here for example). Analysts immediately connected this incident to the Cambridge Analytica scandal. How is this data breach different from the Cambridge scandal?

How the data was breached

One should take any announcement of how a data breach occurred with a grain of salt: it's obvious that companies will not publish anything that attracts lawsuits; plus, it's unclear that there is a penalty for lying about the real reasons for a data breach. There is no external auditing, and the companies control the narratives and statistics surrounding these events.

Based on what Facebook told us, the Cambridge Analytica scandal involves Facebook partners gaming the system to obtain data about Facebook users. At worst, it is a violation of community standards, i.e. claiming to be doing academic research. The research firm uses tools provided by Facebook to obtain the data. (The firm maintains that it disclosed to users that it was using the data for non-research purposes.)

In the present case, Facebook claims that a combination of coding bugs (i.e. unintended features) enabled unknown partners to gain access to the user's entire account. Not even that, but any accounts on third-party apps or websites that the user signs on to using Facebook.

 

How the breach was discovered

The current scandal demonstrates the value of the business intelligence/business analytics functions within companies. We were told that Facebook first realized that certain metrics were showing unusual trends, and upon investigation, they discovered the bugs.

This is entirely believable. That's what happens when you have good data reports. They surface anomalies. These then have to be investigated. These investigations are extremely tricky because all you know is the trends are different. There are a thousand reasons for the shift. The analyst's job is to establish a cause-effect. Especially since the development community adopted "agile" practices, all kinds of self-imposed changes are occurring all the time with no warnings. For a site as large and complex as Facebook, it takes a huge effort just to get a list of all site changes within some specified time window!

What complicates this situation more is that the vulnerability was traced to multiple bugs, not just one. I could imagine the twists and turns and the false alarms that were generated during the investigation.

The nature of this problem is no different from an investigator trying to chase down an e-coli outbreak, which is detailed in Chapter 2 of Numbers Rule Your World (link).

The data science community is guilty of talking down on the business intelligence function. There is a misperception that BI is for less skilled people doing boring things. The reality is there is more science in BI than in so-called data science (defined here as software engineering). Science, after all, is about figuring out why things are as they are. Engineers, by contrast, use our understanding of science to change the way things are.

 

You've been warned

In my debut Youtube video, released a few weeks ago, I explain how Facebook collects data about you. My biggest tip about protecting yourself is to not use Facebook to log onto other websites. Convenience is the drug that lures you into the trap.

Think about those one-password services. Instead of hundreds of password, you have one. So the thief needs only one password to assess hundreds of sites. Using Facebook to log onto other websites makes Facebook the centralized point of attack by the bad players.

Note that this is not limited to Facebook. Using Google, Yahoo, Amazon, etc. to log onto other services is no different!

If it's more convenient for you, it's also more convenient for your enemies. Remember that.

Also in the video, you will learn

• why did Facebook invent the Like button
• how do Facebook know what you are doing outside Facebook
• why not clicking doesn't mean you're not being tracked
• why you shouldn't use Facebook to log into other services

See the video here, and send me ideas for future episodes!

The most famous calendar in America right now

In the leadup to today’s hearing, U.S. Supreme Court judge nominee Brett Kavanaugh produced what he claimed to be calendar entries from the summer of 1982 as evidence that he did not attend a specific party, and therefore he could not have committed the unsavory acts alleged of him.

This story perked my interest from the data perspective. What kind of information is contained in calendar datasets? And how can such data be used to support or invalidate hypotheses?

This discussion is of great relevance beyond the Kavanaugh case because the majority of the data being collected today - surveillance data, transaction data, clickstream, etc. - are all event data: they record when someone did something.

***

When I started this post, no actual pictures of the most famous calendar right now has been published. USA Today did publish a few pictures yesterday. Here is one page to give us a visual:

This post is not about the specifics of Kavanaugh's youthful activities but about the nature of calendar datasets.

A calendar dataset consists of calendar entries. When converting the above image into a spreadsheet, one would create a row for each event (so multiple rows for multiple events on the same day). Each row should contain the following data, laid out in columns:

• The date and day of week of an event
• The time of the event
• The location of the event, address and/or directions to it
• Other people involved in the event
• Duration of the event
• Miscellaneous notes relevant and specific to the event
• Other notes unrelated to the event

In industry parlance, the above list is called a “data dictionary.” It describes the structure of the data. Any data analyst who has used such things realize that these definitions are imprecise. If one randomly selects any two calendar datasets, there will be variance in what one actually gets.

A lot depends on how the calendar owner uses the calendar! Here are some considerations:

Planner or diary. Most people I know use calendars for planning future events. Kavanaugh appeared to use it also as a diary of sorts. Some entries record past events, e.g. a basketball score. Other entries pertain to future events, e.g. going to a movie. This is a key issue in interpretation: if it’s used as a planner, then the data become frozen in time once the event is over but if it’s a diary, the analyst should assume that edits or appends may change the data even after the event date.

Planner entries are tricky because the information may not hold true beyond the date of the event – an event might get cancelled, the calendar owner might decide to skip the event, times or venues or attendees may shift, etc. To prevent mis-interpreting the data, the analyst should strive to annotate the data with whether a calendar entry is for planning or diary.

How changes are made. If the calendar is used as a diary, the owner may revise the information after the event happened, e.g. correct the list of attendees. Even if the calendar is purely used for planning, edits will be inevitable, e.g. the venue of an event might change before the event happens. How are such changes enacted? Some owners may erase and overwrite; other owners may black out and append; still others may strike out and append. In the first case, we have no trace of the change at all; in the second case, we know something has changed but not the specifics; in the third, we have both the old and the new versions of the information.

How complete is the data? A fallacy is to assume WYSIWYG. There is no guarantee that every event in someone’s life is recorded on the calendar. We can assume that every event that the calendar owner wants to record is found on the calendar. Even that assumption is not safe. It happens a few times a month when I forgot to put meetings down on my calendar (most of these I did attend, and then there are the ones that I forgot about because they are not recorded). Missing events appear as missing rows on the dataset.

Kavanaugh did not seem to care as much about the time of events. There are many entries that did not contain when he was supposed to do something. So his calendar dataset contains a large number of missing entries under the column of “time of event.”

How consistent is the data? An analyst can’t even assume that a specific person would follow strict guidelines in terms of filling out a specific item on the data dictionary. Take Kavanaugh’s calendar as an example. On many entries, he did not record the attendees of the event. But on some, he did. When the dataset shows no attendee names for a particular event, does that mean he was alone, or does that mean he chose not to list the attendees? Even if there is a list of attendees, the analyst is hard-pressed to know if the list is complete.

How reliable is the data? Some people are sloppy about details, others are meticulous. Some people correct the entries, other don’t. Some data elements might be deemed not important enough to correct.

How the data was generated. Most of the data will be transcribed (by hand or by optical character recognition software) from the paper calendar to a database. Note, however, some of the data must be inferred. A good example is item (a), the date and day of week of an event. When we read the calendar, we know by the location of the text which date the event is supposed to occur but there is no handwriting of the date! So, in order to generate that column, the analyst must extract (using my own calendar as an example) the year from the cover, the month from the page title and the day from the column name.

Wait – it gets more complicated. Imagine you fill out the space for a given day, so the additional entries are written in the margins with a guiding line to that entry. (Ouch!)

***

I can keep expanding the list of issues but you get the idea. Data is a dirty business. The worst thing that an analyst can do is to presume that the data in front of him/her is (a) complete and (b) accurate.

Now, let’s get to the logic of the Kavanaugh defense. The basic premise is that the calendar did not contain an entry showing that he was at the same party as the accuser, and therefore he was not at such a party, and therefore he could not have committed those alleged acts.

For that logic to hold, we have to believe a further set of assumptions that are not proven by the calendar dataset:

1. That the calendar contained every party that Kavanaugh attended during those months
2. That Kavanaugh primarily used the calendar as a diary recording past events, rather than as a planner for future events, that is to say, information related to the event is true or corrected after the event
3. That for the specific event in question, he was in “diary” mode rather than “planning” mode so that the absence of the party can be inferred to mean he wasn't there.
4. That when he revised entries in his calendar, he never erased past writing.
5. That none of the blacked-out entries contained relevant information to the current situation.
6. That he always listed all attendees of parties he went to.

It’s really hard to use data to prove “absence,” and this is no exception.

***

Be careful next time you interpret your event dataset!

 

2017 Review and 2018 Outlook on Cybersecurity

Another key recurring news item in 2017 was cybersecurity. This issue gets tied up with data because data, particularly personal data collected by businesses and organizations, have value, and criminals are using stolen data as ransom.

What happened in 2017

Throughout 2017, high-profile companies admitted to losing huge piles of private customer data. Yahoo! led off the parade, disclosing that billions of account data were stolen. At the time, the wayward Internet pioneer was selling itself to Verizon, which shrewdly used the negative publicity about the data breach to re-negotiate the acquisition price. In a matter of months, Verizon itself copped up to losing data of millions of customers. The next ground-breaking leak ensnared Equifax, one of several corporate giants making a business out of compiling and selling profiles on every American. This data theft is egregious as it involves much more private data than usual, such as driver’s license numbers and social security numbers. Not to be outdone, the Republican National Committee left the cyber-door open on voter data of all 200 million voters in the U.S. By year’s end, the bad kid on the block, Uber, revealed that it had paid off hackers to hide the theft of data on close to 60 million customers.

And those were just the big ones, the ones we are told about. The known incidents are the tip of the iceberg (this site has a long list of 2017 data breaches) as there appears to be no legal jeopardy for the screw-ups. Government regulators are absent from the crime scenes. Businesses and the cybersecurity industry are shaping the entire narrative.

Also worth mentioning is the cover-up and profiteering in the aftermath of data breaches. In several cases, executives are suspected of personally profiting from these mishaps by selling shares prior to the public announcements. If the allegations are true, the executives violated insider-trading laws but don’t count on law enforcement.

 

What to expect in 2018

Some first questions might be:

• Will our cybersecurity improve in 2018?
• Will there be fewer data breaches?
• Will businesses win the war against cyber-criminals?

These questions are too easy -- the answers are no, no and no. A better question is: Will 2018 be the year in which businesses finally get serious about cybersecurity?

Truth be told, I did not write much about cybersecurity on this blog in 2017 but I will be covering the topic in 2018. To give you a taste of what’s to come, I see cybersecurity as a problem created by the tech industry, therefore unsolvable from a technological standpoint, thus the demand for cybersecurity solutions is bottomless. No company can get serious about cybersecurity because it’s not a solvable problem.

Here’s a useful analogy: they are selling you flood abatement solutions all the while they are clearing the vegetation, which causes more flooding. They say that the devegetation is for the greater good, as it makes space for building new homes, which relieves over-crowding. They sell you a dam, which you build. Subsequently, as the flooding gets worse, you have to reinforce the dam, over and over again.

Long time ago, the technology industry decided that our cybersecurity is secondary to their other objectives such as convenience, automation, speed, utility, profits. Many technological decisions involve a trade-off between cybersecurity and one or more of those other objectives.

One of the biggest digital infrastructure trends in recent years is the migration of data to “cloud” storage. Instead of businesses storing our data in proprietary computers they own and maintain, they upload the data to huge networks of shared computers run by third parties such as Amazon, Microsoft and Google.

This paradigm shift yields a wide array of benefits to consumers. For example, the data are located closer to the end-user, reducing time we spend waiting for things to load (equivalently, computations to finish). Tasks can be split up among several computers, and run in parallel, accelerating computations, and thus decisions. It is also much easier to exchange data between companies since these cloud services have open interface protocols, unlike proprietary computers that have heavily guarded access points. The cloud also supports a high degree of automation – computers can communicate via bots without humans intervening.

But those same features create opportunities for cyber-criminals. If it’s easy and convenient for the good bots, it is easy and convenient for the bad bots as well. Cloud services require data to be replicated and stored on multiple computers, and they make the data travel. The more copies there are, the more trips they take, the higher chance of data being stolen.

Cloud services are not the sole cause of data breaches. I am just using it as an illustration of the two-sidedness of technological decisions. The tech industry has made a bet that consumers will accept those benefits at the expense of cybersecurity.

So, in 2018, we should see more data breaches, and an expanding cybersecurity industry. Don’t hold your breath for the ultimate solution. The Wall Street Journal recently reports that governments might lay down the law - it sounds like Europe might have to take the lead. Governments, though, are compromised by their own desire to get a hand on that data.

 

Visible and invisible errors, and the nefarious power of suggestion

In Chapter 4 of Numbers Rule Your World (link), I explain how predictive models make errors. One of the key ideas in the chapter is the visibility of errors. Modelers are much more likely to care about visible errors than invisible errors. Most classic discussion of predictive accuracy ignores this important issue, and assumes that all errors, and all types of errors, are visible. In real life, many errors are invisible, or can be deliberately hidden from users.

Consider the failed child-abuse prediction model used by Chicago and described in a previous post. The errors made by this model are highly visible, and frustrating to case workers. This is why this scam was exposed. Case workers have had to work through thousands of cases of false positives. False negatives are equally visible, in the form of at-risk children who were not flagged for investigation.

Errors by auto-correction software are visible and offensive like mosquito bites. But we mostly react to the false positive errors, when a perfectly fine word has been changed to the wrong word. It's less noticeable if the software misses a wrong word or a typo. Not surprisingly, such software gets a bad rep.

***

By contrast, lots of predictive algorithms generate errors of prediction that are invisible to users.

For example, many drivers use Google's Waze for navigation. Waze gives predictions of how much time the driver will save by following its route. There is no way for the driver to measure whether that prediction is accurate.

I highlighted in my book - anti-doping tests to catch cheating athletes. If the test commits a false negative error, you bet the cheating athlete will not call a press conference to taunt the testing labs for their big mistake. If the test gives a false positive, however, the pleas of innocence would be deafening.

***

Sometimes, it takes expertise to notice the errors. Take Google Translate. If the target language is completely unknown to the user, the user can't tell if the translation is good or not.

Rainfall forecasts usually come as probabilities - 30 percent chance of rain. It takes quite a bit of effort to validate the prediction of 30-percent chance; someone who's not a scientist is unlikely to have to tools to do it.

These errors can be visible but usually stay in the shadows.

***

Errors tend to feel worse if they are visible. Invisible errors present opportunities for mischief.

Take Waze for example. At the start of your journey, Waze will proudly announce "I have chosen the most optimal route for your trip. You will save 8.3 minutes by following this route." Drivers have no way of verifying whether (a) the route suggested is indeed the most optimal or (b) whether this route saved 8.3 minutes relative to some other route or 10.5 minutes, or 2.4 minutes, or any other number. But many drivers are convinced that they have saved time. The uncorroborated perception of time saving demonstrates the power of suggestion. By contrast, the old-school GPS such as TomTom or Garmin does not make claims - to their detriment.

Such suggestions only work if the errors are invisible. When the developer of the child-abuse prediction model for Chicago tried the same trick as the developer of Waze, they ran into trouble. The Chicago Tribune reported that the predictive model generated alerts to social workers of the form: 

Please note that the two youngest children, ages 1 year and 4 years have been assigned a 99% probability by the Eckerd Rapid Safety Feedback metrics of serious harm or death in the next two years.

It's rather easy to disprove this statement because the errors are easy to spot. Eckerd has agreed to tone down the language.

 

Overselling predictive models (Hint: future performance matters)

Those who have read my books or taken my courses would be the least bit surprised by the news that just came out about the naked emperor of Chicago's much ballyhooed predictive algorithm for child abuse. Chicago Tribune has the story (link).

Key points:

• Data scientists (over-)sold an algorithm to predict children "at risk for serious injury or death."
• The director of the government agency that purchased this product concluded after two years that "[the model] isn't predicting much."
• At least $366K were spent i.e. wasted on this program. This number most likely does not account for the costs of support staff and infrastructure and the actual waste described below.
• "More than 4,100 Illinois children were assigned a 90 percent or greater probability of death or injury. And 369 youngsters, all under age 9, got a 100 percent chance of death or serious injury in the next two years" [a false positive problem - not to bring up a sore subject but several infamous outlets proclaimed that Hillary Clinton would win the Presidency with 90% or higher probability!]
• "high-profile child deaths kept cropping up with little warning from the predictive analytics software; Predictive analytics (wasn't) predicting any of the bad cases" [a false negative problem]
• Selling the dream: "If it is possible to use big data to spotlight a child in trouble and intervene before he or she is hurt, then doing so is government's moral obligation, advocates for the technology say."
• The algorithm, despite being paid for by public funds, is hidden from public view but the Tribune report pieces this together: "Eckerd [the vendor] retrospectively analyzes thousands of closed abuse cases and from them draws data points that are highly correlated with serious harm. The parents' ages could be a factor — or their previous criminal records, evidence of substance abuse in the home, or the presence of a new boyfriend or girlfriend."

The story does not invalidate the practice of predictive analytics but points out pervasive problems within the industry. Most importantly, we are over-hyping this technology. I cover the statistics of such models in depth in the marketing chapters of Numbersense (link). Predictive models are impressive in a relative sense (in marketing, we use the term "lift") but not usually in an absolute sense. Every social science based predictive model I have seen misses a sizeable number of targets while falsely labeling lots of "cases".

When a vendor does not report on accuracy metrics, or when such metrics are not interpretable in practice, or when the metrics are generated by third parties, or when metrics are computed purely from historical data, one has to be very careful about separating the good products from the scams.

***

It bears repeating that the "big data" used in this type of analysis has multiple complications which I collectively describe as OCCAM.

Observational - we cannot run an experiment and assign people to have criminal records, or other characteristics at random, so it is tough to get any read on causal mechanisms. Thus, there is a good chance to suffer from spurious correlations.

Seemingly Complete - but not really. Many cases are unreported and not in the training data. One of the excuses used by Eckerd when confronted with inaccuracy is that certain children are not even scored because they do not appear in the data. That is technically correct but does not change the fact that the model did not predict those deaths.

No Controls - many predictive algorithms do not use controls explicitly. Another issue is that the possible controls all come from cases reported to the authorities. Without a doubt, if we take the profiles of the at-risk children as determined by the algorithm, we will find many other children in the general population that do not show up in the database at all.

Adapted - the data collection process was not designed specifically to support predictive modeling.

Merged - various datasets get merged during the analysis, which introduces errors.

More discussion of OCCAM here and here.

***

While I appreciate that the Chicago Tribune wrote and published this article, this is yet another media report on predictive modeling using "big data" that does not contain any data or quantified metric of predictive accuracy.

The two chapters most relevant to this post are Chapter 4 in Numbers Rule Your World (link) and Chapter 5 in Numbersense (link).

 

Did web scraping just receive a legal boost?

Something truly game-changing might have happened. According to TheNextWeb, a judge has told Linkedin (now owned by Microsoft) that it cannot stop people from using scripts to scrape public data off the Linkedin website.

I have mentioned this issue before - web scraping is in some kind of legal gray area. Web scrapers appear like human beings to a website, and these bots just exist to collect data off the website. It's clear that owners of many websites (Kayak, Linkedin, ESPN, etc.) do not like web scraping. They implement technologies to block such bots.

There are several legitimate business reasons for opposing web scraping. For example, a retailer like Amazon does not want competitors to know all of its pricing. The downside of convenience - 24/7 shopping - is that it reduces information asymmetry. In the pre-Internet days, a store can run special prices in certain locations without word spreading to other locations - but this is no longer the case. Information asymmetry allows profits to materialize.

Besides, the data might have been purchased for real money from some vendor. In this case, the vendor will forbid the buyer from publishing all the data, otherwise the vendor loses revenues when some potential customer scrapes it off the buyer's website. The buyer also has an incentive not to give way the data assets, say, to a potential competitor.

From the web scraper's perspective, the data are publicly displayed so why should there be a restriction on usage? For social-media sites, the data are shared by users, not generated by the site owners, and so ownership of the data is confused. When site owners include a prohibition on web scraping on their terms and conditions (knowing full well that most people do not bother reading them), this creates at minimum an ethical issue, and possibly a legal issue. However, with the Linkedin ruling, the cloud may be clearing.