Big Data, Plainly Spoken (aka Numbers Rule Your World)

The get-rich-quick scheme of the English

The World Economic Forum published this chart:

The "EF EPI Score" is a measure of English proficiency. So the evidence is clear as day: "Better English and Income Go Hand in Hand," as their headline blares.

Last time I was in the New York subway, the panhandler spoke good English.

What's a blogger to do? I pulled out the EPI scores from the EPI report, and downloaded the Gross National Income per Capita (PPP $) data from the World Bank. But I failed to reproduce the above scatter plot. The EPI scores match what's on the chart but I couldn't find the exact series of GNI data they used. I tried both 2012 and 2013 and neither matched. This is the closest I can get to the original chart:

Notice that the income of Singapore is over $70,000 on my chart but it is just above $60,000 on theirs. The general shape of the data is approximately preserved so what I discuss in this blog post should still hold.

But there is something else that bugs me a lot. To get to the above reproduction, I have to take out some of the data. To be precise, all the orange dots in the following chart were removed:

A clear effect of removing those orange dots is to steepen the slope of the regression line. In other words, the GNI data are more correlated with English proficiency in the first chart than in the second. Correspondingly, the "R-squared" (how well the line fits the dots) jumps from 6% to 42%.

***

I will return to the missing countries later. The other huge problem with this type of scatter plot is that it imposes a misleading analytical frame on the readers. Readers of such charts are led to believe that English proficiency is the only - or most powerful - factor that explains a country's per-capita income.

To illustrate this point further, I found some data on urbanicity, the proportion of population living in urban areas, and added this factor to the regression. Here is the resulting model:

The first thing to note is that the slope is much steeper in the % Urban panel, meaning that GNI is much more correlated with % Urban than with EPI Score.

Even more telling is the following chart. Notice that the EPI regression line is now almost flat.

In the first panel plot, the EPI regression line assumes an average degree of urbanity (in the 50s). In the second panel plot, I set the urbanity to over 90%. For countries in which the vast majority lives in urban areas, EPI score is essentially uncorrelated with income levels.

When trying to explain numbers like income levels, one must use complex models involving many variables.

***

Removing data is sometimes acceptable, if the analyst has a good reason to do it. Improving R-squared alone is not a good reason. So I am curious if there is a good reason to remove those orange dots (missing countries).

Let's see if there are any patterns.

The removed countries exist in the lower right corner of this scatter plot, meaning that they are countries with higher urban populations but below average English scores.

Several of these countries have very high incomes, as you can see in GNI vs. EPI graph above. Those high-income, low-EPI countries are Kuwait, UAE and Saudi Arabia. These countries present challenges to the one-factor, EPI-only model as they contradict the hypothesis that speaking English well makes one rich.

You can also see why the two-factor model fits the data better. Those three Middle Eastern countries have high urban populations, which can explain the high per-capita income, even when the English factor can't.

Iran and Iraq were probably omitted because of unreliable GNI data. It seems like they excluded all, or almost all, of the Middle East. The reasoning isn't explained by the author. Notwithstanding, I believe Egypt is included.

Outside those three countries mentioned above, the other omissions do not affect the regression line much since there are other dots that live nearby. Many of the countries are probably left off inadvertently, lost in the merging of the datasets. For example, Venezuela is labeled as "Venezuela, RB" in one of the data series so this country may disappear (unexpectedly) when the datasets were combined. Similarly, South Korea was known as "Korea, Rep." in one of the datasets. This is a very tricky one because those two names appear at very different levels of the data sets.

P.S.

(1) Even the charts I created do not contain all of the countries. It only contains countries that have valid EPI Scores as well as available GNI data. In general, there are many more countries for which we have income data than EPI Scores.

(2) There is no reason to stop at a two-factor model. There are many other factors that are correlated with GNI.

 

 

 

 

Confused by machines, or spooked by the machine-makers

This New York Times article draws attention to real trends in the financial investments industry but gets completely lost in the smoke around those pushing "machines" and "data".

The trend most concerning to the investments industry is the sustained, large-scale outflow of money from "actively-managed" funds, mutual funds being the biggest category of such. The industry makes loads of money from management fees by promoting the idea that investors are paying for expert stock-picking by star investment analysts. The outflow of funds directly impact the revenues of the investment industry.

A key sentence in the article is: "Mr. Fink [Blackrock's CEO] has become convinced that BlackRock must bet big on the power of machines, be it Aladdin, the firm’s risk management platform, robo-advisers, big data or even artificial intelligence."

***

The new game in town is the ETFs, or exchange traded funds. These belong to the category of "passively-managed" funds, for which the investment industry earns no or low fees. ETFs are a type of index funds. But index funds have been around for a long time. John Bogle, of Vanguard, has been a long-time advocate for such funds, since the 1970s! So the only real trend is the waking up of the masses.

But ETFs and index funds are neither "artificial intelligence" nor "big data," as alleged in the quotation above. ETFs and index funds attract passive investors; instead of taking bets on individual stocks (or bonds, etc.), these investors are buying a chunk of the market - they are simply taking advantage of the law of large numbers. They eschew stock-picking. To the extent that computers and machines are involved, they are not introducing intelligence but discipline.

These funds also do not need a lot of "data". They usually track some well-defined market index, e.g. S&P or Nasdaq. They do not try to outsmart the market. So the embrace of ETFs has nothing to do with data or machines.

***

There are indeed experiments to use "big data" or "machine intelligence" to run funds - but this is a completely separate topic and trend. The article offers two examples. In one, it is claimed that PhDs are using satellite images of Walmart parking lots to evaluate the health of the Walmart business. (I read this with amusement, since in the old days, one would send low-wage bean-counters to those parking lots and count vehicle flow in and out of them. We called those time-and-motion studies, later operations research.)

In another example, it is said that active managers were fired, "all varieties of computer models are crunched in pursuit of stock picking ideas," and management fees are lowered. Readers might get confused at this juncture: this effort is about averting the active management confidence crisis - has nothing to do with ETFs and index funds, which, as explained above, require no machine intelligence.

The people who design these computer models are cut from the same cloth as the earlier generation of stock-pickers. They believe that investors should pay management fees for their expertise in finding stock picks. When we are told that "machines" are replacing humans, we frequently forget that behind every machine are human faces - the computer scientists who write the code that tells the machines how to pick stocks. By investing in such funds, one is buying the concept of stock-picking; the only difference is who is doing the stock-picking.

The banks are not embracing ETFs or passive investing here. The Big Data and AI talk is about saving active fund management. Ironically, because they are charging lower fees, apparently they value the ability of these "machines" less than those of the previous generation, or believe that the consumers are less willing to pay for such a product.

There is no evidence offered in the article that the funds that make use of "big data" or "artificial intelligence" have a better long-term track record than anything else, nor is there a sudden influx of investor interest in such funds.

tl;dr

The investment industry is facing a revenue crisis as investors move their money into ETFs. This article misleadingly connects ETFs to a different initiative of using Big Data and AI for stock-picking. The latter is an attempt by the investment industry to avert the crisis by restoring trust in active investing and stock-picking, which counters the current trend toward passive investing. There isn't a movement of money into funds using AI or Big Data at this time.

 

 

Collaboration with New York Public Library

For many years now, the field of Data Science and Business Analytics has been booming, and hiring managers are finding a severe dearth of high-quality job-seekers. Meanwhile, there are a good number of people interested in entering the field but keep bumping into walls. Hiring managers like to hire experienced people for a host of reasons, including the fear of other hiring managers poaching their trained employees. For a number of years now, I have been interested in solving this problem.

In the next two weeks, I am collaborating with the New York Public Library's Job Search Central unit to offer two free events.

The first is a public lecture on "How to Start a Data Science and Analytics Career." I will be spending significant time explaining what this field is about, and what data science and analytics teams do. I will also discuss the structure of the job market, and provide tips for how to land a job. You can find more information about it here. The talk is scheduled for April 1.

The second is a resume review workshop. From my experience as a hiring manager, I know that many job-seekers fall short in selling themselves to organizations. It is particularly challenging for those who are seeking to entering this field, and thus do not have direct experience to call upon. This is a hands-on workshop in which I will offer advice on individual resumes. The event is free but obviously, we have limited space, so you must pre-register here. Note that you will need to upload your resume to complete the registration. The Workshop is being held on April 8.

***

I have updated the Events list on the right column of the blog. This is where you learn whether I will be appearing in a city near you. You can check out future and past events here as well.

 

One statement employing two resistance tactics to fend off the data

I want to parse this statement by new EPA Chief Scott Pruitt, as quoted in this New York Times article:

I think that measuring with precision human activity on the climate is something very challenging to do and there’s tremendous disagreement about the degree of impact, so no, I would not agree that it’s a primary contributor to the global warming that we see.

I'm not going to talk about the politics of climate change in this post but rather to point out that Pruitt used two popular resistance tactics adopted by people who don't like what the data are suggesting to them. I have been party to countless such dialogue in business meetings.

Tactic #1: Claiming that data should be ignored until it is made "precise"

Since nothing can be 100% precise, the request for more precise data is akin to an ask for eliminating "terrorism". Imprecision, like terrorism, is a feature of the world we live in. One can work to decrease imprecision, or reduce terrorism, but one can't and won't eradicate either.

This tactic is specifically tailored to data analysts, who being logical thinkers, will never vouch for anything 100%. It works particularly well with ethical data analysts. People using Tactic #1 are daring the analysts to stand up and make false claims of 100% precision. (The tactic might not work on, say, IBM marketers, who have made some stupendous claims.)

The demand for more "precision" always leads to a demand for more analysis. The cycle continues.

Tactic #1 stipulates a black-and-white world. The data is either 100% precise (good) or 100% imprecise (bad). Anything in between is lumped with 100% imprecise. If this criterion were to be applied to all business decisions, there would be no risk-taking, no investments of any kind, and  capitalism would grind to a halt. If Pfizer would not spend any money developing new drugs unless the data is 100% precise, it would never start any projects. If a real estate developer would not take out a loan unless he is 100% sure all of the space could be rented out at desirable prices within the first year, then he would never undertake any projects.

However, when it comes to such decisions, the same decision-makers who fear the scourge of imprecision suddenly re-make themselves as "betting men."

Tactic #2: Pointing to disagreement as a reason to refute the conclusion

An easy way to derail meetings is to have a few people blow smoke at the data. The questions raised are frequently trivial, sometimes irrelevant, but the questioning produces an air of doubt.

For example, an analyst might conclude that the overall customer satisfaction rating has been trending down. Now even if the aggregate rating is in decline, there will definitely exist a few counties or neighborhoods in which the rating has sharply risen. The people in resistance will insist on investigating those counties - it doesn't matter if those counties only include 0.01% of the customer base.

Imagine you are a third party to the debate, and you have no knowledge of the subject matter (for example, you are the controller attending a meeting about customer loyalty). You listen to a prolonged discussion of cases that may or may not contradict the data analyst's conclusion. You only know what you are hearing at the meeting. It's not surprising that you think there is "tremendous disagreement" and that the conclusion may be dubious.

In reality, there are only a few loud dissenters, who are objecting because the conclusion does not confirm their pre-conception.

For any budding data analyst, you have to be prepared to handle these situations. Pruitt's quote is a perfect encapsulation of the common data-busting tactics. He is saying (a) the data is imprecise and (b) some people disagree, and therefore (c) I reject your conclusion and (d) I am free to believe whatever I want to.

 

PS. Scott Klein pointed me to this related article by the great Tim Harford, which covers similar ground, while filling in some history. It's a Financial Times article so you may need to figure out their leaky paywall ("The problem with facts" by Tim Harford. The link goes to his website, not to FT, which is behind a paywall.).

 

In the name of diversity

I've said it in Numbersense (the chapter on school rankings), and here it happens again.

According to the New York Times (link), Harvard Law School decided to allow applicants to submit GRE test scores instead of LSAT test scores. The announced purpose is to "encourage more students in the United States and internationally from a greater degree of disciplines to apply".

The school has a fixed cohort size, 560 per class, according to the article. It rejects 84 out of every 100 applicants (profile data here). Three quarters of those apply had GPA over 3.76 and one out of four applicants had GPA over 3.94. By making it easier for more people apply, selectivity is enhanced by expanding the base while fixing the numerator of the ratio.