Big Data, Plainly Spoken (aka Numbers Rule Your World)

Why are there so many open positions in analytics?

Last week, I gave an information session on the next-gen data analytics bootcamp called Principal Analytics Prep that we just launched. A recap of the session is available here as a podcast.

To prepare for the session, I did a job search on Linkedin, and found over 80,000 open positions in the U.S. matching the word "analytics". Of these, about 3,500 positions are junior or entry-level positions in the greater NYC area. Over half of those positions have been posted in the last 30 days. There are lots of analytics jobs in the business world, and hiring managers are having trouble filling them.

One reason for the difficulty is that everyone wants to hire from a small pool of experienced people. This is rational behavior on the part of hiring managers.

First, there is a huge gap between book knowledge and practice in this field. This is most apparent, for example, in design of experiments (the theory behind A/B testing). Academic courses focus on design and analysis but in real life, the most crucial matters are getting buy-in from various parties affected by testing, and making sure that the design is operationalized properly.

Second, a good data analyst needs to have a wide-lens perspective, which must be developed over time.

Third, corporate America has cut training budgets to nothing. Managers are not rewarded for training workers; in fact, they may be chastised for hiring untrained talent. It typically takes 12 months to get a new employee up to speed, for the employee to turn from being a cost driver to a productive worker.

Finally, training frequently backfires on the supervisor. It is common to be offered a big wage hike (say 20%) when switching employers but it is almost unheard-of to be offered more than 5% even when an internal promotion is rewarded. Thus, after 12 months of training, the hiring manager might learn that the employee has been poached.

***

Principal Analytics Prep helps bridge this gap by offering the kind of broad-based training in the fundamentals of analytics - instead of begging the hiring manager to train you, you bring the training to the employer.

Our instructors are all practitioners who typically hold senior level positions in industry. They help bridge the theory-practice gap, and also provide a deep network of professional contacts within the industry.

If you or your friend is looking to enter the exciting analytics field, check out our website: http://principalanalyticsprep.com

Even more incompetence in reporting survey results: Washington Post on the hotseat

Previously, we looked at the claim that 7 percent of American adults believe that chocolate milk comes from brown cows. The media, including mainstream outlets such as the Washington Post, demonstrated incompetence in reading survey results.

That was before I investigated the other survey covered by the Washington Post in that article about brown cows (link). The Post should immediately apologize for their horrible reporting. For this study, the data are published, and so no excuses. The reporter used this study to make a case for why the brown-cow study is credible:

One Department of Agriculture study, commissioned in the early ’90s, found that nearly 1 in 5 adults did not know that hamburgers are made from beef.

Turns out they just made this up.

I found this right there in the abstract:

Data were collected from 2,005 respondents representing the following groups: purposely selected primarily white Indiana high school students and primarily black Michigan high school students, randomly selected rural Missouri adults attending one of several town meetings, and randomly selected urban Missouri adults contacted in various settings (including churches, libraries, and grocery stores).

Searching for the word "hamburger," I learned that respondents were asked to agree or disagree with the statement "hamburger is made from the meat of pigs". The results were presented in multiple tables, covering the aggregate sample population and subgroups. For each statement, the researchers listed "% correct", "% incorrect" and "% don't know."

The next time "hamburgers" appeared was on page 37 where the statement about hamburgers was highlighted as one of the top five statements that respondents were most likely to get right (!) To the extent that this report demonstrated illiteracy of agricultural facts, the hamburger example is one of the least powerful evidence.

Moving on to the two adult subgroups. As shown in Table 19, rural Missouri adults answered the hamburger question correctly 94 percent of the time (!!) And in Table 20, urban Missouri adults were correct 80 percent of the time (roughly the same proportion as the aggregate sample).

***

So, let's count the ways the Washington Post messed up:

1. What they reported applied to urban Missouri adults only, and definitely not "adults" nationwide.

2. The urban Missouri adults in the survey were not randomly selected from all urban Missouri adults, and so they might not even represent the average Missouri urban adult.

3. Eighty percent correct is not the same as 20 percent (1 in 5) "did not know," since the survey had a third category called "Don't Know".

4. The statement respondents reacted to was "hamburger is made from the meat of pigs". If someone said this statement is false, we still do not know if this person knows if "hamburger is made from beef".

5. We don't even know if hamburgers are made with 100% beef. In fact, there were reports in recent years that horse meat and other meat might have been mixed in.

 

 

 

 

How to read survey results: chocolate milk edition

Apparently, the Washington Post decided to assist the dairy industry in its latest advertising campaign by publishing a weird survey result, which claims that some adults believe that chocolate milk comes from brown cows. (link)

Just start by thinking about the survey design. In order for people to express this opinion, the survey had to contain a choice of "brown cows." If they had done an open-ended survey, the result will likely be quite different.

The Innovation Center for the dairy industry, whose name is attached to the survey result, does not publish any data or anything else that can provide context to understanding this result. The only thing I can find is this page, which is labeled "infographic" but contains nothing of the sort.

I'm happy to see that this writer for Huffington Post, which also reported on the survey, posed some questions to the Innovation Center. The spokesman claimed that Edelman Intelligence conducted the online survey. "The center, though, was unable to provide a full copy of the survey. And when asked about the survey’s methodology, [the spokesman] said it was 'conducted online'.”

There are some other plausible explanations for the 7% who supposedly said they thought chocolate milk came from brown cows. For example:

• There weren't any other reasonable answers. Notably, in none of the reporting on this survey are we told what the correct answer was, or what the other choices were.
• There were bots or people that were just guessing. We do not know if the answer choices were randomized for each visitor, nor do we know how many choices were presented to each respondent.
• There were bots programmed to select a fixed choice. We do not know if the answer choices were fixed for each visitor.
• There were people offended by the idiocy of the "brown cows" answer who decided to usurp the survey.

We do not know whether the question is multiple-choice, and if so, how many answer options were displayed, or indeed if the survey was open-ended.

Nor do we know where they placed the survey, how they recruited respondents, whether these respondents truly represent American adults, etc. Responding to the Huffington Post request, they claimed that they had representation across all 50 states. That raises even more questions because they would have neede to run a stratified sample overweighting the small states if they only had 1000 respondents.

The Washington Post isn't exhibiting much statistical literacy when its reporter concludes: "If you do the math, that works out to 16.4 million misinformed, milk-drinking people." This reporter naively believes that the 1000 respondents are truly representative of all American adults. I also do not see where the Innovation Center discloses that they pre-select for "milk drinking" people.

So, don't believe this story unless they are willing to publish the data. This sort of thing gives survey research a bad name.

 

Data bite back: a Harvard lesson

New York Times reports that Harvard rescinded admission offers to 10 students (link).

These kids allegedly engaged in horrible behavior on "private" chatrooms online.

Not reported is how Harvard admission officers got a hold of such information. Is it possible there are jealous classmates?

I am not condoning the bad behavior - I put this link up to remind people: (a) no data are private, not even "deleted" data (b) data collected for one purpose can be used for another (c) there is a difference between what one does with the data, and what one can do if one chooses to (d) everyone has enemies under the appropriate context.

 

P.S. Additional reporting indicated that the private Facebook group was splintered off of an official Facebook group that Harvard admissions set up for admitted students to make friends with each other. We still don't know how they knew what happened inside a private group.

 

Why Tesla owners pay more for insurance

A nice article in Zerohedge about the recent rate hike for Tesla owners (link). Of course, Tesla is crying foul.

The gist of the story is that Tesla owners tend to get into more crashes and the repair cost is higher than for other brands. The former is a matter of statistics - but also self-selection: in my limited encounters, the first things Tesla owners tell me about their cars are the fast acceleration and the top speeds. The high cost of repair is widely acknowledged.

Tucked into the second half of this article is a little advertisement for a startup called Root. It explains:

Root is an insurance startup that sets premiums based on individual driving behavior. Customers download the Root app and drive with their smartphone in the car for two weeks. Rates are determined from the habits observed over that period.

This product has some "brilliant" features:

• It prices insurance rates using individual data for two weeks instead of decades of historical data for all kinds of drivers for a long time
• It only sells to drivers with smartphones
• It assumes that drivers who are being measured by their smartphone app for two weeks behave normally
• It assumes that two weeks is sufficient time to measure one' driving behavior
• It assumes there are enough claims during those two weeks
• It assumes that no one will manipulate their data to fool the algorithm
• It assumes that no tech-savvy engineer will sell software that manipulates the data

I say "brilliant" because Root is getting into a very dangerous game. In Numbers Rule Your World (link), I discuss the statistics of the insurance business. Warren Buffet, who is one of the savviest in this business, often warns against insurers who compete on lower rates, because they will superficially win business for a short term until the losses blow over. The riskiest customers are the ones who most need a price break.

What you have here is a pile of data - all biased to indicate lower risk, and thus justifying lower premiums.