« Know your data 18: your location is our product | Main | How do you react to a probability forecast? »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Jimmy

I agree with your points about lack of standards in subscriber reporting, and a bias towards 'good news'. I would also like to shine light on the problem with data management and data quality. These fields in technology are overlooked because they are not as sexy as 'Big Data' and 'Data Science'. I believe there might be an impression that a data team is a plug and play effort but in fact it is a long term effort.

Duncan Garmonsway

"This means someone changed the formula used to compute the metric without disclosing the changes. And then, there was no auditing of the calculation before its publication."

It was probably just a lack of defensive programming. Cooking (the books) is just as much about the ingredients as about the recipe, and bad data arises in nature without the need for a malevolent creator. Data cowboys have raided IT departments, wrecked the application-development lifecycle so that they can make their own beds as messy as they like, and now they're sleeping in them. (Enough metaphors in the mix?)

Audits may depend on scripts and assumptions, too, moreover an auditor probably lacks the domain knowledge to spot many types of mistake.

Kaiser

Jimmy: you're right about this. In my talks, I suggest that there are three growth areas in data science/analytics. There is the software development world, the original "data science"; there is the business-unit analytics world, which used to go by various names like business intelligence, strategic analytics, operations research, business statistics, etc.; and then there is the data infrastructure world, which encompasses all the new developments relating to noSQL, graph databases, etc. The need to organize, maintain, clean, transform, structure the data has increased, not decreased.

Kaiser

Duncan: We are speculating at this point because their statements are just too vague to know what really happened. One big reason is what you bring up in your mixed metaphors: a cultural issue.

zbicyclist

Just a slight change in the company's slogan:

"Lose money now! Ask me how!"

Nick F

We've heard this sort of thing before -- move over London Whale! Risks halved in JP, profits more than doubled at Herbalife.

Spreadsheet error again? So Pershing Square had it right all along.

Common Sense 1, Fancy Algorithms 0.

The comments to this entry are closed.

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.
Numbers Rule Your World:
Amazon - Barnes&Noble

Numbersense:
Amazon - Barnes&Noble

Search3

  • only in Big Data

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here

Courses

R Fundamentals, Principal Analytics Prep

Numbersense: Statistical Reasoning in Practice, Principal Analytics Prep

Applied Analytics Frameworks & Methods, Columbia

The Art of Data Visualization, NYU

Signed copies at McNally-Jackson, NYC

Excerpts: Numbersense Ch. 1, 7, 8. NRYW

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Community