« Clocks and pies | Main | The eyeball test »


Will Dwinnell

"It must be said that many forms of neural networks have been shown to be equivalent to more sophisticated forms of regression.)"

All predictive techniques in data mining are "regressions", of a sort. What varies is the functional form of the output, and the performance function being optimized. For instance, the most commonly-used neural networks (multi-layer perceptrons) are more or less collections of overlapping logistic functions, which attempt to minimize squared error.



Regression and other "traditional" statistics were developed for situations where data was relatively scarce, and the goal was to extract as much meaningful information as possible from the hard-won data. Data mining seems to be oriented more towards the opposite situation, where the volume of data is so overwhelming that special techniques are needed to make sense of it.

John S.

Russ Roberts had a very interesting interview with Ayres on the EconTalk podcast. Surprisingly, Roberts questions the value of correlations, regressions, etc. There are very few examples where statistical analysis provided a definitive answer, and changed minds in the process. His point is that most of the phenomena studied in economics and the social sciences are the result of processes so complex that it is very difficult to isolate single causes. A lot of analysis just winds up confirming the researcher's biases.


For the counterpoint, there's Ehrenberg's article "Regression is useless" (can't find cite at the moment) and Ted Goertzel's "Myths of Murder and Multiple Regression". http://crab.rutgers.edu/~goertzel/mythsofmurder.htm

Regression is a powerful statistical tool and I am not going to contend it should not be used, but in the wrong hands -- and there are lots of wrong hands -- it's like giving a teenager a Corvette and a credit card.


John: thanks for the link. Roberts certainly made his disdain for statistical analysis quite clear, didn't he? I have two problems with this critique: first, what is the better alternative? second, if researchers are intellectually dishonest - ignoring decades of statistical wisdom to run thousands of regressions looking for structure - the problem is not with the tool but with the toolsmith.

The podcast and the other citations do provide a more balanced view of the topic. I also recommend the writings of David Freedman at Berkeley.

For a different kind of counterpoint, I am reading "Gut Feelings" by Gigerenzer. He presents "evidence" showing that intuition beats regression or logical analysis all the time. Reading these back to back points out the trouble with popular science publishing today: these authors all present only one side of the story, unknowing readers may think it's the only side.

Nick Barrowman

Describing data analysis as "number crunching" rubs me the wrong way, as I noted in a blog post last year. The suggestion of a mindless, mechanical approach that can somehow yield useful answers seems so wrong-headed.

Karl K

The critics of Ayres' book -- perhaps one might consider calling them "Luddite critics" -- fail to grasp one thing:

The "supercrunching" techniques WORK. Randomization WORKS. Regression analysis WORKS.

Properly organized and structured, of course.

It was not surprising to me, nor should it be to anyone else, that many of the examples that Ayres uses are from the business world. We should all recall the great line from Dan Ackroyd in Ghostbusters when Murray and his fellow paranormal "scholars" are booted from the university and wondering about their future. Says Ackroyd: "I've worked in the private sector. They expect results."

These techniques deliver results.

u binary

I love reading an article that will make people think. Also, thanks for allowing me to comment!

Preorder Thief

When you park your car for any amount of time leave it in parking lots and garages that have security or a parking lot attendant watching out for it. CREDITS: The Book Thief stars Sophie Nélisse, Geoffrey Rush, Emily Watson, Ben Schnetzer, Nico Liersch, Sandra Nedeleff, Kirsten Block, and Joachim Paul Assböck. Not so long ago, there weren't so many passionate souls carrying around bags full of expensive equipment all over the world.

The comments to this entry are closed.

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR.

See my Youtube and Flickr.

Book Blog

Link to junkcharts

Graphics design by Amanda Lee

The Read

Keep in Touch

follow me on Twitter