« The hazard of casual analysis of hazards | Main | A skewed view of ten Indian states »



Just checking: are these dollars inflation-adjusted?


This feels like lazy criticism. There is no reason to suspect "coordinated cheating" because the NAEP tests are not "high-stakes" and not tied to school or state funding in any way. In fact, NAEP results are not even released at the school level. And they are actually fairly well designed and are pretty good metrics in my opinion.


Chris: You're picking on one of my seven or eight points and putting words in my mouth. My comment on cheating in standardized testing does not mention NAEP. It is well known in educational circles that "test scores" do not measure "quality of education"; it may measure one aspect of it but to equate the two is folly.

PP: I don't know whether it's inflation adjusted or not, probably not.


Wasn't the whole point with this graph to show that spending and test scores are have varied independently, i.e. are not correlated? Then I guess it is a matter of power of the test of correlation and how much a change of test scores is considered "large enough" to talk about. Now, we have to take their word for it that there is no correlation between math scores and spending (although it looks like there might be). Am I missing something?

I agree on the points of the weird scale of test scores and the comparisons to other countries though.

Jackie Conrad

I can see two possible talking points here:
1. The cost of K-12 education has increased to keep up with the cost of HEALTH INSURANCE in the US. In Europe, socialism has not driven the cost of salaries+fringes up since there is universal health care.
2. Schools have done well to have a slight upward trend in NAEP especially when you consider the INCREASE of students living in poverty.



I'm pretty sure health care costs have gone up in the socialist countries you mention (like my country, Canada) as part of the costs are related to increases in obesity, cancer, and an aging population. However, I'm sure that our increases have been no where near as bad as the total increases in the US.


Yes, the charts are inflation adjusted. They aren't adjusted for land prices which have risen well over inflation for decades, or for health care costs as indicated by Jackie, or by rising costs of supplies (ie computers in schools).


This blog article provides some nice insights on some of the fundamental statistical problems of the graph and claims/inferences made about it by Gates. http://www.epi.org/publication/fact-challenged_policy/ provides some further critique of Gates interpretation.

Stephen O. Jambor, Ph.D.

AND (in addition to the statistical methods questions), there is still the more fundamental question regarding "validity" (both construct & criterion).
It has not been demonstrated that these high stakes tests (of achievement) are in fact not behaving as though they were "masked measures of intelligence".
Until/unless it can be shown that these alleged achievement tests are in fact assessing something above the givens (IQ), we are chasing after student-level factors, something that is not entirely within the school's grasp to control!


Gates has made many of these misleading claims before and been debunked before: http://www.epi.org/publication/fact-challenged_policy/.

The comments to this entry are closed.


Link to Principal Analytics Prep

See our curriculum, instructors. Apply.
Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR.

See my Youtube and Flickr.

Book Blog

Link to junkcharts

Graphics design by Amanda Lee

The Read

Keep in Touch

follow me on Twitter