« Choosing reference levels | Main | What is the (expensive) miracle drug that cut colon cancer deaths by 20%? »


Feed You can follow this conversation by subscribing to the comment feed for this post.

Michael Thompson

Hmm, I suppose this would be a problem if they were explicitly saying the ROI is for the entire population. But they added the clause that the ROI calc is just for the population that finishes school within the timeframe. Since the population is discrete, those data points can be segregated into separate populations. As they seem to have done here.
Still, it's a good point, and a common manipulation.


Michael: You're right that it's not the way they describe the result, it's the way they conducted the analysis. This falls into what I call "true lies". It's akin to a bank claiming that their loan portfolio has tremendous returns for those loans that did not default. It is a statistic that can be computed but it is useless, and misleading.

Michael Thompson

What makes this topic very interesting is also that it is an attempt to quantify the value of an education by reducing it to a monetary equation. if it were for trade schools, I think that it would be more relevant. But I think people are actually waking up to the fact that 1) schools are not guaranteed paths to income and hopefully 2) schools are as much about enlightenment and inquiry than about career choices.

The other buyer-beware statistical fudge is portfolio returns for portolios with funds that have closed. I'm not current on my disclosure compliance rules, but I recall that a fund manager's return didn't used to have to disclose returns for all funds, both current and discontinued. Not sure if that has been universally mandated yet.

Lydia Frank

PayScale actually did consider the issue of dropouts when determining the methodology for the College Return on Investment (ROI) report. While it's true that we didn't include wage data from dropouts directly, we did weight our ROI measure by overall graduation rates for each school, as provided by the Department of Education, to approximate dropouts in our calculation. Schools with lower overall graduation rates will take a bigger hit to their ROI than schools with higher overall graduation rates.


Lydia: Thanks for clarifying but I don't understand how you adjust ROI without wage data. Can you explain what you mean by "a bigger hit" to ROI? What is the nature of this "hit"?

The comments to this entry are closed.

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR.

See my Youtube and Flickr.
Numbers Rule Your World:
Amazon - Barnes&Noble

Amazon - Barnes&Noble


  • only in Big Data

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here


R Fundamentals, Principal Analytics Prep

Numbersense: Statistical Reasoning in Practice, Principal Analytics Prep

Applied Analytics Frameworks & Methods, Columbia

The Art of Data Visualization, NYU

Signed copies at McNally-Jackson, NYC

Excerpts: Numbersense Ch. 1, 7, 8. NRYW

Junk Charts Blog

Link to junkcharts

Graphics design by Amanda Lee