« Super Crunchers | Main | Red-lining by marriage »



I don't buy your analysis. Why don't you compute correlation, and see what you get then?


I kept wondering why you weren't looking at the slope of a running average as I read.


anon: it's not a simple matter to compute (cross)-correlation for two time series, and when that's done, the intuition is not there. In fact, the procedure that I followed is very closely mirroring how correlation works. Roughly speaking, cross-correlation is the average value of the product of the standardized deviations of each series from its respective means.

bob: can you explain more what you mean? I'm not sure I'm following.


What I'm trying to say is that your own eyeballing method doesn't convince me. If you actually run the numbers, you'll see that those series are indeed similar.


What level of correlation do you define as "similar"?


Create 1,000 new series by randomly reordering the energy series, and I bet none of those will have as high a correlation with commercial building as the real energy series.

Or, for eyeballing, how about doing a scatterplot of energy vs. building? I'm guessing that would show some definitely structure as well.


A quick question; can you use the "standardizing" when the variables are not normally distributed?


Kikollan: This is a controversial one. For me, "standardize" is to express the data in relation to its dispersion. Typically - and always in the case of normal data - we center and divide by the standard deviation. However, one can standardize by dividing by the range, for example.

A totally separate issue is whether it makes sense to standardize non-normal data.


Very good test. I will show it to my students. Thanks

The comments to this entry are closed.


Link to Principal Analytics Prep

See our curriculum, instructors. Apply.
Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR.

See my Youtube and Flickr.

Book Blog

Link to junkcharts

Graphics design by Amanda Lee

The Read

Keep in Touch

follow me on Twitter