Economists have their misery index; dentists, it seems, have a mystery index.
Laird Harrison, senior editor at DrBicuspid.com, an online newsletter for the dental community, pointed me to this chart when he interviewed me about how to interpret the findings in the latest Quarterly Survey of Economic Confidence, conducted by the American Dental Association. (Note: you have to register to read his article. Registration is free.)
When faced with an index, the first thing to do is to find out what the reference level (here, the zero level) means. Although the report is littered with dozens of similar graphs showing all kinds of indices, I cannot find any definition of the reference level, not even in the methodology appendix. The closest is the following directive for reading the chart I printed above:
For example, [this figure] illustrates that the Net Income Index improved by approximately 10% between 3rd and 4th quarters in 2009, an increase that was driven by 6% fewer dentists responding that net income had declined, approximately 2% more dentists indicating that net income was about the same, and 5% more dentists reporting that net income had increased.
For this survey question, respondents could answer that their Net Income increased, stayed about the same or decreased, and correspondingly, these answers were scored +1, 0, -1. But we still do not know what zero means in the Net Income Index.
Fortunately, the raw data was also provided. I plotted the net score differential, essentially the difference in proportion between those who reported income increase and those who reported income decrease:
The shape of this line looks eerily familiar. But what is the zero level?
After some investigation, I found the answer. The reference level is the net score differential, averaged over the six quarters shown on the chart. In essence, the blue line from this chart, if shifted up by the average net score differential, becomes the green line from the first chart.
How would we interpret such an index? The current quarter's differential was about -40% which was 3% below the average net score differential between 2008Q3 and 2009Q4 (which was -37%).
This index is very problematic. The choice of the past six quarters seems completely arbitrary and ignores any seasonality effect. The use of an unweighted average to average the score differentials assumes that there are no quarterly variations in the data.
But the biggest problem surfaces if one focuses attention on, say, 2008Q3. The top chart says that the net score differential for 2008Q3 was 2% above the average differential from 2008Q3 to 2009Q4. But this is a forward-looking number because in 2008Q3, it was not yet known what the net score differentials would be in the next 5 quarters. Usually, indices are constructed using historical data to establish the reference level.
The mystery is why indexing is even needed. What's wrong with plotting the change in net score differentials?
Reference: "Quarterly Survey of Economic Confidence, Fourth Quarter 2009", American Dental Association, Jan 29 2010.