Projecting iPad sales

At Business Insider, the headline screams: "The iPad is on fire: Apple sold 120,000 yesterday". (Original article at Brainstorm Tech.)

This sort of predictions is issued by pundits every day on all kinds of things. Are they believable? How can we tell?

***

Here're a bunch of things I discovered about this projection, along with questions you'd want to ask:

• The projection comes from an Apple Sanity Board at Investor Village (private), which from the article I presume includes someone called Deagol who has been following Apple stock for a long time. They analyze the index of web order numbers to estimate number of orders on iPad's Day One, substract an estimate of baseline order rate, then multiply the average number of iPads in each order to project iPad sales.
• They use a really creative and promising methodology for collecting data: they ask customers to email in the specifics of their orders.
• Excess orders: they assume that every order in excess of the normal rate contains the average number of iPads (1.1). But the average number of iPads per order is computed from those who reported at least one iPad purchased. So do you believe the assumption that all those excess orders bought at least one iPad?
• Selection bias 1: the web order numbers are collected by an opt-in process. This absolutely creates selection bias, probably making the estimate overly optimistic. Responders are more likely to have bought iPads -- what's the incentive (apart from a hearty desire to help statisticians) of an Apple customer who did not buy an iPad to respond?
• Selection bias 2: web orders may or may not be indicative of store sales. They acknowledge this, saying it does not count reservations (probably taken at Apple Stores). To get a real estimate of total iPad sales, one would need to extrapolate from web orders to in-store orders.
• Selection bias 3: who would know about this opt-in survey? how did the Sanity Board publicize its data collection campaign?
• Second-order effects: typically, successful new products have a halo effect, meaning that they bring customers into the "store", and these customers will likely buy other products as well. That is the logic of "door-buster" deals and so on. So the assumption that all other products sold at their normal rates is probably too simplistic.
• Decomposition: the way a statistician might think about this problem is to use the pre-iPad data to establish a projection of the baseline sales (instead of a flat average), look at the differences over time between the observed volumes and the projected pre-iPad sales, and model the "residuals" including a term for the halo effect. (Adjusting for selection biases also need to be addressed.)

• Subgroup breakdown: Given the small sample size, you must be skeptical about drilling down further to look at preferences for WiFi or 64GB. The preference for WiFi is very large and can be believed; I wouldn't bet anything on the equal split between 16, 32 and 64 GB models.
• Customer segments: the people who buy iPads on Day One are probably not representative of future customers. I'd treat this as an estimate of the rate of early adoption, as opposed to an indicator of future sales.
• Unverifiability: they did not verify the orders or detail how they deal with partially bad data. Since they are looking for a rough estimate, this is probably not an important enough issue. But in order for this type of methodology to have legs, someone ought to figure out this issue.

Comments

[CrisisMaven]

For statisticians, educators, students and researchers: I have put one of the most comprehensive link lists for hundreds of thousands of statistical sources and indicators (economics, demographics, health etc.) on my blog: Statistics Reference List. And what I find most fascinating is how data can be visualised nowadays with the graphical computing power of modern PCs, as in many of the dozens of examples in these Data Visualisation References. If you miss anything that I might be able to find for you or if you yourself want to share a resource, please leave a comment.

[Kaiser]

CrisisMaven: I'm debating whether to delete your irrelevant comment. You have the temerity to come to my blog and advertise your own site, and yet you fail to include Junk Charts in your Data Visualisation References, or Numbers Rule Your World in your Statistics Reference List. I see that you are peppering the blogosphere with these comments; my advice to you is to learn some blog etiquette.

[MIKE SELBY]

Kaiser:
Let me illuminate!
AAPL SANITY is a private board. A person has to request to become a member and has to be accepted by the members. There are perhaps 300-500 different active members on a given day.
TMO AFB is another board for Apple fanboys. There are over 17,000 members and XXXXX visitors per day. DEAGOL is a valued member at both boards. So am I. There is a certain amount of duplication.

DEAGOL has been making "estimates" of all things Apple for 4 years.

HE is usually MORE accurate than 90% of the financial analysts who follow Apple.

Anyone wishing to read Daniel Tello’s (AKA DEAGOL) comments from the last 4 years can start: http://www.macobserver.com/tmo/forums/observer/7971/
Just click on: VIEW ALL POSTS BY THIS MEMBER.

As you will determine by reading, Daniel usually beats MOST of the analysts following AAPL.

[Kaiser]

Mike,
As I indicated, I am impressed by the method. Thanks for pointing to the detailed messages; I was blocked before. I hope that you guys will consider some of the issues I pointed out, and improve it further. In particular, improving the estimate of the number of iPads bought per order (not per order that includes an iPad) would help a lot. I am not surprised that this method works better than some of the others out there. Good work.

[MIKE SELBY]

Kaiser:
AAPL Sanity will block you, TMO AFB should not. Daniel has over 400 posts there. He also has his own website listed on his posts at TMO AFB.
Hope to see you there soon.