One oft-repeated "self-evident" tenet of Big Data is that data end all debate. Except if you have ever worked for a real company (excluding those ruled by autocrats), and put data on the table, you know that the data do not end anything.
Reader Ben M. sent me to this blog post by Benedict Evans, showing a confusing chart showing how Apple has "passed" Microsoft. Evans used to be a stock analyst before moving to Andreessen Horowitz, a VC (venture capital) business. He has over 25,000 followers on Twitter.
I'll get to this chart later but feel free to tour around the comments area. You will get a feel for the types of conversations that happen when an analyst offers data to a corporate meeting. It turns out that everyone has an opinion about everything, and while data people think the only way to make an argument is by presenting data, there are plenty of other people from other background who use other modes of persuasion, like rhetoric, anecdote, philosophy, and HIPPO (i.e. HIghest Paid Person's Opinion).
The original post plus these comments present a mishmash of metrics that can be used to measure the gap between Microsoft and Apple. Here is the list roughly in chronological order up to the point where the post had 70 comments:
From the original chart, we have PC shipments, iphone and ipod shipments, shipments of all devices running MacOS or iOS (possibly excluding AppleTV), PC shipments against all devices running iOS, PC plus smartphone (excluding tablets and music players) shipments against all devices running iOS.
Then in the comments, people are bringing up: Xboxes (a gaming device running on Windows OS which is almost a PC), Android devices (which is part of the Google universe), profits instead of units, servers, Google, Macs that are running Windows OS, whether it is accurate to say "Apple passes Microsoft" using the blogger's own metric, whether one needs to wait more than one quarter to make such a conclusion, TVs, printers, the difference in technology that resides inside the same device (say the PC), whether the fourth quarter result can be generalized, embedded devices, self-built PCs, video-editors on the Web, and Amazon.
Evans responded to many of these comments by complaining that readers are not getting his message. That's an accurate statement, and it has everything to do with the looseness of his data. This reminds me of Gelman's statistical parable. The blogger here is not so much interested in how strong his evidence is but more interested in evangelizing the morale behind the story.
His primary thesis is quite likely correct. There is a huge trend of U.S. consumers spending more time on mobile devices, mostly smartphones and also some tablets. I see this at work, and the trend is widely recognized in the tech world by this time.
But the readers are also right to point out the deficiency of that chart.
Using my Junk Charts trifecta checkup, we'd say the failure of the chart is due to the Data corner. The chart addresses an interesting Question, and the Graphical elements are acceptable but the Data just do not do justice to his credible thesis.
This reader for example supports Evans, and blames other readers for an outdated way of thinking about the computer industry:
That's fair, except that Evans is also guilty of old-school thinking when he used unit sales of devices as the metric to compare Apple and Microsoft. What he really needs is data on time spent. This can be obtained from Nielsen or Gallup polls or some such venue. Just count the number of hours the average person spends on iOS devices versus Windows devices - regardless of whether they are phones, PCs, laptops, gaming consoles, music players, or embedded devices. This is sometimes called a "mindshare" metric.
To get even more sophisticated, we should merge mindshare with revenues (or profits) generated. Different companies have different business models. Those that give away services (or hardware) for free (or close to free) can gain mindshare at the expense of revenues (or profits). I like a metric such as dollars generated per hour used.
Back to my original point. Doing data analysis is a good first step. The bigger challenge is pushing people with preconceptions to believe the analysis (with all its imperfections and assumptions) and to change their minds.