The meaning of most

Megan McArdle started the war on infographics (link). And reader Danielle A. contributes this example, from KissMetrics.

This is one part of a big infographics poster. Needless to say, a bar chart renders this data much better:

The categories are sensibly sorted, and useless tinges of color removed.

***

But I want to draw attention to their conclusion:

Most participants in the survey would wait 6-10 seconds before they abandon pages.

Now we know writers of opinion pieces in the major newspapers have long lost control over the titles of their pieces. Is it true that graphic designers have ceded control over their conclusion statements?

It would appear so. The category being labeled as "most participants in the survey" accounted for 30% of the respondents. When is 30% considered "most"?

Also, surveys are typically tools for generalization so we expect conclusions about the general population of mobile users. Here, whoever wrote this conclusion timidly restricted the remark to "participants of the survey". This is probably an oversight because in other panels, they talk about x% of consumers or y% of mobile internet users. If the survey was probably designed and executed, they should be confident about the whole population, not just the sample.

Finally, nowhere on this poster can you discover which survey this data came from. We have no idea what the sample size is, nor the margin of error.

Comments

[Ananda]

Short of coming up with a much better chart, they might be better off stating their results as something like "Almost half of the participants in the survey expect your page to load within 10 seconds" rather than using such a flimsy definition of "most".

[John]

I don't think a pie chart is so bad for this, though this one isn't good: the slices are in a bizarre order. If they were ordered differently you could, for example, much more easily than in the bar graph sum up adjacent slices. This would make it easy to tell what fraction of readers would wait for _at least_ 6 seconds.

[Rahul]

Actually isn't a histogram exactly what you are looking for here? Twist it around 90 degrees.

[Phil H]

As I've just blogged on philviz.blogspot.com, if you turn this around and look at the cumulative loss/retention at each time you see that there is a consistent decay curve all the way out. From the graph it seems that 50% of users have gone at around the 8s mark, and the first quarter of users have gone at around 2.5s.

I tried to ask the question that site owners/developers would ask; how quick does my site need to be? And the answer is as quick as possible, and really inside 2.5s.

[Doug]

It's a case of being accidentally correct: Since 81% of respondents would wait greater than 5 seconds, "most" would wait 6–10 seconds (some even longer).

[Ben]

The pie chart also fails in how the segments are positioned. They've clearly ordered it In terms of increasing percentage, but a more useful order would have been the increasing time ranges. If you did it that way it's much clearer that 49% of respondents would wait less than 10 seconds.

[Ben]

It's also worth pointing out that there are more accurate ways of determining the abandonment rate than can be achieved by a survey: actual user metrics. I would suggest that self-reporting over-represents the amount of time people are willing to wait, but that would require a study that I'm OT aware has been done.

The data being presented in this way is also non-discriminatory to types of content. You'd expect people to be more willing to wait longer for a YouTube video to load than a news article, so somebody who watches a lot of video content is more likely to answer with a longer time than somebody who doesn't. Either way, without breaking the data down it's not overly useful for a web developer.

[Anna Lopez]

thanks for the charts it makes it easier to understand your piece, but as an avid blogger and participating on online discussion your version of a pie chart is easier to read,

LowkeyMedia Marketing Team