« The charting process | Main | Google trends »



Isn't it unsurprising that around 5% of the 1000 simulations turned out to have a p-value of <0.05? This is an artefact of choosing 0.05 as the significance level.


You raise a good point; p-values are of course not the end-all of any analysis.

The point to note is that if each simulation contains say 50,000 players instead of 761, then we'd expect every line to look flat.

Unfortunately, when sample size is large, p-values or t-tests do not help us. Using those, we'd still have to conclude that some of the lines are not flat; however, if plotted, it'd be obvious that all of the lines are essentially horizontal. That's the "practical significance" conundrum rearing its head again.


thank you this is enlightening.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)


Link to Principal Analytics Prep

See our curriculum, instructors. Apply.
Marketing analytics and data visualization expert. Author and Speaker. Currently at Columbia. See my full bio.

Book Blog

Link to junkcharts

Graphics design by Amanda Lee

The Read

Good Books

Keep in Touch

follow me on Twitter