You can follow this conversation by subscribing to the comment feed for this post.

More initial thoughts:
9. Graph your data. Look for the unexpected
10. Consider re-expression to simplify structure: make distributions of individual variables more nearly symmetric, make scatterplots straighter, make tables more nearly additive
11. Then (AFTER re-expressing) look for and deal with outliers--both in each variable and possibly in pairs or multiple variables together (e.g. the tall, thin subject who is neither extraordinarily tall nor extraordinarily thin, but together is a medical outlier.) Don't allow outliers to dominate your analysis
12. Graph your data again in at least one other way than you did at step 9.

Along those lines, a different set of posts is needed for fixing problems, and how not to make things worse.

This is great - thanks so much!

I strongly agree that graphing data in various ways can help. A great example of how your data can fool you is "Anscombe's quartet".

Very appealing dinosaur-based animated examples are given in the paper: "Same Stats, Different Graphs: Generating Datasets with Varied Appearance and Identical Statistics through Simulated Annealing" by Justin Matejka and George Fitzmaurice, available here: https://www.autodesk.com/research/publications/same-stats-different-graphs

AB: Agreed. I do lots of boxplots, pdfs and cdfs because the visuals are clearly much more efficient and effective than staring at a list of summary statistics.

Also frequency histograms are useful. I found one where the data was given in two different units. Also with consulting I make it clear that they wont receive anything until the data is clean, so they had better answer my e-mails.

Hey Kaiser,

One tips I'd to look for data's that looks like from top down created.
Here is some data from ground view in covid. So build up picture levels should matching somehow with this

https://doi.org/10.1007/s00134-020-06267-0

Hey,

So some new bottom up data dumping. Same sources up to date but bit mess.
ASD

Can you see interesting things?

The comments to this entry are closed.

##### Get new posts by email:
Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

## Search3

•  only in Big Data
Amazon - Barnes&Noble

Numbersense:
Amazon - Barnes&Noble

## Junk Charts Blog

Graphics design by Amanda Lee

## Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

## Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here