« Not taking numbers at face value, even if they come from Google | Main | Prestigious college desperately plays down shameful manipulation of statistics »


Feed You can follow this conversation by subscribing to the comment feed for this post.


Very good question. A similar idea exists in finance where people strive to find "undervalued" stocks. However, in finance there exists much more information that past stock price (i.e. past ratings). Seeing that most facts are known by everyone, insider information is very useful. (But regulated by the government, so don't do it)

I propose that a restaurant can be 'rated,' (i.e. valued) using a number of public metrics:

Age of Restaurant - (long lasting ones are probably better than new ones)
Rent of Building - (ambiance, price etc.)
Prices on Menu - (low price with high rents means we're getting a good deal and/or volume is high, indicating many people eat there)
# of different meats - (options, variety)
# of Vegetarian/Vegan options - (again options)
Demographics of the zip-code - (population etc.)

One thing that would be very good to discover is where the ingredients come from and how recently they were ordered. Luckily companies are becoming more open to their sourcing and it may be possible to ID that. I know of a local seafood company: http://www.goincoastalseafood.com/ that specifically tells the diners were the seafood came from and when it was caught. Could text mining of the menu + demographics give a better idea than 'ratings'?

Hmmmm maybe I'll get cracking on this idea.


When I rank schools or restaurants or towns for Boston/NY/Philly magazines, I usually fit a model and compare (for example) predicted mean/median house prices in a town with actual prices so the towns that have prices below the prediction are 'underrated' or 'best bang for the buck.'

I don't think a survey would be a good way to do this.


I would argue that "underrated" doesn't mean what it sounds like, "less well rated than it deserves to be," but rather the slightly different "less often mentioned or thought about than it deserves to be." You see this interpretation reflected in a lot of the responses at your link: "You know what people aren't talking about any more? Pig foot heaven..." "When it comes to sushi these days, you only hear about..." "Yes, they got a good review in the Times, but they've been ignored otherwise." It's not that people don't think they are bad restaurants, it's that they don't think of them at all.

Overrated thus seems to reflect a particular imbalance between popularity and quality, possibly focused on being popular among a subset of relative experts. While quality is one factor of popularity, especially among experts, so too is location, trendiness of cuisine, novelty, existing hype, and lots of other factors. The statistical meaning that I could imagine, then, would be something to do with number of reviews, blog posts, and tweets about a restaurant, or maybe even a questionnaire about restaurant recommendations, (perhaps along with number of bookings or lunch turnover for, say, a food cart) versus the rated quality of food among those places. You'd then want to look places with a quality consistently as high as the best, but a low outlier in terms of discussion frequency.


I agree with Casey's definition of underrated. Asking people what restaurants they feel are "less discussed" eliminates well-known restaurants from the discussion. Therefore, the population of "all restaurants in NYC" is effectively reduced and your "average" opinion is therefore not a true population average.

Avner Shahar-Kashtan

One method of identifying underrated items is used by the AnimeNewsNetwork site. It maintains a rating of anime movies and TV shows based on user rankings, but normalizes the results based on the number of votes. If a show has fewer votes, it is pulled towards the average score for a show. The more votes, the more the score is the true mathematical average of votes. This serves to prevent a small but rabid fan base from pushing a show up, but it also serves to create the Underrated metric: a show that has few votes, but those votes are very high, is considered underrated. The premise is that this show isn't very well known, but very well loved by those that do.

Here's the link to the data:


The population rating can't be the intrinsic rating.
If I *personally* think something's underrated, that means I believe it deserves a higher (intrinsic) rating than the current (population) rating, whether the latter's estimated from a poll or counted by a census.

So "underrated" can only have a consensus meaning if there is a consensus on how to measure the intrinsic rating. That can't be measured by the same census/poll used to get the population rating.

Instead, let's say everyone agreed that you can measure restaurant quality by taking a panel of trusted expert food critics and forcing them to reach unanimous agreement.
(Or say we all agreed "volume of food per dollar" was a good metric, or whatever.)
You could do this; then take a poll or census among the general population; and see if they differ.

As for sub-populations: If you can't get a general consensus on the metric to use, at least you could find subgroups who agree on a metric, and then say "people who use metric X think it's underrated but people who use metric Y think it's overrated."

(Also, great points above: it's underrated if it "deserves" to be rated high, but most people rate it low *or don't think about it at all*. The population poll would have to treat "I don't know, I never think about it" as a useful response category.)

I'm really not sure how to describe what the Eater poll is measuring.


Two meanings come to mind:

Under-rated = under-hyped (more attention should be paid)

My personal rating is higher than the average rating


Can it be one of the options that is not the ideal choice from the subset of all ideal choices?

The Econometrician

Kaiser: Interesting question! I wrote up a few thoughts of mine at my blog, casualinference.com. (Shameless plug-but I'm hoping there are complementarities between our readership). I assume that there's a true ranking though, which helps a lot!

John S.

One way this could happen is if the rating measures the wrong thing, or is influenced by factors other than the most important ones. For example, a restaurant close to the city center could be rated higher than an identical place in an outlying suburb due to a halo effect from the surrounding area. The effect of sample size would also suggest that frequently rated restaurants have more accurate ratings than restaurants that are less frequently rated. If you could discover these hidden variables, you could possibly control for them and discover some restaurants that are underrated.

The comments to this entry are closed.

Get new posts by email:
Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.


  • only in Big Data
Numbers Rule Your World:
Amazon - Barnes&Noble

Amazon - Barnes&Noble

Junk Charts Blog

Link to junkcharts

Graphics design by Amanda Lee

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here

Principal Analytics Prep

Link to Principal Analytics Prep