« Not taking numbers at face value, even if they come from Google | Main | Prestigious college desperately plays down shameful manipulation of statistics »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Kris

Very good question. A similar idea exists in finance where people strive to find "undervalued" stocks. However, in finance there exists much more information that past stock price (i.e. past ratings). Seeing that most facts are known by everyone, insider information is very useful. (But regulated by the government, so don't do it)

I propose that a restaurant can be 'rated,' (i.e. valued) using a number of public metrics:

Age of Restaurant - (long lasting ones are probably better than new ones)
Rent of Building - (ambiance, price etc.)
Prices on Menu - (low price with high rents means we're getting a good deal and/or volume is high, indicating many people eat there)
# of different meats - (options, variety)
# of Vegetarian/Vegan options - (again options)
Demographics of the zip-code - (population etc.)

One thing that would be very good to discover is where the ingredients come from and how recently they were ordered. Luckily companies are becoming more open to their sourcing and it may be possible to ID that. I know of a local seafood company: http://www.goincoastalseafood.com/ that specifically tells the diners were the seafood came from and when it was caught. Could text mining of the menu + demographics give a better idea than 'ratings'?

Hmmmm maybe I'll get cracking on this idea.

Datacooker

When I rank schools or restaurants or towns for Boston/NY/Philly magazines, I usually fit a model and compare (for example) predicted mean/median house prices in a town with actual prices so the towns that have prices below the prediction are 'underrated' or 'best bang for the buck.'

I don't think a survey would be a good way to do this.

Casey

I would argue that "underrated" doesn't mean what it sounds like, "less well rated than it deserves to be," but rather the slightly different "less often mentioned or thought about than it deserves to be." You see this interpretation reflected in a lot of the responses at your link: "You know what people aren't talking about any more? Pig foot heaven..." "When it comes to sushi these days, you only hear about..." "Yes, they got a good review in the Times, but they've been ignored otherwise." It's not that people don't think they are bad restaurants, it's that they don't think of them at all.

Overrated thus seems to reflect a particular imbalance between popularity and quality, possibly focused on being popular among a subset of relative experts. While quality is one factor of popularity, especially among experts, so too is location, trendiness of cuisine, novelty, existing hype, and lots of other factors. The statistical meaning that I could imagine, then, would be something to do with number of reviews, blog posts, and tweets about a restaurant, or maybe even a questionnaire about restaurant recommendations, (perhaps along with number of bookings or lunch turnover for, say, a food cart) versus the rated quality of food among those places. You'd then want to look places with a quality consistently as high as the best, but a low outlier in terms of discussion frequency.

Kenny

I agree with Casey's definition of underrated. Asking people what restaurants they feel are "less discussed" eliminates well-known restaurants from the discussion. Therefore, the population of "all restaurants in NYC" is effectively reduced and your "average" opinion is therefore not a true population average.

Avner Shahar-Kashtan

One method of identifying underrated items is used by the AnimeNewsNetwork site. It maintains a rating of anime movies and TV shows based on user rankings, but normalizes the results based on the number of votes. If a show has fewer votes, it is pulled towards the average score for a show. The more votes, the more the score is the true mathematical average of votes. This serves to prevent a small but rabid fan base from pushing a show up, but it also serves to create the Underrated metric: a show that has few votes, but those votes are very high, is considered underrated. The premise is that this show isn't very well known, but very well loved by those that do.

Here's the link to the data:
https://www.animenewsnetwork.com/encyclopedia/ratings-anime.php#most_underrated

Jerzy

The population rating can't be the intrinsic rating.
If I *personally* think something's underrated, that means I believe it deserves a higher (intrinsic) rating than the current (population) rating, whether the latter's estimated from a poll or counted by a census.

So "underrated" can only have a consensus meaning if there is a consensus on how to measure the intrinsic rating. That can't be measured by the same census/poll used to get the population rating.

Instead, let's say everyone agreed that you can measure restaurant quality by taking a panel of trusted expert food critics and forcing them to reach unanimous agreement.
(Or say we all agreed "volume of food per dollar" was a good metric, or whatever.)
You could do this; then take a poll or census among the general population; and see if they differ.

As for sub-populations: If you can't get a general consensus on the metric to use, at least you could find subgroups who agree on a metric, and then say "people who use metric X think it's underrated but people who use metric Y think it's overrated."

(Also, great points above: it's underrated if it "deserves" to be rated high, but most people rate it low *or don't think about it at all*. The population poll would have to treat "I don't know, I never think about it" as a useful response category.)

I'm really not sure how to describe what the Eater poll is measuring.

zbicyclist

Two meanings come to mind:

Under-rated = under-hyped (more attention should be paid)

My personal rating is higher than the average rating

Jonathan

Can it be one of the options that is not the ideal choice from the subset of all ideal choices?

The Econometrician

Kaiser: Interesting question! I wrote up a few thoughts of mine at my blog, casualinference.com. (Shameless plug-but I'm hoping there are complementarities between our readership). I assume that there's a true ranking though, which helps a lot!

John S.

One way this could happen is if the rating measures the wrong thing, or is influenced by factors other than the most important ones. For example, a restaurant close to the city center could be rated higher than an identical place in an outlying suburb due to a halo effect from the surrounding area. The effect of sample size would also suggest that frequently rated restaurants have more accurate ratings than restaurants that are less frequently rated. If you could discover these hidden variables, you could possibly control for them and discover some restaurants that are underrated.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Marketing and advertising analytics expert. Author and Speaker. Currently at Vimeo and NYU. See my full bio.

Next Events

Mar: 26 Agilone Webinar "How to Build Data Driven Marketing Teams"

Apr: 4 Analytically Speaking Webcast, by JMP, with Alberto Cairo

May: 19-21 Midwest Biopharmaceutical Statistics Workshop, Muncie, IN

May: 25-28 Statistical Society of Canada Conference, Toronto

June: 16-19 Predictive Analytics World (Keynote), Chicago



Past Events

Feb: 27 Data-Driven Marketing Summit by Agilone, San Francisco

Dec: 12 Brand Innovators Big Data Event

Nov: 20 NC State Invited Big Data Seminar

Nov 5: Social Media Today Webinar

Nov: 1 LISA Conference

Oct: 29 NYU Coles Science Center

Oct: 9 Princeton Tech Meetup

Oct: 8 NYU Bookstore, NYC

Sep: 18 INFORMS NYC

Jul: 30 BIG Frontier, Chicago

May: 30 Book Expo, NYC

Apr: 4 New York Public Library Labs and Leaders in Software and Art Data Viz Panel, NYC

Mar: 22 INFORMS NY Student-Practitioner Forum on Analytics, NYC

Oct: 19 Predictive Analytics World, NYC

Jul: 30 JSM, Miami

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Search3

  • only in Big Data

Community