Further thoughts on the Facebook business model, and the challenge of Web analytics

I have often grumbled about "story time!", the practice of spinning grand stories based on tiny morsels of data. It's not that I disapprove of story-telling per se -- it is that the story-teller has got to find evidence to support his/her stories.

A few days ago, I dissected the Trefis financial model used to support a $100 billion valuation for Facebook. A handful of aggressive assumptions must be believed to make it happen. So it is very pleasant to find in Business Insider some actual data to help us assess the credibility of some of these assumptions.

This chart gives the clickthrough rate (CTR) and cost per click (CPC) of Facebook ads.

For those not in this industry, CTR is the number of ads that get clicked on divided by the number of ads shown to Facebook users; and CPC is the average dollars paid by advertisers to Facebook for each click on their ads.

The chart does not provide per-advertiser data. Instead, advertisers are grouped by the industry they are in (health care, internet, etc.), and the aggregate results are shown.

The one thing that should jump out at us is the range of clickthrough rates: it's mostly in the range of 0.01 to 0.1. (The last one -- Tabloids and Blogs -- seems mislabeled, and the last two rows are sufficiently different from the rest that one would want to check the numbers again.)

Mind you, that is 0.01% to 0.1%. What does 0.01% mean? Yes, that's 100 clicks per 1 million ads shown to Facebook users. (My friend Augustine has long ago pointed out Facebook's abysmal metrics, relative to other advertising platforms. Look here for his perspective.)

***

Now, put yourself in the shoes of say Pfizer showing ads to Facebook users. Clicks don't equal revenues. Only some proportion of clicks would turn into sales. For illustration, say 5% of clicks lead to sales. With 100 clicks, they get 5 sales. They have to show 1 million ads to get 100 clicks. The clicks cost them $130 according to the data in the chart. If the value of each sale is more than $130/5 = $26, then Pfizer just about breaks even on the ads. 

What's on the advertiser's mind? 

One strategy is to flood Facebook with ads. More ads mean more clicks, even if the clickthrough rate is tiny. Perhaps unexpectedly, this sort of tactic works only to a limited extent. It bumps up against the law of diminishing returns. The clickthrough rate typically falls say when you double the number of ad impressions.

Another strategy is to use statistical models to selectively show ads only to Facebook users most likely to click on them. This raises the clickthrough rate. What it doesn't solve is the "quality" of Facebook users, or put differently, their tendency to pay attention to advertising.

***

Now, let's fancy yourself the person trusted to build these statistical models. You are trying to predict who's going to click on a given type of ad and who's not. What you have at your disposal is historical data on who got shown what ad, and whether they clicked or not. (You would typically want to grab any other data you can get your hands on, such as what the user has been doing on Facebook recently. You can get more creative, such as what Facebook has been secretly doing, described here.)

Let's say you are given the data on 1 million ads that were displayed. According to the above, you will find 100 clicks in this data... and 999,8999 999,900 non-clicks. Now, assume that you discover that there are some commonalities among the 100 users who clicked on the ad. Say, 50 out of the 100 signed on to Facebook after midnight, and live in the East Coast. That's a very strong signal.

Now, how many users would you find among the non-clicks who signed on after midnight and live in the East Coast? Oops, there would be multiples more than 50 such cases who did not click on the ad. This is all due to the tiny clickthrough rate.

That, in brief, is the challenge of Web analytics. If this excites you, there are lots of opportunities out there.

 

 

Comments

[Ken]

Pfizer would be expecting to get something from the non-clicks, in the same way that they get something from all their other advertising. In other forms of advertising effectiveness is usually evaluated by restricting the advertising to certain geographic areas and looking at the effect on sales.

[Matt]

Rightly or wrongly, the impressions/GRP argument will be made for it. However, targeting has always been a game at the margin(see Precision/Recall).

[San Nayak]

This is a wrong image. Last item is "Tabloids & Blogs", CTR should be .165% instead of .0165% I guess..

[Stefvanef]

Am I reading correctly: 1 milion ads, minus 100 clicks equal 999.899 non-clicks? Does that mean 1 undecided? It's probably a real-life example, since in my WA tools, I find glitches like this.
Great post, though. And it's probably what Ken means: Pfizer will be trusting/hoping for 'post-impression' converions.

[Staci Burruel]

Media and entertainment-related ads being on top of the CTR list was not a surprise for me, since most people go to the internet exactly for these things. Local search engine marketing firms are also getting a lot of clients wanting their entertainment-related campaigns to be advertised, meaning these people are definitely in-the-know about the advantages of advertising media and entertainment-related campaigns in social networking sites like Facebook. Local internet marketing, specifically in Facebook, requires a huge amount of R&D before the actual application to ensure that your money won't be wasted.

[noodle]

OK. So, I'm not in the industry but unless the industry uses a different system of mathematics your maths is out by a factor of 100.

0.01 = 1 click per 100 impressions

or 100 clicks per 10,000 impressions not 1,000,000 as you state.

Please feel free to correct me but if I'm not wrong I certainly won't be buying your book.

[Kaiser]

Noodle: the chart states 0.01% not 1%. The fact that you're looking at miniscule numbers is the reason for this post. And my book is not an arithmetic textbook - it's about statistical concepts.

Stevanef: you caught a typo of mine. In this industry, missings or data errors are treated as non-clicks. Only a positive click event is counted as a click. And while there should really be undecided events, say when an ad did not load completely, this type of situation is not typically tracked.

Ken: you're pointing out a hot-button issue in the advertising industry, which is that almost no one puts real value onto the value of "branding". Given that Facebook uses a cost per click, they have already conceded that only clicks are worth anything. I don't agree with this but what you infer from their pricing schedule is this.

[David]

You still have an arithmetic error: 1 million ads - 100 clicks = 999,900 non-clicks. Even a statistical conceptualist should get that one right...

The problem, though, reminds me of looking for genetic markers for rare disease -- one has very few cases but huge variation within the cases. No-one has figured that out yet, either.

Out of interest, what are "normal" CTRs? (say, for the nytimes?)

[Kaiser]

David: That's fixed now. Thanks for your comment. There is certainly a similarity with genetics. The difference is that people put a lot more trust in the genetics findings just because of the subject matter, which in my mind, is misplaced. That is the subtext for my post on genetic testing here.

All: While I appreciate errata, I don't understand why they are being delivered with a smack on the face. After all, this is a blog and not a refereed journal.

[David]

It was meant more as a wry prod rather than a slap in the face... Surely there is some humour in quantitative pros making basic errors of arithmetic. I fear your friend Noodle didn't find it quite so amusing.