How to Value Advertising

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

– John Wanamaker

Recently, Andrew Eifler has been thinking about the possibility of a bubble in digital advertising.

He raises a number of good points. In summary, they are:

1. People see unlimited growth potential for digital advertising. In fact, most internet-based companies launch intended to depend exclusively on advertising to support themselves.
2. Advertising is hard to value.
3. People are buying “new media advertising products” without a clear idea of how much they are worth.
4. Therefore, there is the chance of a bubble.

I’m going to agree with Mr. Eifler’s core conclusion, but disagree with a few of the assumptions he uses to get there.

First of all, advertising is inherently difficult to measure, as Mr. Eifler points out. You can do all the advertising you want, but the only metric you really know – or care about – is units sold.

There are two problems with this metric. The first is the (classic) issue of counterfactuals. Just because you sold x number of units with advertising, it doesn’t mean you wouldn’t have sold x number of units without advertising. Furthermore, just because sales declined by e units, you don’t know if the advertising was effective, or that without it units sold would have declined by 5e units.

Second, there’s the problem between long-term and short-term sales. A great deal of advertising is “informational” – you let the right people know that your product is available at a certain price. But more and more advertising is “aspirational” – attaching your product to a particular lifestyle. It’s done, crudely, in artistic or entertaining advertising; and a bit more subtly as product placement in TV shows.

The problem is how you measure the ROI of short-term and long-term advertising. For all you know, the ROI of your long-term advertising strategy could be negative – but how the hell would you detect that?

This problem of measurement is compounded by the proliferation of new advertising channels.

Now, I’d going to disagree with Mr. Eifler and say that the digital advertising is better than traditional forms of advertising, because you have (vastly) more metrics. In fact, you can even (sometimes) track an individual all the way from initially seeing an ad to purchase!

The additional metrics makes valuing easier, though I have to give a nod to Mr. Eifler and point out that it’s possible to “worship a false idol.” That is, you could construct an elaborate and mathematically beautiful valuing equation using all those metrics… that means precisely bugger all when it comes to ROI. The temptation of having metrics is using those metrics; but metrics are not reality, they are only a reflection of reality. They are subject to both measurement error and conceptual error – where what you think you are measuring is not, in fact, what you actually are measuring (the quintessential example is that IQ tests do not measure intelligence).

However, digital advertising leads to a larger problem – which is the proliferation of advertising channels.

Let’s say you spend $1m on TV advertising. Now, we can’t make many assumptions about how the effectiveness of that advertising will change as you raise spend to $10m, but it’s reasonable to assume that the relationship between spend and effectiveness is monotonic; that is, the effectiveness of your advertising campaign will not decrease if you double your spend.

However, the relationship could be anything from linear to (more likely some kind of S-Curve:

As you increase spending early on, advertising becomes more effective, per dollar. However, at some inflection point – in the middle of the graph above – advertising begins to become less effective per dollar. The effectiveness is still increasing; but each additional dollar gets you a little less than before.

(I’d appreciate it if some people in the advertising could check me on this wildly unfounded assumption).

Now, however, let’s say you spend $10 on a bunch of different channels – TV, radio, billboards, online banner ads, online search ads, sponsored viral videos, whatever.

How do you measure the effectiveness of not only each channel, but also all channels combined? This is not a trivial question; at the least, you need to consider 2ⁿ-1 factors, where n in the number of channels.. If you have TV and radio advertising, you need to consider (1) the effect of TV, (2) the effect of radio, and (3) the effect of radio and TV together.

If you bump that up to 3 channels – let’s say TV, radio, and billboards, then you have to consider (1) the effect of TV, (2) the effect of radio, (3) the effect of billboards, (4) the effect of TV and radio together, (5) the effect of radio and billboards, (6) the effect of TV and billboards, and (7) the effect of TV, radio, and billboards together.

To make matters worse, any significant interaction at the “higher” levels will invalidate a straightforward effect of any less complex combination. So if there is a 3-way interaction, then you cannot analyze TV independently; because if you say “TV works” that is conditional on both radio and billboards; you cannot then increase TV spend independent of radio and billboard spend and except to see it work more.

If you have 6 channels, let’s say, you’re looking at 63 different factors to understand.

To introduce another wrinkle – I know, sorry – the above assumes that everyone sees all advertising channels, i.e. that everyone comes from the same statistical population.

Except they don’t. Now, it would be simple if people who saw one channel didn’t see any of the other channels; but that isn’t true. Since you are not dealing with either the same population or independent populations, you need (somehow) to divide customers into “channel segments.” That is, people who see TV and billboard but not radio; people who see online ads and TV ads but not radio and billboards – etc.

This complexifies the problem, because you now need to figure out how each channel reacts to each combination.

Additionally, the above outlines a scenario where advertising is identical, e.g. increasing spend gets you more of the same. But not all advertising campaigns are made equal – the sheer creativity in the market means that the same dollar will get you a different product depending not only on which agency you hire, but also on your company. The agency may make a campaign which would be great for a company that is very similar but not the same as you. Or one that just slightly misses your target market, reducing its effectiveness.

So: the inputs are highly variable, the outputs are difficult to measure; even if you could measure them, interpreting them would be very close to impossible, even if you could do all that it’s still a counterfactual (you don’t know how close your conclusion is to the truth, since there’s no way to test it).

Therefore, accurately valuing advertising is pretty much impossible. You just have to be satisfied with the assumption of monotonicity, and vague claims that the interaction across channels is positive.

In the end, then, I agree with Mr. Eifler. But not because new media is hard to value; I think it’s easier to measure the post hoc effectiveness, which is a pretty good proxy. The digital scene may improve matters, due to additional metrics.

However, given the difficulty of valuing advertising and the expectation of unlimited growth, you certainly have the chance for a bubble.