Half of advertising is wasted. Which half is now knowable

Ask any working advertiser to name the founding text of their profession, and most will recite the same line without knowing where it comes from: half the money spent on advertising is wasted, the trouble is nobody can say which half. It is almost always attributed to John Wanamaker, the nineteenth-century retailer whose department stores made him one of the shrewdest merchants of his age, and for a century the industry has repeated his supposed confession as a kind of professional creed — proof that even the best practitioners operate half-blind. There is one problem. Nobody has ever found Wanamaker actually saying it.

Quote Investigator, which traces disputed sayings to their first appearance in print, found the earliest confirmed version in an August 1919 speech at the Winona Bible Conference in Indiana, where a minister named Roy L. Smith quoted Wanamaker as saying: “I am convinced that about one-half the money I spend for advertising is wasted, but I have never been able to decide which half.” That was three years before Wanamaker’s own death, and it still arrived secondhand, through a sermon rather than a ledger. The sentiment is older still: Printers’ Ink ran a version in October 1890 — “don’t forget that half of the money spent in advertising is wasted” — and in 1898 a Wanamaker executive, Robert C. Ogden, told The Bankers Magazine he believed “fully fifty per cent. of the money spent on advertising was wasted through being improperly placed.” The tidy, self-deprecating version seems to have assembled itself over three decades and been handed to the most famous merchant available; the same maxim has separately been credited to a soap magnate, a tobacco man, a chewing-gum manufacturer, and the adman David Ogilvy, who used it but declined credit for coining it. An industry that could not agree who said the sentence had even less hope of testing it.

The blackout that told eBay the truth

For a century that hardly mattered: there was no practical way to isolate an ad’s effect from everything else in a customer’s life. Digital advertising promised to change that — every click, every purchase could in principle be traced to its source. What it mostly delivered was a more precise-looking version of the same confusion, because a shopper who clicks a paid ad for a brand she already meant to buy from looks, to an attribution dashboard, exactly like a shopper the ad persuaded. Both count as a win.

Working with eBay’s own research team, the economists Tom Blake, Chris Nosko and Steven Tadelis set out to separate the two. They halted eBay’s paid-search bidding for every keyword containing the word eBay — “ebay shoes,” “ebay coupon codes” — on Yahoo and Microsoft, while continuing to buy the identical keywords on Google, which served as an internal control group. eBay was, by their account, managing more than 100 million keywords and keyword combinations across all three engines at once. The result of switching off the branded ads: almost all of the forgone clicks, and the sales credited to them, simply reappeared in unpaid organic search. The advertising had not created the sale — it had intercepted a customer who was coming anyway.

The team then went further, holding a random thirty percent of eBay’s United States traffic out of all non-brand search advertising for sixty days. The effect on sales was small and statistically insignificant on average. Splitting frequent shoppers from occasional ones sharpened the picture further: the ads moved new and infrequent customers but did nothing measurable for loyal ones, who nonetheless accounted for most of the sales credited to the ads. The short-term return on eBay’s search advertising, the paper concluded, was negative — not because the ads did nothing, but because the company was mostly paying to reach people who needed no reaching at all.

Advertising has a quantitative veneer that belies the true underlying uncertainty.Randall A. Lewis and Justin M. Rao, “The Unfavorable Economics of Measuring the Returns to Advertising,” Quarterly Journal of Economics, 2015

The price of certainty

The eBay result made a persuasive case for testing over trusting a dashboard. It also, inadvertently, showed why so few companies do it well. Around the same time, economists Randall Lewis and Justin Rao pooled twenty-five randomized experiments run with nineteen retailers and six financial firms, together representing $2.8 million in digital ad spending, in most cases reaching millions of customers. Individual sales, they found, are so volatile relative to the cost of a single ad exposure — a “coefficient of variation” of about ten was typical — that a genuinely profitable campaign can look statistically identical to a worthless one unless the test runs at real scale. Their median retail experiment produced a standard error on ROI of 26.1 percent, implying a confidence interval more than one hundred percentage points wide, wide enough to contain a rousing success and an outright loss alike; for financial-services experiments the median error was 115 percent. Telling a dead campaign from a profitable one, they calculated, can need sample sizes past ten million person-weeks — an honest measure of how large, and how expensive, Wanamaker’s problem really is.

Testing by geography, not by person

One answer, developed independently at Google around the same period, gives up on individual-level measurement altogether. Researchers Jon Vaver and Jim Koehler proposed geo experiments: dividing a country into non-overlapping regions, randomly assigning each to receive advertising or not, and comparing the aggregate results. “The most rigorous method of measurement is a randomized experiment,” they wrote, and a geo experiment achieves one without following any customer’s clickstream; grouping regions of similar size before randomization, they found, could tighten the confidence interval by ten percent or more. The eBay researchers noted, in a footnote, that their own non-brand test used a methodology “similar to one proposed by Vaver and Koehler” — two teams, at a retailer and a search engine, converging on the same fix: randomize something cruder than a person, and the noise that defeats individual-level testing averages out.

That idea has migrated well beyond Google. Meta has open-sourced a tool called GeoLift, which estimates incremental lift using synthetic-control methods across geographic markets rather than individual devices. Its documentation states the motive plainly: “lift studies are often gated by the access to online signals (pixels, APIs, SDKs, etc.),” and advertisers who lack that access, are losing it, or would rather not share it “weren’t able to leverage incrementality” until geography offered a workaround. As tracking individual users grows harder and more contested, geo experiments have moved from one search engine’s research curiosity to freely available code any retailer can run.

What Wanamaker never got to know

None of this hands a marketing department a tidy percentage for a slide deck. If anything, the research argues the opposite: measuring advertising’s true effect is harder, and costlier, than any dashboard lets on. But something has genuinely changed since a preacher in Indiana put a tidy epigram in a dead merchant’s mouth. The shrug that made “which half” an unanswerable joke is no longer mandatory. A company willing to turn a campaign off in some regions and not others, and wait long enough to see what happens, can now find out — not with certainty, but with an honest number attached to the uncertainty, which is more than advertising has ever previously managed to say about its own spending. Wanamaker, whoever actually coined his line, would probably have settled for that.