All Insights

Measuring Business Results Beyond the Last Click

The promise of digital advertising is that it is more measurable and actionable then traditional TV, print, radio, or billboards. But in practice, it can be complicated. How many sales did that homepage takeover drive? Do display ads boost the efficiency of search spend? Did the retargeting buy really help close a sale, or was that consumer going to purchase anyway, motivated by upper-funnel activities?

As a result of this ambiguity, the digital advertising industry has focused on the last click, giving credit for a conversion to the last ad a person interacted with, the ad that delivered the consumer to the advertiser’s website. But doing this doesn’t tell the whole story.

Imagine watching a soccer match where you can only see the shots on goal. Some shots score, and some miss. You can only see the scoring kick but none of the other players on the field. This is like last-click attribution. Who’s your most-valuable player on the field? Well, you don’t really know.

With view- and click-tag data fed into a multi-touch attribution model, you can suddenly see all of the players on the field. You’re able to determine which players set up the play, provided the assist, and delivered value beyond just that final shot on goal. Without that crucial assist the goal might never have happened!

When you focus on the last click, you often see that retargeting buys and search ads get a large share of the credit for your sales. But what ads drove a person to your site, or to search for your brand, to begin with? Which ads drove that action that you could subsequently retarget? When a person searches for “Samsung Galaxy S III” rather than “cellphone” or “VW GTI” rather than “car,” it is clear that they’ve been exposed to your product and messages further upstream. Last-click attribution ignores all of those marketing efforts and leaves you blind to the value they create. Optimizing your campaign under a last-touch view means you could inadvertently remove the portion of the campaign that’s actually delivering the results.

Multi-Touch Attribution (MTA) models take into account all of the touch points in the path to conversion. Rules-based models like Even Credit and Time Decay, are simple to implement and provide a useful check on your understanding of value. While neither represents the true marketing funnel, toggling back and forth between models can illustrate just how arbitrary last touch actually is. By comparison, statistical MTA models analyze the correlation between ad exposure and conversion teasing out the effect of advertising to the various touch points. These models leverage your historical ad serving data to build a more realistic picture of the complex journey to purchase.

While implementing MTA takes some effort, there are real costs to inaction. Continuing to optimize for the last click rather than for your actual business results is a waste of money. A 2013 eBay study reports, "Results show that brand keyword ads have no short-term benefits, and that returns from all other keywords are a fraction of conventional estimates.” By optimizing for the last click, eBay failed to measure whether that last click actually drove sales. When eBay focused on business outcomes rather than clicks, it “found that most customers would have clicked through to a particular site without being prompted by an ad for the company.”

Beyond MTA, Causal Lift Tests

Lift Tests, exposing some users to ads while holding out others, are the gold standard for ad effectiveness. They are the only way to isolate the causal impact of advertising on conversion events. Think about how the Food & Drug Administration (FDA) approves a new cancer treatment. They randomly split a group of patients into Treatment and Placebo groups, administer the drug or a sugar pill, and monitor health status over the subsequent months and years. This randomization allows the tester to isolate the impact of treatment on outcomes. In a similar way, online marketers use randomization to treatment groups to estimate the impact of different advertising treatments.

The current practice of online Lift Tests falls well short of the standard clinical trial approach. Michael Dell recently said that the average person will have 10 internet-enabled devices by 2016. As a result, a typical user will span multiple cookies and often ends up in the Treatment group on one device but in the Placebo group on another device. The Treatment and Placebo groups are contaminated which invalidates the results. Think back to the FDA cancer treatment scenario. A patient is assigned to the Placebo group, but he slips out and sees an outside doctor and receives the cancer drug. The results from the test are mooted, because the Placebo group was actually exposed. Advertising Lift Tests are flawed unless the test is structured around a single, unifying people-based identifier across devices and platforms, rather than cookies that simply can’t span devices or platforms.

Lift Tests help answer several key questions. For example, what is the difference in conversion rate between the Treatment and Placebo groups? How many incremental sales do digital campaigns actual deliver? If the CFO approves additional marketing budget, how much more should manufacturing expect to build? By how much should Sales quotas be raised?

Lift Tests also inform the appropriate lookback window for MTA models. At the start of the campaign, the conversion rate for the Treatment and Placebo groups will be the same. As ads deliver to the Treatment group, the conversion rate should rise and diverge from the Placebo group. When the campaign ends, that effect will linger for some period of time before the two conversion rates converge again. The period of time over which the Treatment group continues to convert at higher rates, even after the advertising has stopped, indicates the residual effect for advertising. This time window provides guidance for MTA data collection and view-based conversion lookback windows.

What is the role of Multi-touch Attribution when people-based Lift Tests are available?

Multi-touch attribution has several key benefits and a few limitations relative to people-based lift tests. First, MTA is an always-on solution that can be applied to any campaign, even retrospectively. Lift tests need to be constructed in advance holding out a sufficiently large sample to ensure statistical significance. Lift tests require you not to reach 100% of your target audience. If you’re a product manager who’s very close to meeting your quarterly sales goals, you may not want to serve a placebo ad to 10% of a highly valuable audience. The quarterly earnings of your company may rely on meeting your sales goals and marketing research is simply too much of a luxury. Placebo tests can also be expensive! If 10% of your ads are serving Placebos, then 10% of your marketing budget is intentionally going toward ads that don’t help you sell product.

Multi-touch attribution, by comparison, analyses the correlation between ad exposure and conversion parceling out the effect of advertising to the various touch points. MTA models can analyze all publishers, campaigns, ad formats, devices, and frequencies simultaneously, using statistical techniques to tease out their contributions. Note that this is merely correlation, not causation. If the MTA model omits a key variable that is correlated with conversion, then other variables that are correlated with that omitted factor will pick up some of that effect and be incorrectly estimated. The results may still be useful and directionally correct, but there’s a reason the FDA chooses Treatment-Placebo tests when life and death are at stake.

The ad-effectiveness toolkit is growing.

In summary, the tools available to measure marketing effectiveness are growing more powerful. Atlas strives to enable people-based Causal Lift Tests across the web, utilizing Facebook identity without revealing the identities of individuals to maintain clear separation between the treatment and placebo groups. Multi-touch attribution enables a cost-effective, always-on solution to tease apart your various marketing efforts and understand their contributions. Atlas recommends a combined approach. Implement MTA consistently punctuated by periodic Treatment-Placebo tests as validation of your models findings.