Attribution in Digital Advertising: Insights from Nate Silver’s Political Calculus

Nate Silver and I are kindred spirits in a number of ways.  We both started playing poker professionally during the poker boom of 2003, when ESPN began broadcasting the World Series of Poker during primetime.   Silver is the man behind Baseball Prospectus’ PECOTA projection system, which predicted that Dustin Pedroia would become an all-star, even when all of the scouts wrote him off because of his stature.  PECOTA correctly picked that the Tampa Bay Devil Rays would win the World Series in 2008. We are also both political horse race junkies, and Silver’s FiveThirtyEight blog, which uses a linear regression model to predict the outcome of Presidential and Senate races, has been a daily visit of mine for years now.  His model correctly picked 49/50 states in the 2008 Presidential Election (and 35/35 Senate races), and got all 50 correct this past November (even Florida).
Silver’s success across multiple disciplines showcases his ability to differentiate the signal from the noise, and within his FiveThirtyEight model are many lessons on how to properly utilize big data.  Let’s take a deeper look at couple of these insights and how to apply them to an intelligent digital advertising campaign:
Recency: While I was surfing on Facebook last week, I was clicking on a friend’s link, when, out of the corner of my eye, I noticed an ad for the upcoming They Might Be Giants concert at The Blue Note.  Having already navigated from the page, I didn’t have an opportunity to click on the ad, so I searched Google, was taken to the purchase page, and bought a $20 ticket for the concert.  If I was managing The Blue Note’s ad campaign, I would want to assign a value to both the Facebook ad that originally caught my attention (even though I never clicked on it), as well as the SEO work that brought The Blue Note’s website to the top of the SERP, search engine results page.
An accurate attribution model should give the Facebook ad the majority of the credit for leading to the sale in this instance; conversely, it shouldn’t give nearly as much credit to a different ad served to me weeks ago.  Silver’s FiveThirtyEight model accounts for this recency issue by giving more recent polls higher weight than older ones by using an exponential decay formula to discount old data points.  The majority of online advertisers, however, don’t do this; they use a “last-click” model, which would attribute 100% of the $20 conversion value to Google instead of Facebook.  This is one example of a lingering statistical bias in many digital advertising campaigns.
Skepticism of Data: While this may seem counter-intuitive, in the age of big data, digital advertisers may get more mileage out of the data sets they throw out more than those they choose to keep.  For the FiveThirtyEight model, most of the raw data comes in the form of polls, and Silver chooses not to consider data from pollsters with a history of questionable methodology, as well as internal partisan polls conducted on behalf of candidates and campaign committees (campaigns tend to only release polling that is favorable to their client, thus introducing cognitive bias).  Moreover, the pollsters that are included are ranked by their historical accuracy, along with the sample size of the individual poll in question, and given an appropriate weight in the model.
As of right now, digital marketers are limited in the ways they can verify much of the 3rd party demographic data they use to serve ads to target audiences.  Soon though, comScore will have access to a trove of Facebook’s data (which has incredible reach and is highly accurate by internet standards), and can then begin to assess the accuracy of data aggregators like BlueKai, Exelate and Bizo.  Some preliminary evidence suggests that these data aggregators may not be that accurate; multiple people may use the same family desktop, for instance, making it difficult to identify the age and gender of a specific IP address.  Tracking capabilities on mobile devices is not particularly accurate either, as many devices do not accept cookies.  All in all, 3rd party data is a noisier statistical environment than most advertisers realize; I personally put a lot more faith in 1st party data, like that collected by Google Analytics.
Multi-variant Linear Regression: The biggest overlap between the FiveThirtyEight model and an attribution model for advertising is its use of multiple data channels.  Silver’s model doesn’t just use the aforementioned weighted polling averages, but also the Partisan Voting Index, the demographic composition of the electorate, incumbency status, and several economic factors, before determining the snapshot of a race.
An accurate attribution model does the same thing; it takes data from display advertising, pay-per-click, organic results, social media channels, pre-roll video, mobile and offline advertising, and weighs the contribution of each placement along the way.  Admittedly, measuring each of these data points is inherently difficult, time-consuming and expensive to execute.  Furthermore, data from mobile and social channels are not readily available to advertisers, adding further complications.  Nonetheless, this type of analysis should be the goal of savvy internet marketers.
The Role of Humans and Computers: One important thing to note regarding attribution modeling is just how crucial the human element is to its accuracy and success.  Data without context is completely useless, and as more and more data is aggregated, the more likely it can be used in biased, unsound ways.  It is an advertiser’s responsibility to give context to data points, to accurately account for unknown factors, to acknowledge the margin of error, and to add qualitative analysis to quantitative results.
In Nate Silver’s book The Signal and the Noise, he discusses how a forecaster at the National Weather Service produce forecasts that are 15% more accurate than what the computer models tell her, by adding the knowledge of her own experience to make small changes to the computer forecasts.  Likewise, a savvy, experienced marketer adds value to his campaigns by viewing big data through the right lens to accurately inform his analysis.