The “Single Source of Truth" myth
Written by Chris Schimkat – Reprise Regional Analytics Director, Albany Woo – Initiative Regional Analytics Director and Ben Tuff – UM Chief Product Officer APAC
For analytics professionals, conversations about seeking a “single source of truth” are not uncommon. Even within popular industry literature, the concept of a single, consolidated view of all media and web analytics is touted as the holy grail of analytics and reporting. However, like the holy grail, many have sought it without success.
The Truth Leaves Room for Interpretation, Even in Statistics
We’ve all seen and heard examples of how the truth can be interpreted different ways. From political rhetoric, to the news, scientific debate, and legal disputes, there is rarely absolute consensus on anything. When it comes to our marketing statistics, we often believe that this couldn’t be the case because we’re looking at cold, hard numbers – how could there be room for interpretation?
The above cartoon presents a visual representation of how our marketing data can be interpreted in differing ways. No, it’s not because one person is reading the report upside-down. It comes from the many platforms and methodologies that we use for measurement and reporting.
Media vs Web Analytics
While the web team may see Google Analytics as their source of information, the marketing team may look to Campaign Manager. Even when the two platforms are tagged with identical tracking points and use the same attribution model, there will still be “discrepancies”. The source of the discrepancy often doesn’t come from something “wrong” or “broken” in the way the data is collected or reported, but rather from differences in the scope of what each platform covers. Below are some differences in how each platform works, that can cause differing statistics:
As you can see in this example, there will always be a trade-off if you’re looking to choose just one platform to inform your decisions. Although Campaign Manager has an edge on Google Analytics by capturing impressions and not just clicks, Google Analytics is more effective at measuring organic traffic and on-site behaviour. This example doesn’t even consider the “walled gardens” of other platforms.
Platform vs Analytics
In another common example, you will encounter different statistics reported between advertising platforms as compared to web analytics or campaign manager. For example, you might see higher conversions reported from Facebook as compared to what Google Analytics says Facebook has contributed. A common reaction here is “Facebook must be taking credit for more conversions to make their platform look better than it is”.
When in reality, this is once again a matter of perspective. Specifically, Facebook cannot see or take into account the contribution of other channels towards conversion. If a user clicks a Facebook ad and then converts tomorrow after clicking through from organic search, Facebook does not have visibility over the organic click.
Further, the conversion attribution window may be set to 1 day view, 7 day click, so it has been configured to take the credit for this conversion up to 7 days after the click occurred.
Does this make Facebook data inaccurate? No. Should we just abandon and disregard any data coming from Facebook? No. But the key is to use it for the right purposes. The data Facebook collects, and reports is the best source for optimising Facebook activity.
Above the Line vs Digital Media
The final example that we will address is a common tension between above the line media and digital. Where digital media has some ability to attribute impressions, clicks, and conversions on an individual user-level, the same could not be said for investment in TV, out of home, sponsorships, or radio. While there may be a tendency to only attribute conversions to sources where there is a complete “chain of custody” from impression, and clicks through to conversion, we cannot disregard the contribution of above the line activity.
Long term success relies on improvements in both long-term (brand) and short-term sales activity. To understand how offline and online activity both contribute to online and offline conversions, we use market mix modelling.
Done at the channel level, market mix modelling (MMM) is used to estimate the impact of previous media spend, marketing tactics and other influences on sales or other hard business KPIs. MMM uses statistical techniques to isolate the relationship between media activity and sales. This allows us to pull apart and quantify the impact of media versus other influences (e.g. seasonality, promotions. etc). The insights delivered from analysis identifies the sales return from each media channel for each $1 spent and the potential for each channel to further drive sales in the future if more money were to be put behind that channel. This approach can also be used to quantify halo effects across a portfolio or identify optimum media budgets across markets.
Modelled Transaction Contribution
Using Multiple Perspectives
So, with multiple approaches to understanding marketing performance, how do we reconcile them all? All platforms will present different numbers, but the key is to accept and understand the strengths of each platform and apply it to the right purpose.Using Multiple Perspectives
Just as multi-focal glasses allow you view objects that are close or far with different lenses, we can apply the same thinking to our multiple analytics perspectives. To guide teams and clients towards the best approach, we developed a framework called the Streams Hierarchy (as pictured below).
It proposes that when optimising or analysing at the platform (stream) level, to look at data from the platform. Similarly, when assessing social media, you should look at combined social media dashboarding to understand how the platforms work together at a channel level.
Then, we get into sources that have a broader view of our activity. In the lakes level, we’re able to see all of our digital activity through a web (Google Analytics) or media (Campaign Manager) lens.
Finally, to understand the impact of the complete marketing mix and external factors, we can use full funnel or market mix modelling. This is used for high level strategic buying and planning decisions.
To illustrate why this is so important; you won’t be able to optimise paid search bids towards a market mix model, but your Google Analytics data also won’t help you to understand how the entire marketing mix contributes to conversion. We’ve put together a table to help with understanding which methodologies and data sources to use for each application.
Communication is key
Ultimately, requests for a single source of truth are not a technical problem at all. It’s a more of a communication and education task. Educating stakeholders on the benefits of moving from the pursuit of a single-source-of-truth to a multi-perspective approach will see better performance at the platform, channel, and over-arching level. It also means that stakeholders at all levels and in different business functions have the data they need to make the right decisions. In an ideal world, all perspectives are brought together and reported in a single place to allow for easy comparison.