Datran Media’s Scott Knoll Interviews Experts from Carat, Mullen, Rapp and Microsoft -

Audience measurement is a nebulous term for many market stakeholders – even for those who use reporting solutions to identify the audiences reached across multiple media channels. For years now, organizations (including the American Association of Advertising Agencies (4As), the Advertising Research Foundation (ARF), the Interactive Advertising Bureau (IAB), and the Association of National Advertisers (ANA), as well as advertising agencies and brands) have been discussing audience measurement, advancing standards, touting methodologies and offering views on how to roll in fragmented audience measurement approaches and toolsets into a total audience measurement framework. It’s a useful discourse with a big pot of gold at the journey’s end. But, to date, we still lack common understandings.

Since digital media measurement is a subset of total audience measurement, potential buyers struggle to understand exactly what it is, let alone how to gain value from it.  Consider the following scenario:  Recently, the 4A’s published, “A Marketer’s Guide to Understanding the Economics of Digital Compared to Traditional Advertising and Media Services.”

The guide is being applauded and critiqued at the same time. It wins applause for its who’s who of industry’s contributors and its valiant effort to compare digital and traditional advertising media economics, something desperately needed.  It earns critiques for a number of invalid claims and flawed definitions – something common to many first versions.

Case in point – despite the guide’s long appendix and various definitions of types of audience measurement, page 76 of the guide defines the term viewer as, “person viewing content or ads on the Web. There is currently no way to measure viewers.” At this stage of the game, if a published guide produced by the 4As claims Web viewers cannot be measured, why should we expect agencies and brands to understand digital media measurement, how to do it or its value?

Recently, Scott Knoll, senior vice president of display for Datran Media, hosted a panel on digital media measurement in which four industry heavyweights contributed their expert points of view on the topic in the context of total audience measurement and their own roles.  Scott’s questions were informed by industry wide dialogs around audience measurement, digital media measurement and direct agency and brand clients’ questions he fields in his role overseeing Datran Media’s Aperture digital media measurement tool. Aperture is a first-of-its kind technology solution that makes it possible to measure digital media audiences at a verified household level.  

During the panel, Scott St. Mary, VP of CRM & marketing services for Carat, Julie Coulton, senior vice president and director of digital media for Mullen, Andrew Goldman, vice president and group planning director/solutions lead for Rapp, and Ian Thomas, director of yield BI product management for Microsoft, addressed a series of questions focused on their points of views on digital media measurement and how it corresponds to their current roles. The transcript of the panel follows.   Please feel free to comment and suggest other questions for these experts.

Scott Knoll: What is your definition of audience measurement, and how does it correspond to your definition of digital media measurement?

Julie Coulton- Simply stated audience measurement is the process of monitoring consumer behavior across all touch points, throughout their daily routines, across the Web.  Whereas digital media measurement is the process of tracking media placement to conversion, establishing an ROI, digital media measurement is a closed loop process and is just one piece of the overall measurement plan.  As the digital landscape continues to evolve and becomes a more open forum, the ability for brands to measure audience engagement and behavior will ultimately help companies embrace the potential of the online space.

Given the explosion of social media, we are seeing a need for measurement technologies/tools to converge enabling brands to track activity from their own Web site, digital media, search and consumer conversations happening on blogs, multimedia sites, social networks and microblogs in an effort to complete the overall measurement picture.   This idea of open source reporting provides a spring pad for integration of solutions to help provide companies with the information needed about their entire online presence so they can effectively engage and connect with their consumer.

Andrew Goldman  - Audience measurement, to me, encompasses not only your media measurement, but also the attitudinal, social behavior, psychographic, transactional history, and all the other factors relative to the consumer’s brand experience. Digital media measurement can be a significant portion of that assessment, but is channel-focused, whereas audience measurement is consumer-centric. I see the former almost as a sub-set of the latter when considering a full customer view integrate marketing purposes.

Ian Thomas - To me, audience measurement is the practice of understanding who your audience is, and how they’re behaving – how they’re responding to online marketing campaigns, and how they’re behaving once they actually visit your site. It’s a broader and more effective way of looking at the effectiveness of online marketing, because it provides a consumer-oriented view rather than a media oriented view.

If you’re truly doing audience measurement, you have to consider the full range of media that the audience is exposed to, and how they behave before, during and after the click. Furthermore, as it becomes easier to buy media on an audience basis, understanding the nature of that audience and its ideal composition becomes ever more important.

Scott Knoll: What are you doing differently today than you were doing 1-2 years ago in terms of digital media measurement?

Julie Coulton - Digital media measurement has become a much more holistic practice geared more towards measuring multi-channel media plans than last-click measurement methodologies.  Given the demand for multi-channel measurement, our reporting approach has to encompass the integration of different reporting technologies in an attempt to tell a unified story. 

As digital media plans become more complex (i.e.  search, OLA, video, social media, mobile, etc…),  our analytics teams are testing ways in which to integrate data and develop scoring models establishing value across channels.  Another key difference is the way in which we report data. Currently, we are deploying more descriptive digital dashboards that compare channels and engagement to establish value and ROI in easy to follow graphs and charts that speak directly to campaign objectives.

Scott St. Mary - There are three primary differences in how we’re measuring digital media today vs. several years ago.  First, practically all of our clients are looking to better understand cross-channel performance through detailed attribution reporting vs. single channel, last-click reporting.  Doing so has led us to invest in technology solutions for both data aggregation as well as data visualization and manipulation tools – all of which has led us to a new level of analysis and optimization.

The second area of change we’ve seen is clients looking to better understand to what extent the resulting response behaviors are truly incremental vs. organic or potentially driven by offline media.  We’ve approach this primarily through “view-through validity” testing and baseline response analysis.

Finally, more and more clients are looking to understand exactly who is it that is both engaging with their digital media, visiting their online properties and transacting.  Clients want to know who these audiences are and why some are more responsive and valuable than others.  We’ve experimented with a research tool that can help us begin to understand these consumer segments, their differentiated needs and predicted outcomes.

Ian Thomas - We’re fielding a lot more requests for more sophisticated campaign attribution from clients. They understand that simple last-click attribution is too blunt an instrument for them to make decisions about media strategy, but they don’t really have an idea of what model they should be using. Should they analyze digital in a vacuum, or attempt to factor in the nebulous and hard-to measure impact of offline? And what relative weight should they give to, say, search vs. display in online?

What’s the relative value of a display impression and a search click, for example? Clients are looking to us to provide the answers to these kinds of questions, but the insight we have here is that the answer is different for every client – even every campaign a client runs.

Andy Goldman  - Today, measurement and analysis (M&A) plans are no longer channel-defined and campaign-focused, but rather are tied to more holistic business objectives, like lifetime value, operational investment return, and the like. The data we can capture to drive measurement and analysis and the rigor put behind it is now a starting point for digital media measurement, rather than an afterthought of the media plan.

Perhaps afterthought is strong language, but certainly M&A plans were behaviorally-driven (links, referrals, visits, opens, etc.). Today we are bringing in data from enterprise-wide business units to enrich our understanding of customer brand experiences online, such as customer service complaints, time on file, channel-engagement comparisons, and more.

Scott Knoll: What would you like to be doing in the next 1-2 years?

Julie Coulton - Defining the value of digital media beyond a CPA metric.  To have digital media truly be regarded as a leading awareness channel which offers a rich experience that can be compared to TV.  Because the perception of the digital media space is accountability I think it will be important to establish an integrated measurement processes which brings together sentiment metrics of the social media space, Web site data, OLA data and search data which then could account for awareness values across all digital channels that drive value to the bottom line for advertisers.

For example is having 100,000+ fans valuable to a brand?  How can brands effectively measure the value of social dialog in awareness/sales?  As the media landscape becomes more digital, how do we truly bridge the gap between TV and online video and the value of the experience on relative screens? Can this idea of dual screen viewing ever become one?

Scott St. Mary - The biggest leap I’d like to see digital media make in the next few years is arriving at a point where we’re able to define brand metrics that roll up to a “brand engagement index.”  This would allow a marketer to aggregate a consumer’s total exposure and engagement across all display, search, social and video assets.  Through complete cross channel analysis we’d be able to define at the cookie level a rolling (gaining as well as decaying) brand engagement index.

This index could be correlated to traditional brand metrics focused on brand perception, likelihood to purchase and an approximation of revenue value.  Doing so would help us to begin understanding the value potential of not only investment within channels but development of supporting content and points of engagement.

Andy Goldman - The push to integrate with both syndicated and proprietary customer data for better targeting is something we discuss with every client. Most of our clients are looking to reach audiences in the hectic, white-noise “Consumerscape” of today. Personalization and basic demographic relevance are no longer enough. Today we are looking at social media activity, Facebook and MySpace profile data (friends list size, music prefs, etc.), and viral influencer potential, to name a few, to arrive at data points that are more than behavioral – they are contextual, social, and emotional indicators.

These are helping for better targeting direct communications and more value-based messaging. This includes going after “earned” or “unpaid” media that comes from buzz and viral exposure. Much more advanced than a few years ago and extremely necessary given today’s challenging environment.

Ian Thomas - We would like to be working with each client to develop their own attribution model – or, in the case of our agency partners, working with those agencies so that they can build sophisticated attribution/engagement models for their clients. As the actual buying of advertising media becomes more commoditized, our agency partners are looking to drive extra value for their clients, and building an engagement model for those clients  both adds value and strengthens the relationship between agency and client. Factors like long-term brand attitude and social media exposure can be rolled into such a model, repaying a long-term relationship between the agency and advertiser.

However, it should be noted that an engagement model is not a panacea – such a model is only as good as the results it delivers. And there’s a real danger of analysis paralysis as clients spend more and more time tweaking their models  rather than trying stuff, seeing if it works, and adjusting if it doesn’t, which lies at the heart of good marketing in these fast-changing times.

Scott Knoll: What are your clients asking you to measure today?

Julie Coulton - Our clients are asking us to constantly prove the value of engagement. I am fortunate to work with clients that have digital as part of the overall media mix, and my team is constantly being asked the value of online engagement vs. offline engagement.  For example, what is the engagement value of a .30 second spot vs. a pre-roll?

Another hot topic for our clients is accountable measurement in the social media space and being able to model a value for social conversations and then integrating these metrics into an actionable social media strategy that makes sense for their business.  Of course we are also being asked to quantify media attribution across digital channels and the role of one digital channel vs. another, and how that effects media planning and ROI.

Finally, another key request from our clients isn’t in what is being measured, but how it is being reported back to our clients.  We have found that producing personalized digital dashboards that are based on key business initiatives makes for actionable data that is used across departments for our clients.  

Scott Knoll: We measure traditional media through the audience reached. Does it make sense to apply this same measurement goal to the Web? Especially for channels that help shape a consumer’s buying habits but don’t necessarily lead to immediate purchases (e.g. display, video or social media) or for products that aren’t typically bought online (e.g. toothpaste or fast food)?

Julie Coulton - Given that online media is considered a mass medium, I feel  it is essential that we measure it as a reach medium rather than just a direct response channel.  Many PG brands have leveraged the Web as effective as they would branding on TV, and it can be said the Web is the new :30 second spot.  Given the penetration of video as well as sites that aim towards replicating the TV viewing experience, like Hulu and Google/YouTube, I believe it is only matter of time that consumers no longer differentiate between either screen; thus the digital space needs to be willing to adapt to more traditional types of measurement, and advertisers need to be comfortable relying on those established metrics as well.

Andy Goldman - I don’t think it’s just about the mass anymore. The ability to co-create, publish, shape and share brand experiences has created new engagement metrics that need to be considered when thinking about emotional brand experiences. Particularly, I’d say, as we are in a tough economy where the reasons to buy are much more social, communal, comparison-driven, and emotional than ever before. The impact on offline behavior of online experiences ideally needs to be considered from a vantage point far more rich with consumer-centric motives than media channel response rates and eyeballs.

Ian Thomas - Reach is still a very valuable measure in online, but it needs to be calculated in a much more refined way than it is offline – and even than it currently is for online. For example: If a Web site publicizes that is has a certain demo, such as 40% female users, then you would assume that by running a campaign on that site, 40% of your audience will be female. But many Web sites’ audience constitution varies widely across the site itself, and with other factors, such as time of day or geography. So there’s a wide variance between what you might think you’re getting, and what you’re actually getting. Contrast with TV, where for a given 30-second spot you can be pretty confident of reaching something close to the network’s advertised demo makeup.

As an advertising network, we’re also very concerned with reach in the sense of being able to offer some access to as close to 100% of the Internet population as we can, partly because we will then be able to compete for more of an advertiser’s spend, but also because it improves our ability to make concrete predictions about the nature of the audience we’re offering to an advertiser. We’re no longer offering a pseudo-random selection of the total Internet audience if we have some access to the entire audience.

Scott Knoll: If not, what metrics and tools does the industry need to develop to bring more branding advertisement on the Web?

Andy Goldman - What do we mean by “branding?” Is it really about impressions, exposure and recall anymore, or do we need to begin to redefine the components of brand measurement. If we say “yes,” then what are the leading indicators of brand measurement that should be added to evolve how we measure brand advertising and find ways and places to do more of it? These could include site-behavior/marketing objective customer sentiment, voice of the customer, social engagement (as discussed in my POV for #1), and more. The benefit? More consumer-centric KPIs around messaging and exposure.

Scott Knoll:  How do you make cross-channel (digital) buying allocation or optimization decisions?

Scott St. Mary - Data aggregation across search, display and our clients’ transactional data enables us to analyze cookie level date that demonstrates how customers migrate through the conversion funnel and deliver differentiated value outcomes.  It uncovers the placements, keywords and Web hits that are playing an important supporting role to the resulting conversion.  We’re finding that this has actually reduced the frequency in which we’re optimizing after the initial “gross adjustments” are made as we move from last-click to detailed attribution modeling.

Ian Thomas - I feel that clients should treat a multi-channel campaign optimization task as an ongoing experiment. As with any experiment, the trick is to hold as many variables constant as you can whilst varying some factors, and then using the results of that change to feed back into the model. This may seem difficult with online marketing, which is often a series of dicrete campaigns, often with differing objectives and parameters. But techniques borrowed from the world of Multi-variate testing (MVT) can be applied to isolate the most influential factors in campaign choice, especially if you are prepared to take the long view and conduct this kind of optimization over a long period of months, or even years.

Scott Knoll: In your opinion, what are we likely to see in the future in this area? (cross offline and digital channel measurement)?

Scott St. Mary - Tools exist today that correlate offline channel spend and digital search and display engagement.  For example, within a certain level of confidence we can come to understand that certain key word groups are say 8% more active when there are 100 GRP of national TV running.

While that helps marketers begin to make some decisions regarding the overall efficiency and effectiveness of the buy, it doesn’t help the brand marketer understand to what extent it’s reaching a completely unduplicated audience.  We’re still unable to define to what extent our online efforts truly extend the reach and frequency of brand exposure and engagement.  While complete digital convergence is some time away, I believe we’ll begin to see more and more methodologies for understanding the true online and offline behaviors of individuals.

Andy Goldman - We’re doing some really interesting Pro Forma analytics work for some clients. These are historical looks at multichannel responses all throughout the customer journey. They are tied to budget and engagement so that if you move money around, you can see where greater efficiencies may exist.

They cover paid, owned, and earned media and while frought with assumptions, are a great way to see the directional impact of budget allocation and prioritize media spend. We’re also playing with database centralization capabilities, which allow for econometric modeling and tier migration strategies so that low-hanging fruit for direct conversions can be graduated into a high-priority conversion tier.

Ian Thomas - We will see the offline and online worlds converge on a number of fronts. For example, the continued digitization of TV advertising will render this an increasingly targeted and accountable medium, with advertisers able to execute “video” campaigns that span the lean-back (10-foot) and lean-forward (2-foot) environments.

To make this scenario work, however, advertisers will demand that they can de-duplicate the audience across those two environments, which will require significant co-operation traditional TV cable companies and their Web counterparts (so you can measure brand exposure across Family Guy on Fox, and on Hulu, for example). At the DR end of the equation, advertisers will continue to look for creative ways to identify consumers offline who have consumed online advertising, through down-to-earth tactics like coupons, and through online ‘proxies’ like social networks and blogs.