FOE: Changing Audiences, Changing Methodologies

Session 2: Changing Audiences, Changing Methodologies

Audience Research has long been a vital part of the media industries: research helps determine which ideas get produced, where content is distributed, and how content is monetized. Transmedia storytelling has forced media researchers to re-evaluate their notions of the audience since transmedia, by definition, allows audiences to engage at different levels across platforms. Research must now determine how to value audiences across different sites of engagement as they participate in different ways.

This panel will explore how research practices have adjusted to new ways of gauging audiences and making that knowledge useful. How does research understand and predict audience behavior? How does research contribute to monetization models for transmedia properties? How has traditional research adapted to keep up with the demand for better metrics? This panel will draw from a variety of industrial and academic perspectives to understand how we imagine media audiences and how we make them valuable.

Moderator: Eleanor Baird – Director, Partnerships & Analytics, Tube Mogul; Panelists include: JuYoung Lee – Co-Founder & Chief Scientist, ACE Metrix; David Spitz – Director of Business Development, WPP; Trapper Markelz – VP Products, GamerDNA; Joel Rubinson – Chief Research Officer, The ARF; Jack Wakshlag – Chief Research Officer, Turner Broadcasting System, Inc.

EB: Covering the role of research in TS.   In the last panel we were talking about measuring success. Research can help us assess impact; what they do as a result of engagement. Does it change behaviour, encourage spreading? Also what are the goals of TS. Research helps us figure out our goals and if we have met them

• EB: What does it mean to you and some examples:
• JYL: technology doe snot make TS. Effectiveness is driven from audience psychological context when using tech and that is what we focus on research. we look at same advertising across TV and online
• TM: in gaming, its the different types of experiences – mobile to tv, tracking content and artifacts across the platforms. we look at all the ways gamers interact with content and IP. follow them as they change context.
• JR: we don’t use the term. We use cross-media, or 360 media marketing. What it means is living in a 360 world, with human lives a the centre. The questions coming from marketers are quite different; what meshes with a brand strategy has got more complicated – but more interesting in a way. we;’re having more fun!
• JW: trying to create more opportunities to immerse themselves, when when and for as long as they want with content we can offer.  large scale and long term, , CNN TV vs CNN.com etc.  Some simultaneous or some asynch
• EB: so what role does research play?
• TM: the amount of research we can do is reliant on the data available.  Prior to the last gen of consoles, it was less, fewer people to generate data. We aggregate explicit interactions from data networks trying to merge with behavioural targeting types etc, browsing and watching videos, sharing, plus the network data on the playing.
• JR: It’s not a force in marketing that is happening in isolation. We are learning a lot about how the human mind works. Interested in behavioural economics – it’s not all rational for decisions. Have to look at what people are looking for and understand what is up for grabs. Engagement counts for a minority in customers; advertisers should be asking different questions, purposing touchpoints in different ways. each touchpoint does a different things I see digital and shopper marketing will merge – the mobile device is the main point.  Were not constructing these metrics nor have the right tests in place to do this. How does the ability TV create meaning and shopping create activation come together?
• JW: we can do almost anything if we have enough money,. the question is what should we spend our money on. We need metrics to assess size/scope of businesses, what we should stop etc. If we can’t measure it you don’t know how big/strong it is or what you can do it./ figuring out social media does not fly, you have to ask what you want to know – hoe many people, what they do. It is not the figuring out we have to focus on questions about business metrics. it can be behavioural or financial. don’t care if transmedia, uni or what. There are 3 things i need to know – how may, how often. how long. Need to know it about every medium I play in. Not all I need to know, but key for all channels.
• JYL: Our metrics are different from many/long/often.  But what we focused on the vast majority of audience that take no action after seeing a commercial. Advertisers need to understand effectiveness evenif no one takes actions there and then. We focus on developing ways to assess from behavioural data.
• EB: What do we have now or where are we falling short?
• JW: we have nothing except what people say. we don’t believe what people tell us (if we did, no-one watched the OJ trial). We need to find a way to go deeper. there are companies that are helping us explain non-concious decisions making, we can get info. Asking people what they want does not give you.
• TM: easy to measure when not, or don’t or actively choose not to.   We looked at Amazon, looking at what people actively did not vote. So the got presented options and watched what they flicked by.. It’s about trying to present choices, so they can actively chose to consume or not, to skip by.  Present as many options for choice and using implicit actions to discern the process
• JR: Media companies used to think of themselves in terms of the platform they are on. Now content is the organising principles. Thinking about it this way, is a media or publisher perspective and may not be the right way for an advertiser. Not sure impressions are additive. not sure you can add mobile, tv web impressions etc. So we need to consider measuring the holistic user experience, to calculate the multiplatform reach. or get accurate on each platform and put together as a secondary measure.
• JW: the average person on TV is 35hrs a week, average is 6hrs a week online.  Your world may be different, but that’s the world I live.  Online video is 3hrs month, about half is YT shortform, then Hulu. Neither of them have made a nickel in profit. We have to deal with ad impressions and time spent and count. Sometimes we have to live with the reality is 95% is TV, 4% is timeshifted. Those are the facts that I must grapple with.
• JYL: Online vs TV is different mind set. TV is passively engageed, so enough brainspace to accept advertising. Online they are more engageed, so not enough room to process advertising. Watching TV is driving with no distraction…online is like driving and texting…
• JR: from a branding POV…you need to have strong properties.  One of the ways MTV evaluated the different environments for The Hills (online, tv), they looked at brand perceptions. If they consumed in all environments, then highest engagement metrics. but is it that people are more engaged are are looking for the environs or are they engaged because the environments. But it does not matter. you have extended the brand experience in places that are new, it is up to you to assess whether they will add value and then build it. you should do it as entertainment or brand – eg Zappos. their culture is online via social media as well. It’s not just about media measurement it is also about what you want to be.
• JW: Cause and effect is really important. If building cross platform makes people more interested, but I’m convinced it’s the other way. More interested people look for cross platforms. We provide opportunities to immerse themselves as deep as they want. heavy users of CNN.com are heavy users of CNN. People are watching more TV than ever before.
• JYL: from an advertisers perspectives it s simple – can I use the same TV ad and put it online. Our online people have created something can we put it on tv. When they have something good they want to use it different platforms.
• TM: looking at the gamin space, none of the TS things in gaming, non have been very successful in driving engagement over and above the ones who would be engaged anyway. Metrics not there yet as data is not there, not mature itself. We need unified identity to help. few have made =the idea of TS to move more units. No-one has unlocked how to take a relevant message and target it by behaviour.
• JR: if you are creating a brand and deciding strategy, you have to decide which media will be the essence which matches the essence of the brand.  Some brands have specific channels. There are multiple plays in the playbook, one of the aspects in media measurement is a testing aspect. You may have alternate strategies, you need to test them. There are organisations that have an experimental design approach to assess effectiveness.
• EB: Looking to the future, if you could have metric/tech what would you want..is it worth measuring?
• JW: In my dream, I;’m not sure how to do it. I need lots of metrics..but the ones I need for tv and mobile, is many, often, long. I can’t get those three now.  Everyone Knows it but no one does it! We re decades away from answering the fundamental metrics of those 3 platforms
• TM: we are going to have to unify a lot of data silos.  Huge data warehousing problems. the data may be collected but consumers are uncertain about it being used. It is up to us to package the data as value add so it does not matter it being used. In gaming, data is useful as it is packaged back as profile etc, but does not work well in TV if they track everything you watch etc. If we can bring all the data together and make it portable then we may be able to do this.
• JYL: After the many/often/long, then advertisers want to know what kind of content can be used across the platforms. can I re-use stuff, can I use the 10sec from online on tv and still get ROI. most people do not take action, but most is not designed to generate immediate action,. have to be able to measure cumulative impact and then on the impact on behaviour. Standards is good, so they can compare against competitors and past behaviour.
• JR: if you have a multiplatform brand communication effort, and if it is important to you then you will probably commission a study. They are expensive discrete projects that are hard work to assess the impact. they are expensive, lots of time so limit usage. It is hard t build up a knowledge base. I would love to find a way that media becomes self-measuring, so it is generated as it goes. So the energy goes into not collecting the response but in analysing the response. In a digital space it is much more imaginable about how this works. It’s passive self-measuring is what I want.
• EB: How can the research side work with  the experience teams in a way that is constructive.
• JR: I got interested in engagement and dimensionalising it. I tried to calculate the % of sales from engaged customers. Most is more than half sales, but 10-15% of customers. You need to dimensionalise for the situation you are in,, the need to build the platform. For others it is important, for others it is OK to have a transactional view. you need to have a business perspective to create a proportional view.  For a storytelling perspectives, the story still needs to stand up on core platforms, or most won’t go and find the rest.
• JW: We are fortunate that the core business is growing, people are watching more TV. this allows us to do new and experimental things. We want to provide the options. Do we think the numbers will be large – no. Even of the numbers are not large, I can still be successful, the most engaged customers go, that is where the people who like the product go. You come to us and be prepared to try new stuff but accept that the numbers may be small. Not everything you try is a success. We learn form our failures, not our successes.
• JYL :Engagement is hard to figure out. Engagement with a programme is not the same as brand etc. Until there is answer to what is engagement, then advertisers have a problem with using that as a standardised metrics.
• TM :one of key successes in gaming is UGC, and how gamers self-organise in other channels. In MMORPG, guilds were not planned, but they happened. Now gamers are trying to get that to happen elsewhere. it is too new, there are not enough examples. the companies are having to change the models the games are sold, you can lower upfront cost and pick it up for new content. Or a free-to-play model with microtransactions. there is a shakeup and it is too early/
• EB (via backchannel). If you could quantify engagement, would it change business.
• JW: we fool ourselves that small niche projects make a model. I would generalise that big brands have more engagement and loyalty than small brands. Coke is coke…if you are trying to create small niche products that is not the way products get big. Successful bigger brands have a different approach.   the difference between TNT and FOX, it is not the time spent, it is the number, Far more people spend time with FOX, even if the time spent is very the same. Small brands suffer on two counts – few people choose them and not very often…law of double jeopardy (1963..missed name). There are exceptions,,eg religious, language – lower reach, longer time spans. Looking at transmedia opportunity, be realistic in how big it can be. It does not have to be big to work for us.
• JYL: shopping and consumption is a social activity, you want to buy things your friends have.
• JW: it is partly to do with brand awareness as well, you don’t choose brand you have never heard of. Even if equally good. (measures on perception and return). Even if 2 places are equally good, you tend to go to the one that more has heard of.
• JR: My problem is that whenever i try to measure it with respect to branding, it is a different brand equity measure, close to brand loyalty, so does not add much. In entertainment, there is a slightly different measure. the big win would be about finding away to apply behavioural economics to marketing problems. Understanding the heuristics that shoppers use to make split second decisions is a more fruitful area than new metrics.  Whereever you can analyse it, it exists. It happens in media, If you want to find light tv users, you look for them in the big shows, not the long tail.
• JW: of you don’t know how you want to measure it, then you will never know. If you ask them, they will never know, it is unconcious process.  Innerscope we have used…they will measure pupils, heartbeat, sweating etc. Tey are measures that make sense to me, not asking people.
• JYL: there are ways of asking questions, that do not rely on rational opinion. Innerscope are good for measuring reactions, but practically it is hard to attract people to that study…no-one likes to to be in a machine to watch a commercial. We are trying to look for alternates, they act as a proxy for physiological changes.  Fear of unknown is one reason you go for the larger brand.
• JR: I took over a marketing class and asked them to write their favourite brand. I asked her how they knew about a brand -no-one said tv. it was peer recommendations.  eg for a mac. But they all knew the Apple commercials by name. the influence of those commercials was not something that could be said.
• AudQ: how can you do AugReality/mobile in a store. One of the things a web does is give comments…so people can add their own impressions of products. Is there for marketing to get aroudn this? Will it push large bransds to get better
• JR: there will be brand teams that will be in denial and prevent it and the others will embrace it.  Will also embrace it as manufacturers fight with retailers. So apps allow manufacturers bring their message right into the store, bypassing the retailers. Eg an app to show IUPC code which then shows video about the organic growers, tell the story. you have to be able to embrace it. The smartest people are some of the smaller guys.. but also P&G and Unilever who are all over this stuff.
• AudQ: Do we need the big measurements or do we need to redefine it?
• JW: I have 140 people who comb through data and that is there job. Data turned into information and turn it into knowledge and that is their job. You want overriding rules of the road as you don’t need to test everything and ask data questions everyday. There’s tonnes of data out there, but we don’t have people who make sure it is really good, etc, I will not do business with any company that hides behind a black box. Quantcast is not someone I do business with as it is a black box. I need people who will take the data and analyse it to provide information., We don’t know who is online..a lot of people share their computers, erase cookies etc. We have data, we don’t know the demos, how much money etc, we know they visited and then someone used the computer to go elsewhere.
• TM: When we look at the data directly on the wire. There’s value but it seems really small.  franchise and genre loyal people, with TS experiences, you never get the huge initial audience, it is hard to get back to the main release. there is value created but not a lot.
• JW: researchexcellence.com has data you could use, funded by Neilson, they did a large scale multiplatform observational study.  It’s as big and as close as any study has come.
• JR: you can also mine social media, get insights into hat people think about you and your competitors, plus lifestyle issues etc, Collectively this is referred to as listening. This is about listening for the unexpected.   Comparing research questionnaires vs listening – value in both, but different insights. In research, the researcher controls terminology, in listening ,the user controls it.
• AudQ: About CNN…every network comes underfire. When networks were putting more focus on Letterman/Palin than Iran, then the tweets CNNFAIL appeared to be reacted to. Also Stewart reactions,. So how do you measure the metrics of complaint?
• JW: We monitor Twitter, Facebook, blogs, letters, emails. We respect the feedback from people who spend time to send us a message. We are an audience facing business, we have to respect the audience.  You can’t ignore the feedback but you have to weigh it. as CNN as a professional journalist whether you did a good job. We look at shows everyday and ask us the questions about journalism..dod we do the job we should have. Proud to say that CNN take the job seriously.
• AudQ: would like to respond to some of the statistics.  If you take the average audience, that is a problem. Add age demographics then it is different (JW: that is untrue) So how do you measure?
• JW: we gather data in a public way..I challenge you to show me data about what they DO.  We have looked at it in 3 ways and all show 15-24 watch lots of TV. People under age of 17 spend least time online as they have the least time. (Missed some as it got into a back and forth overlapping discussion)
• AudQ: Many/Long/Often is based on advertising.. But not a lot on why. If you can’t do the how questions is this an opportunity to come up with new measures. So why focus on the old measures if there are opps to talk about the why?
• JW: so I need to know what I want to know the why? It is not that I don’t need to know the why, when I’m doing the media plan, working out how to much to pay people to create, I need to know the How.  We have models on the how
• JR :you need to know both. but there is commercial purpose for all. I did a meta-analysis of ad campaigns. What you heard in echo chamber it said TV was no longer viable, but the analysis showed that is was as effective as ever. Media strategies and brand platforms need to be done together.    Dove’s Campaign for Real Beauty was one of the most innovative campaigns recently. It was all about the customer, not the change to the product.  TV is not everything…it is like a rubiks cube, you have to solve everything together. to do that you need to know they why but you have to quantify it to take it back to the commercial space
• TM: look to gaming space. we know how many and how long, so the why is being played with. games companies are finding that it is not successful way of monetising as it is the same small group.  Now it is the the change of audience that is happening, eg Eve, moving to a FPS platform, to go after a new audience.
• JYL: it also depends on product. One of the most successful campaigns…insect repellent, there is a belt clip version of this product (OFF) not a lot of engagement.  they put on a TV ad, they spent a little amount..sales 400% increase right after. Low interest, no brand loyalty…TV large audience has large impact.
• AudQ: Jack, you are concerned about what people DO rather than say they do. So how do you approach situations where metrics not really reliable…eg Time Magazine poll, Twitter trending topics. Metrics as what is goign on vs cultural attitudes.
• JW: is a big problem, we know people game the figures. Eg ad click farms. It is about data quality, and quality controls. It should in theory be large and stable but people can  game it, so not there.   It is a problem for us. you can criticise the Neilson system, how can they generalise from 18k to all the home, I wish they could do something better but you take the metrics you have.
• TM: Game data is very young but there are the same challenges. People who leave games on. they are anomolies, but not huge. eg gold farmers, It can be filtered out, normalised for outliers.
• JYL: extreme data is online, the vocal minority rules the space. you need 3rd party non-biased, not just the listened to stuff in tweets etc. these opinions are skewered and you need more.
• AudQ: What internal discussion about making the data publicly available for us to use?
• JW: It is not our data, we buy it from another company.  I’d love to know where Quantcast get the data from.
• JYL: as a data provider, we have a business to run. It is hard to make profit by providing data in public. We collect data as well as we can, then make it to other businesses. It is hard to give it away.
• JW: Has access to all TV data..come and talk to me about getting access as long as it serves a useful purpose.
• TM: in the same boat. We have provided lots of data to MIT Lab, there is a lot of data out there. we will work with anyone who will look at it.
• JR: so what analysis would you do?
• AudQ: what is more valuable, data or analysis?
• JR: or is it C….what is the data, so what is the analysis and the now what is the business decisions.
• JW: the data are useless, the information and knowledge is useful. I create the whys, the explanations., Humans observe the world and come up with explanations.We construct explanations for the things I see. My managers don’t want to see data everyday, they want information and knowledge, i need to search for empirical irregularities and come up with why. It is my job to find the secret sauce., If it was easy, I’d be rich! But it’s not and I’m wrong sometimes.
• JYL: yes, it’s the information,. Look at the large amount of financial data, but few people saw it coming
• JR: but it is not good enough, you have to take a stand and do something.  With the TV world changing, about DVR etc, the tv people had to make the call about what to do. It is not the analysis, but the choice to make a choice and be held accountable for it.  Everyone should have a now what moment with respect to their business. this is a cultural change, one of the aspects of being a researcher is doing great analysis. It is not good enough anymore, we need to find a way to drive the business forward.  We need to get the marketers past their fear of control.
• JW: you need to build a model with one set of data..and then test it with another!