SURVEY Magazine by Survey Magazine

B2B Sample Tradeoffs

The Power of a Multi-Mode Approach

How to

a Social Research Panel

SURVEY Authored by: Jackie Lorch, Vice President, Global Knowledge Management , SSI

DEVELOP

MANAGE

market research bulletin

Practical

Big

Data Jon Atkin

John Carroll III

Reasons Why

Big Researchers Should Lead in

Big Data MAY 2014

002 SURVEY MAGAZINE

MAY 2014

George Llorens

Gregg Lavin

Co-Founder, Innovate

Innovate was founded in 2014 by Gregg Lavin, Matt Dusig and George Llorens as a new approach to engaging users in global market research surveys. Innovate will solve a key problem in the market research industry. Around the world, engagement of survey participants has continued to decline in recent years, and new methods of keeping people interested in taking surveys need to be developed.

The Culture of Innovate >L»YL H IV\[PX\L ÄYT focused on delivering hard-to-reach, highly engaged survey respondents globally. We specialize in customizing creative solutions, conducting each aspect of a job with painstaking attention to detail. Above all, the culture of Innovate is personal, since it was founded on a tightly-knit relationship between loyal friends with common goals.

Matt Dusig Co-Founder, Innovate

Culture changes from context to context and can be as permeable as the people who inhabit it. It’s something that is created, sometimes intentionally, sometimes not. It isn’t KPMÄJ\S[ [V ZLL OV^ [OPZ [YHUZSH[LZ into a business environment- business culture revolves around the goals and standards of the people in charge, whether the company consists of three people or three thousand. Either a single person, or a small cohort of individuals must, at some point, establish these principles. This isn’t to say such I\ZPULZZ WYPUJPWSLZ HYL Ä_LK *VTpany culture is dynamic. It adapts and mutates along with people and technology. Most importantly, though, is the affect a culture has on those that work within it.

Within a given company, the question of whether or not an entrepreneur is born or made matters much less than whether or not they have an accommodating business ideology to work within. Do they have wiggle room for new ideas for better collaborations? A great entrepreneur (and a great company for that matter) strive to get as close as they possibly can to the exact needs of their target consumer. Ideally, a company’s culture sets up as little obstruction as possible between client and entrepreneur. When a company is overly rigid, or too ‘corporate’ in the traditional sense, there is less space for an entrepreneur to imagine, experiment, and create something really novel. This puts a great deal of responsibility on the business’ founders to create something that is both commerJPHSS` LMMLJ[P]L I\[ HSZV ILULÄJPHS MVY those that work with and/or beneath them. In other words, the culture of a business emerges from the individualized principles upon which it is founded. (SS [OPZ PZ [V ZH` [OH[ P[ PZ UV[ WYVÄ[ VY WYVK\J[Z [OH[ KLÄUL [OL Z\JJLZZ VM a business, but the culture it exists ^P[OPU *\S[\YL PUÅ\LUJLZ UV[ Q\Z[ the satisfaction of its employees, but the quality and individualization of what it does. Nowhere is this idea more valid than at Innovate, where we specialize in building bridges between clients and hard-to-reach consumers.

MAY 2014 SURVEY MAGAZINE

003

Reasons Why Big Researchers Should Lead in Big Data

Practical

Big Data Authors: John Carroll III and Jon Atkin

Page 16 004 SURVEY MAGAZINE

MAY 2014

MAY 2014 SURVEY MAGAZINE

005

SURVEYMAGAZINE

D E PA R T M E N T S

CONTENTS. 4 (@

B I G D ATA

Practical Big Data . . . . . . . . . . . . . . . . . . . . . . . . 1 6

Five Reasons Why Big Researchers Should Lead in Big Data

F E AT U R E S u r v e y To o l s & M e t h o d s

/V^ ;V +L]LSVW HUK 4HUHNL H :VJPHS 9LZLHYJO 7HULS . . . . . . . . . . . . . . . . . . . . . . . . . 3 0

Research Best Parctice

-\SS 7YVĂ&#x201E;SL *VUQVPU[ (UHS`ZPZ. . . . . . . . . . . . . . 2 4 Powerful and Realistic Tools for Marketing Decision Making

+PMMLYLU[ :JHSLZ +PMMLYLU[ 9LZ\S[Z . . . . . . 4 4 The impact of asking about income and the size of the house in three different ways

Survey Buzz

Hot research topics and news . . . . . . . . . . . . . 1 6

Survey Web Round-Up

Updates from our online properties . . . . . . . . 2 2

006 SURVEY MAGAZINE

MAY 2014

Panel Management

) ) :HTWSL ;YHKLVMMZ . . . . . . . . . . . . . . . . . . 5 5 The Power of a Multi-Mode Approach

Page (--,*;0=( c 7(9;5,9:/07 ;(2,: 8<(30;(;0=, -(*0(3 *6+05. .36)(3

SCompany URVEY Showcase

Research technology collaboration to support and distribute Affdex Discoveryâ&#x201E;˘ in virtually any location

MAY 2014 SURVEY MAGAZINE

007

S URVEY

MAGAZINE

WHAT THE EXPERTS ARE SAYING

“

A truly collaborative relationship will be far more iterative. All parties will be as equally able to absorb inbound knowledge as they fuel continued learning initiatives.

“

01 Bob Yazbeck

QRCA Featured Columnist

“

Capture the “in-the-moment” responses. When conducting an inhome product test, online and mobile research tools give the researcher the chance to be there with the participants.

“

Asking is an

“

The most empowering feature of social panels is the ability for panelists to engage with each other.

“

B2B

Marketing Manager Netquest

The way we ask a datum or an opinion to a person, determines the results we obtain.

Choice Modeling, one VM [OL ÄYZ[ Z[LWZ PU the design of a exercise is to determine the number of attributes and levels to be studied.

“Understand the source of the sample you’re using. How diverse are the sources, and is the relationship owned, allowing for enough quality controls?”

Everyone wants mobile research. Mobile access can put research in context like never before.” ]

Senior Methodologist Questback

05 Annie Pettit *OPLM 9LZLHYJO 6MÄJLY Peanut Labs

06 John Carroll, III Global Head of Clients IPSOS Loyalty

Karin A. Ferenz Presidnet Customer Life Cycle, LLC

08 Matt Dusig Co Founder Innovate

“

MAY 2014

Darren Bosik

008 SURVEY MAGAZINE

Carlos Ochoa

”

[ “Mobile. Mobile. Mobile. Sample Tradeoffs

President, CB Research Services

Or rather, a

Conjoint Analysis

Casey Bernard

“Just as in Discrete 04

ART science. 01

Vice President Gongos Research

09 Jackie Lorck Senior Vice President SSI

10 Jim Bryson CEO, Founder 20/20 Research

SSI Means Business.

Introducing SSI B2B inSSItes sample. TM

Reliable B2B sample, blue-chip sources, phone verification via SSI CATI, realistic estimates, accurate profiling and genuine business opinions from the people who matter to you. Letâ&#x20AC;&#x2122;s talk business. Call 1.855.477.4726

ÂŽ info@surveysampling.com surveysampling.com

SURVEY

essentials. WHAT TO KNOW ABOUT THIS MAY 2014 ISSUE

9 1VU 3LPTHU Editor, SURVEY Magazine

In this issue S U R V E Y EVENTS

webinar lineup

for a complete list of webinars, go to surveymagazine.org/webinars

Page

B2B

SamplePage 55

In this issue of SURVEY Magazine, we’re pleased to feature an article written by Authors: John Carroll III and Jon Atkin from IPSOS Loyalty. They are gracing the cover this month with the FEATURED story; Practical Big Data - Five Reasons Why Big Researchers Should Lead in Big Data starting on page 16. We are also pleased to share our second article in a three (3) part series by Karin Ferenz from Customer Lifecycle LLC. This, second piece is entitled “Full ProÄSL *VUQVPU[ (UHS`ZPZ Powerful and Realistic Tools for Marketing +LJPZPVU 4HRPUN¹ @V\ JHU ÄUK the start of this article on page 24 of the issue.

Darren is the Senior Research Methodologist for Questback, a ÄYT ZWLJPHSPaPUN PU ZVJPHS TLKPH and feedback management programs. On page 30 of this issue, Darren elaborates on the topic with an article entitled “How To Develop and Manage a Social Research Panel”. In this issue, we are also featuring a very thoughtful article written by Jackie Lorch from Survey Sampling International. She provides some great tips when conducting research with B2B research panels. We hope you enjoy this issue of SURVEY as much as we’ve enjoyed preparing it for you.

This past month SURVEY hosted a very enlightening webinar on Social Media Panels. Our guest speaker was Darren Bosik.

R. Jon Leiman, Editor

Tradeoffs

SURVEY

tip

PENNSYLVANIA PO BOX 899, EXTON PA 19341 TELEPNONE: +1 484.478.2774

www.SurveyMagazine.org info@SurveyMagazine.org

010 SURVEY MAGAZINE

MAY 2014

Look for SURVEY TIPS in our featured articles and briefs throughout the issue. Guaranteed to keep your surveys in tip-top shape.

SURVEY is a uniquely positioned publishing company that connects with over 110,000 business leaders around the world. We provide Customized Media Solutions for business-tobusiness (B2B) partners to enhance their content marketing strategies, messaging, and lead generation efforts globally.

In complexmedia mediaenvironment, environment, In todayâ&#x20AC;&#x2122;s todayâ&#x20AC;&#x2122;s complex do whereyour youraudience audienceis? is? do you you know know where Their devices? Their devices?

Their needs?

Their behavior?

Their Their mindsets? mindsets?

Theirscreens? screens? Their

Their attitudes? Their attitudes?

June8â&#x20AC;&#x201C;10 8â&#x20AC;&#x201C;10 June

NEW YORK , NY

2014 2014

Come to Audience Measurement 2014 Come to Audience Measurement 2014

Ĺ? Ĺ?Ä&#x2018;Ĺ? ÄĄ

Ĺ? Ĺ?Ä&#x2018;Ĺ? ÄĄ Ĺ? Ĺ?Ä&#x2018;Ĺ? Ĺ? Ĺ? Ĺ?Ä&#x2018;Ĺ? Ĺ?

Ĺ? Ĺ?Ä&#x2018;Ĺ? Ĺ? Ĺ? Ĺ?Ä&#x2018;Ĺ? Ĺ?

PLATINUM SPONSORS

PRESENTED BY

SURVEYMAGAZINE

CONTENTS. 4 (@

COLUMNS

B o b Ya z b e c k , G o n g o s R e s e a r c h From Insights to Intelligence: The Next Decade of MROCs ........................... 46

BOB Y AZ BECK

@_Gongos

Annie Pettit, Peanut Labs I hereby ban weighting ................................. 48 ; O L 3 P Z[ L U 3 H K `

A NNIE PETTIT

Casey Bernard, CB REsearch Uncover Deeper Insights with Hybrid Qualitative Approaches ................................. 47

Jim Bryson, 20/20 Research The Looming Smartphone Research Crisis: Meeting the Challenge .................................. 49

CA S E Y B E RNA RD

@LoveStats

@QRCA

JIM B RY SO N

@JimBrysonTN

NEWS Decipher Adds MSGâ&#x20AC;&#x2122;s PRO-T-S CATI Dialer Functionality to its Beacon Research Platform ........................................................................................................................ 68 [ DOWNLOAD ] Forrester Report: The Customer Experience Index, 2014. ......................................................... 68 Marketing Workshop Announces New Team Member Addition .......................................................................... 69 Critical Mix Strengthens Customer Care Team with New Hire, Brett McCready .................................................. 70 iModerate Enhances its Research Suite with Private Boards .............................................................................. 71 MetrixLab Renames MarketTools and CRM Metrix ............................................................................................. 71 012 SURVEY MAGAZINE

MAY 2014

“Intelligence: Not because you think you know everything, but rather because you question everything you think you know.” -‐‑Anonymous

Gongos is changing the conversation from market research to decision intelligence, challenging conventional wisdom to find what lies beyond.

INFORM. INNOVATE. INSPIRE.

Gongos Research | O2 Integrated

+1.248.239.2300 | gongos.com

T H E L A T E S T

SURVEY BUZZ NEWS AFFECTIVA-20|20 PARTNERSHIP TAKES QUALITATIVE FACIAL CODING GLOBAL

Practical

Big Data

Research technology collaboration to support and distribute Affdex Discovery™ in virtually any location

FACIAL CODING

Five Reasons Why Big Researchers Should Lead in Big Data

QUALITATIVE

NEXT PAGE >>

IBM White Paper and Infographic +6>36(+

Google Consumer Surveys

Easy answers to every business decision. Act now and get $75 off `V\Y ÄYZ[ Z\Y]L` www.MarketResearchBulletin.com 014 SURVEY MAGAZINE

MAY 2014

Practical

Big Data Five Reasons Why Big Researchers Should Lead in Big Data Authors: John Carroll III and Jon Atkin

Big Data is getting bigger. The Information Age is chugging on, creating vast swathes of rich data on all facets of private and business life. In its wake, the world is coming to terms with what this means in terms of potential and practicality. This new world offers so much opportunity, in theory.

016 SURVEY MAGAZINE

MAY 2014

MORE > MAY 2014 SURVEY MAGAZINE

017

Is Bigger Better? Big Researchers, and by that we mean world class modern market research professionals in every size and type of organization, are grappling with the issues and opportunities surrounding Big Data. With humans around the globe rapidly gaining access to the internet, using smart mobile devices, and generally tracking their every step, task and even emotion, the need to tackle Big Data is clearly growing quickly. The potential to marketers and business leaders is just too massive; if not completely clear.

018 SURVEY MAGAZINE

MAY 2014

At the same time, the concept of Big Data is likely to soon fall from the “Peak of 0UÅH[LK ,_WLJ[H[PVU¹ into the “Trough of Disillusionment” part of the well-worn Gartner hype cycle This disillusionment is grounded in several reasonable criticisms that Big Data is too unstructured to generate insight, too confusing to guide action, and too expensive to use practically. Big Data incorporates information from some sources which don’t Ä[ PU[V [OL [YHKP[PVUHS TVSK of high quality and acceptable research data. Big Data practices aren’t comprehensively addressed and guided by formidable market research institutions such as CASRO and ESOMAR. And of course, [OLYL HYL YLHS HUK ZPNUPÄJHU[ privacy concerns surrounding Big Data.

Practical Big Data:

A Near Term Solution for Big Researchers Big Researchers need to reject the idea that Big Data is rife with big and unsolvable problems. It’s time we move to “The Slope of Enlightenment” where researchers, business executives HUK NV]LYUTLU[ VMÄJPHSZ JHW[\YL MHY NYLH[LY ]HS\L from Big Data.

It’s all to do with practicality.

Big Researchers are (Big) Data experts.

Big Researchers break down organizational silos.

Big Researchers make integrating data easier.

Big Researchers leverage new technologies.

Big Researchers tell stories

MORE > MAY 2014 SURVEY MAGAZINE

019

Practical

Big Data Five Reasons Why Big Researchers Should Lead in Big Data

Practical

Big Data

1 Big Researchers are (Big) Data experts. There is no doubt that leaders thirst for data more than ever to make real time decisions. Big researchers are already amongst the experts in their organizations regarding all matters of collecting, analyzing, and presenting high volume, high variety, and high velocity data sets. The education, skills and experience required to design and redesign data capture systems, to conduct complex statistical modelling, and to present insights using new data visualization techniques is all housed in the PhD laden research department. Big Researchers need to remind their leaders and colleagues that it is the research department which should take the lead on Big Data; teaching people how to drink from [OL KH[H Ă&#x201E;YL OVZLZ UV^ ZV JVTmonly available.

020 SURVEY MAGAZINE

MAY 2014

5 Avoid Big Data Disillusionment: Big Researchers Take Center Stage Big Data is due now for a big backlash. Indeed, there are real complications which must be addressed to capture value from Big Data. No other functional leader is capable and ready to identify and solve the most pressing problems. Without leadership, Big Data will languish in the Trough of Disillusionment. SURVEY

tip

Fortunately, Big Researchers can step up now – it’s time to lead! For WKHVH ðYH ELJ UHDVRQV %LJ 5HVHDUFKHUV DUH EHVW SODFHG WR KDUQHVV Practical Big Data which is the key to addressing issues, unlocking potential, and delivering real value in this brave new Big Data world.

Big Researchers tell stories. Big Data isn’t just about the ability to do really interesting – and often extremely cool – analyses by integrating huge data sets. Big Data is about telling richer more insightful stories and making better recommendations that lead to emotional commitment, investment, and action. Big Researchers know how to combine the quantitative and the qualitative, the structured and the unstructured, the solicited and the unsolicited which are central to Big Data to create stories that stick.

2 Big Researchers break down organizational silos. In the knowledge economy, leaders and indeed all employees, customers and partners rely on access to high quality, rich data. Unfortunately getting different functional, product, and geographical silos to share data sets is easier conceived of than done. Hence the infamous Big Data comment from Behavior Economist Dan Ariely: “Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it…”Big Researchers are highly experienced at working at the center of complex organizations and acting as the central repository and broker of data thereby facilitating the creation of truly big sets of data. Big Researchers don’t just talk HIV\[ )PN +H[H [OL` KV P[

3 Big Researchers make integrating data easier. Going back a decade or so it would not have been uncommon for there to be more software platforms than departments in an organization. A major hurdle to making Big Data practical is combing all sorts of different sets of data which are not designed to be connected, analyzed and reported in an integrated way. Big Researchers know how to merge transactional, psychographic, geo-location, social media, and even biometric data such as pulse and respiration. These skills in data integration make Big Data practical.

4 Big Researchers leverage new technologies. Big Researchers are experienced, skilled, and comfortable with more data capture, analysis and presentation technologies than ever before. Big Researchers are already using cutting edge tools such as Google Glass and other “Wearables” to conduct deep technology enhanced ethnographic studies as well as broad passive data enabled quantitative studies. In terms of analysis, Big Researchers are diving into applying Hadoop and other technologies to allow the storage and large scale processing of truly big data sets. Training and investment in Big Data technologies is already core to the budgets and plans of Big Researchers.

END MAY 2014 SURVEY MAGAZINE

021

May 2014 / Web Roundup

SURVEYMAGAZINE

org

2,,7 <7 ;6 +(;, >0;/ ;/, 3(;,:; )9,(205. 5,>: (; ;/, +,:;05(;065 -69 :<9=,@ 9,:,(9*/ -964 ONLINE RESOURCES AND ANALYTIC SURVEY TOOLS TO QUALITATIVE RESEARCH BEST PRACTICES AND EXPERT INSIGHT, THE AUTHORITY FOR SURVEY RESEARCH IS AVAILABLE ANYTIME ON YOU COMPUTER, TABLET OR SMARTPHONE.

whatâ&#x20AC;&#x2122;s clicking

Some of our most popular online stories...

The Global happiness Report published in January 2014 continues to thrive on SUrveyMagazine.org as one of the most visited and shared articles this year. We all must be looking for more HAPPINESS.

Multi Tasking: The Modern Respondent. First published in our December 2013 issue of SURVEY, this article is one of the most popular full feature articles on the SurveyMagazine.org website.

Get Â SURVEY Â on Â your Â iPad Â and Â iPhone.

In the March issue of SURVEY, we honored the Top Technology Trendsetters for 2014. 4LSHUPL *V\Y[YPNO[ ^HZ JOVZLU HZ VUL VM [OL OVUVYLLZ /LY WYVÃ&#x201E;SL PU [OL 4HYJO issue was the most visited and most tweeted of the 20+ candidates submitted: www. SurveyMagazine.org

TWITTER: @SURVEYMAGAZINE

022 SURVEY MAGAZINE

MAY 2014

PINTEREST: PINTEREST/SURVEYMAGAZINE

FACEBOOK: FACEBOOK/SURVEYMAGAZINE

S U R V E Y EVENTS BRIEFS

webinar lineup

on the

road?

M AY 2 0 1 4 W E B R O U N D U P S U R V E Y M A G A Z I N E . O R G

for a complete list of webinars, go to surveymagazine.org/webinars

Hear it from the experts: Voice of the Customer

ON DEMAND

LIVE

Did you join us April 1st when we presented:

The True Meaning of Representivity

Developing a Social Online Research Panel A Free webinar, hosted by Survey Magazine with guest speaker, DARREN BOSIK, Chief Research Methodologist at QuestBack.

To Catch A Thief: Fighting Respondent Fraud

A Free webinar, hosted by Survey Magazine with guest speaker, PETE CAPE, Director, Global Knowledge at Survey Sampling International (SSI).

Combining the Power of Predictive Analytics and Survey Data Catherine Frye â&#x20AC;&#x201C; Business Analytics Solutions Manager, IBM Joseph Mahoney â&#x20AC;&#x201C; Client Technical Professional, Business Analytics, IBM

no worries...

Get the Real Pulse on your Customersâ&#x20AC;&#x2DC; Thinking with Predictive Analytics

Finding the right research partner for your next research project is just a tap away.

MarketResearchDirectory.org

weekly

research

The research industry struggles with three data quality challenges: poor questionnaire design, respondent inattention and fraud. We hear a good deal of industry talk VU [OL Ă&#x201E;YZ[ [^V" UV^ ::0 SPM[Z [OL SPK VU [OL [OPYK

news updates

A Free webinar, hosted by the IBM Business Analytics Team with guest speakers, Catherine Frye and Joseph Mahoney. Get the real pulse on your customersâ&#x20AC;&#x2122; thinking by Combining the Power of Predictive Analytics and Survey Data.

ON PINTEREST

ON T W I T T E R

follow us @surveymagazine Be sure to subscribe to our weekly e Newsletter for updates sent to your desktop, tablet or smartphone. Sign up at Market Research Bulletin.

Survey Magazine @SurveyMagazine May 22 AFFECTIVA-20|20 PARTNERSHIP TAKES QUALITATIVE FACIAL CODING GLOBAL http://marketresearchbulletin.com/affectiva2020-partnership-takes-qualitative-facial-coding-global/ â&#x20AC;Ś #mrx #survey #research @2020Research

Survey Magazine @SurveyMagazine May 16 [WEBINAR ] Voice of the Customer - Big Data as a Strategic Advantage: http://lnkd.in/dSqHz6N

Survey Magazine @SurveyMagazine May 12

Are your employees helping or hurting your customer experiences? #mrx #survey #research Click here to Download report http:// icont.ac/2n8b4

www.MarketResearchBulletin.com !

DIRECTORY: MARKETRESEARCHDIRECTORY.ORG

WEB: SURVEYMAGAZINE.ORG

NEWSLETTER: MARKETRESEARCHBULLETIN.COM

MAY 2014 SURVEY MAGAZINE

023

In a three part series

024 SURVEY MAGAZINE

MAY 2014

-\SS 7YVÄSL

Conjoint Analysis Powerful and Realistic Tools for Marketing Decision Making

Many marketing decisions require complex decision making processes which incorporate many data points from various sources. Depending on the type of decision you need to make, some statistical techniques are more useful than others. In this three-part series, we explore three different trade-off techniques frequently used for different marketing KLJPZPVUZ 0U V\Y ÄYZ[ HY[PJSL ^L L_WSVYLK +PZJYL[L *OVPJL 4VKLSPUN" PU [OPZ ZLJVUK HY[PJSL ^L KPZJ\ZZ -\SS 7YVÄSL *VUQVPU[ (UHS`ZPZ" HUK [OL [OPYK HUK ÄUHS WYLZLU[HPVU VM [OPZ ZLYPLZ will be Paired Trade-Off Analysis.

Like Discrete Choice Modeling... -\SS 7YVÄSL *VUQVPU[ (UHS`ZPZ PZ H [YHKL VMM and simulation technique useful for studying these types of questions:

Product/service design and pricing issues such as what features maximize preference or revenue

To what extent customers value features and what impact that value has on preference for the feature

Whether and how to bundle product or service features

Anticipated increases or decreases in revenues based on the presence, absence, or combination of features

Combinations of all these issues, as needed

MORE > MAY 2014 SURVEY MAGAZINE

025

Some of these issues can also be addressed by Discrete Choice Modeling. However, not all scenarios require the complexity of Discrete Choice Modeling; they can be quite effectively addressed using Conjoint Analysis.

Furthermore, lower levels of complexity often involve lower levels of investment in the research.

CASE STUDY Automotive & Trucking Parts Consider the following example. An original equipment manufacturer of automotive and trucking parts and services wants to develop a loyalty program to reward customers for their continued business. They have conducted enough preliminary research (including discussions with key internal stakeholders) to identify two different rebate program models, each of which offers a range of percentages for a cash-back program and different options for delivering the rebate. Furthermore, they have developed a list of non-cash incentives they would like to incorporate into the program which may also serve to increase customer loyalty. However, they want to know which rebate program at what percentage level and what combinations of non-cash features will be most appealing to their customers. They also want to know which program and features will have the greatest positive impact on customers’ spending levels with their business. These questions are quite effectively answered in the context of a Conjoint Analysis design.

-\SS 7YVÄSL

Design of the Conjoint Analysis Exercise

Just as in Discrete Choice Modeling, VUL VM [OL ÄYZ[ Z[LWZ PU [OL KLZPNU of a Conjoint Analysis exercise is to determine the number of attributes and levels to be studied. Attributes are the statistical equivalent of product features and levels are the variations within a particular product feature. In our rebate program example, the main attributes are the presence or absence of a rebate and of non-cash incentives to reward higher levels of purchases. Levels for the rebate attribute range from 1, 2, 3, 4, 5, or 10% of the cash value of purchases made and the point at which customers become eligible for the rebate. Different combinations of the non-cash incentives may also be considered levels. In Discrete Choice Modeling design, one of the most challenging decisions to make is determining the optimum number of attributes and levels to include. Because the technique allows for more complexity, it is often tempting to include a number of attributes and levels that has the potential to overwhelm respondents and lead them to abandon the survey. This is not typically the case with Conjoint Analysis since the scope of the exercise, while still realistic, is less complex than that of Discrete Choice Modeling. Because the number of attributes and levels is [`WPJHSS` ZTHSSLY [OHU ^OH[ VUL ÄUKZ in Discrete Choice Modeling, sample sizes are often much smaller than those typically required with a Discrete Choice design. Each respondent in a Conjoint exercise rates each of the options in the design, rather than using the fractionalized design discussed in our Discrete Choice Modeling article.

Conjoint Analysis 026 SURVEY MAGAZINE

MAY 2014

Once these decisions are made, the content of the conjoint exercise is programmed into the survey. At this point, one of the key differences between Discrete Choice Modeling and Conjoint is apparent. Whereas in Discrete Choice Modeling, the choice tasks include several options presented simultaneously, in Conjoint each option is presented individually. Our example included two different rebate plans, each of which had six different levels percentage levels for the rebate. The plans were randomized and within each plan, the levels were randomized as well. This helped to prevent bias as a result of “educating the respondent” as to what levels to expect. Each plan and each of the plan levels were presented to all respondents. Respondents were asked to indicate whether their willingness to purchase the client’s products and services would increase, decrease, or stay the same IHZLK VU [OL H]HPSHIPSP[` VM [OPZ ZWLJPÄJ YLIH[L WSHU and rebate percentage. If the respondent indicated an expected increase or decrease, they were asked to estimate by how many percentage points their spending would change.

The non-cash incentives were handled in a

similar fashion. Eight different combinations of the distinct non-cash incentives were presented in random order to all respondents, who were asked to indicate by how many percentage points their spending would increase or decrease based on the availability of this ZWLJPÄJ JVTIPUH[PVU VM PUJLU[P]LZ The number of non-cash incentives in any particular set was also varied; this helped the client determine the optimum number of incentives to include in order to maximize anticipated revenue gains.

Typical choices include variables to measure potential share of preference, purchase intent, or product appeal.

Non Cash Incentives

Free diagnostics and technical inspections

Additional discounts and special offers from client’s business partners

Free expedited shipping on special orders

Ability to earn points based on dollars spent which can be redeemed for merchandise, travel, or business equipment

Thinking only of these four incentives, would your willingness to purchase this supplier’s products and services increase, decrease, or stay the same if you YLJLP]LK [OLZL ILULÄ[Z HM[LY `V\Y W\YJOHZL&

SURVEY

tip

By how many percentage points do you anticipate your current spending level would increase or decrease if `V\ YLJLP]LK [OLZL ILULÄ[Z HM[LY `V\Y purchase?

MORE > MAY 2014 SURVEY MAGAZINE

027

>OH[ +VLZ -\SS 7YVÄSL *VUQVPU[ (UHS`ZPZ Provide as Results? -\SS 7YVÄSL *VUQVPU[ (UHS`ZPZ `PLSKZ ]HS\HISL PUMVYTHtion about potential share of preference, estimates of purchase intent, estimates of revenue, and can yield important information about competitive products, depending on the design of the choice tasks. Share of preference refers to the percentage of respondents ^OV JOVZL H WHY[PJ\SHY WYVK\J[ JVUÄN\YH[PVU V]LY others in the design. For example, it is possible to identify what proportion of the potential market is likely to increase their spending level based on the KPMMLYLU[ JVUÄN\YH[PVUZ VM [OL YLIH[L WYVNYHT HUK PUcentive package. This is helpful to develop estimates of potential increases or decreases in revenue. ,Z[PTH[LZ VM W\YJOHZL PU[LU[ YLÄUL [OLZL ÄUKPUNZ further by identifying how likely potential customers are to actually purchase the product (or products) they like the best. Just because a product is the most highly preferred among alternatives does not necessarily mean customers will purchase it. Looking at interactions between shares of preference and purchase intent helps to provide more guidance into WV[LU[PHSS` WYVÄ[HISL YLZLHYJO HUK KL]LSVWTLU[ KLJPsions. In some cases, perceived value of a product with a lower price can serve to decrease levels of purchase intent if the customer thinks the product is too inexpensive to consider. By looking at variations in price, share of preference, purchase intent ratings, and additional information from within the organization and other sources, -\SS 7YVÄSL *VUQVPU[ Z[\KPLZ VM[LU `PLSK estimates of market revenue. No study can take into account all market factors not included in the research design, such as advertising, product availability, changes in competitive products or competitive landscape, manufacturing costs, or other strategic initiatives. Consequently, while these revenue estimates should not be expected to allow for all conditions in the marketplace, they can, however, provide accurate estimates based on the quality of the design of the choice tasks themselves and the quality of information available from within the organization.

028 SURVEY MAGAZINE

MAY 2014

In situations where competitive products are included in the design, performance potential against competitive products can also be estimated, particularly in regard to head-to-head comparisons of products or services with similar features.

Utility Statistics

;OL YLZ\S[Z VM -\SS 7YVÄSL *VUQVPU[ (UHS`ZPZ HYL [`WPJHSS` reported as utility statistics or utilities which correspond to the anticipated increase in share of wallet if the specific product, feature, or service is offered. These statistics allow the organization to see at a glance which product VY ZLY]PJL MLH[\YLZ HUK ILULÄ[Z HYL TVZ[ HWWLHSPUN to the customer in the sense of relative importance. A sample display of utility statistics is shown below:

By translating the utility statistics into a measure of relative importance, organizations can see at a glance which features offer the biggest bang for the buck. In V\Y L_HTWSL [OL ILULÄ[ VM MYLL L_WLKP[LK ZOPWWPUN OHZ a relative importance of 34, which is more than twice the value for either free training or additional discounts, each of which have a relative importance of 15. Since the YLSH[P]L PTWVY[HUJL ÄN\YLZ HYL JHSJ\SH[LK [V HKK [V percent, head-to-head comparisons can be made of the ILULÄ[Z ILPUN L_WSVYLK HUK ZLL OV^ [OL` Z[HJR \W YLSHtive to each other. This means that for purposes of this L_HTWSL [OL ILULÄ[ VM MYLL L_WLKP[LK ZOPWWPUN PZ TVYL valuable to this set of customers than the combined efMLJ[Z VM MYLL [YHPUPUN VY HKKP[PVUHS KPZJV\U[Z ;OPZ ÄUKPUN was of great importance to our client and helped them structure an incentive plan that would not only satisfy their customers but also fall well within their budget for the program.

In Conclusion…

0U ZOVY[ -\SS 7YVÄSL *VUQVPU[ HUHS`ZPZ PZ H ]HS\HISL [VVS in both the B2B and B2C spaces to help discover the PTWVY[HUJL [V JVUZ\TLYZ VM Z\NNLZ[LK ILULÄ[Z HUK features to a product or service offering. While it cannot handle the complexity of Discrete Choice Modeling scenarios, the results are still statistically valid and very useful in making crucial business decisions. It has the added advantage of being straightforward to design, WYVNYHT PTWSLTLU[ HUK PU[LYWYL[ ;OLZL HYL ZPNUPÄJHU[ advantages in determining which statistical tools are most appropriate for answering the needs of organizations in a challenging business environment.

About Customer Lifecycle, LLC www.customerlifecycle.us Customer Lifecycle is a global research-based consultancy committed to helping our clients avoid costly mistakes by focusing on thorough front-end planning, appropriate support for research execution, and in-depth deployment consulting and implementation at the back end. Outcomes are rigorous and balanced customer-focused performance TL[YPJZ PTWYV]LK ÄUHUJPHS results, and a superior total customer experience. Its mission is to provide companies with insight into their industries and staff by deploying sophisticated analyses to answer tough business questions, and intelligence that clients can act on with JVUÄKLUJL [OLYLI` VMMLYPUN HU edge in understanding customer choice, engagement, loyalty and advocacy. Each stage in the customer lifecycle—acquisition, service, growth, retention—has its own unique challenges and solutions [V HKKYLZZ ZWLJPÄJ I\ZPULZZ issues. Customer Lifecycle helps both B2B and B2C focused organizations plan and conduct research to accurately identify and measure customer requirements for satisfaction, loyalty, and retention at every stage of the relationship and to deploy and integrate customer requirements for performance into the processes and internal performance metrics of the organization. Liaison We welcome any questions you may have about this thought piece. Please direct all inquiries to: karin a ferenz | principal T: 630 412 8989 F: 630 235 9834 kaferenz@customerlifecycle.us

END MAY 2014 SURVEY MAGAZINE

029

SOCIAL ONLINE RESEARCH

The social online research panel is emerging as an effective platform to engage your customers in feedback through survey responses and other methods.

030 SURVEY MAGAZINE

MAY 2014

Traditional

Panels Traditional panels that reward members with prizes and points for taking surveys have been experiencing higher panelist attrition

PANEL HOPPING? Immense competition is rampant within the market research industry to recruit and keep panelists, resulting in many instances where the same consumers belong to multiple panels. Or consumers go “panel hopping” and join one panel and then quickly leave in search of the most appealing incentives. Further exacerbating the problem is increasing panelist fatigue, which has led to higher attrition rates. And the one-way communication between the traditional panel provider and their panelists has led to disengaged and disinterested respondents.

rates and growing costs per interview, and they face a general deterioration in data quality.

BUILDONG SOCIAL PANELS Building social panels can alleviate many of these panel development and management challenges. Social panels are designed to enhance two-way communication among the panelists, resulting in more highly engaged respondents who provide better insights and become much more loyal members.

& MANAGE How to

DEVELOP

a Social Research Panel WRITTEN BY: DARREN BOSIK, SENIOR METHODOLOGIST, QUESTBACK

MORE > MAY 2014 SURVEY MAGAZINE

031

Socialize the Panel

The panel socialization begins once it is up and running. Panel members are encouraged to interact with one another. The experience for them becomes much more two-way in which they share ideas and opinions. Instead of a panel manager or moderator feeding questions to panelists in a one-way communication format, the panelist is highly engaged by participating in research that appeals their interests, lifestyles, habits, hobbies, and other characteristics they have revealed about themselves.

Phased Approach In this era of social media, social panel development is necessary and even JVZ[ LMÄJPLU[

The panel manager, in essence, is creating a mini-community of likeminded panelists where they can provide feedback about a given Z\IQLJ[ 7HULSPZ[ WYVÄSPUN PZ JYP[PJHS to the success of the social online panel and it is important for the panel manager to keep accurate information about panelists throughout their

Panelist Acquisition Use a double-opt in process to invite your panelists to join through an email, but then get them [V JVUÄYT [OLPY membership via a TVYL YPNVYV\Z WYVÄSL survey that asks questions about their demographics (gender, age, and income) and other important screening information.

lifecycle. Routinely surveying panelists about new life events, such as getting married or having a baby will further LUYPJO [OL WYVÄSL VM [OL WHULSPZ[ @V\ should also survey panelists and grow your knowledge about their attitudes towards your brands, which can change over time.

SURVEY

The socialized feedback model is the foundation of transforming a traditional panel into a social panel. Many companies are increasingly building online communities of interest consisting of key stakeholders such as customers, constituents, employees and partners. It helps them to better engage with their customers, listen to their needs, and act upon them. Such communities represent an opportunity for conducting meaningful, consistent and timely online research by engaging panel members in deliberate, regular feedback.

Reward

Management

Panel

Management Build a panel Web portal and maintain the master panel data by tracking key metrics around the panel health, such as number of surveys completed per panelist and time to complete a survey.

Develop an incentive program to ensure that your panelists are adequately compensated for their time on the panel. These rewards, which typically consist of cash or gift cards, help keep panelists engaged and can increase their longevity on the panel.

Retention Reactivate lapsed panelists or reengage panelists who are at risk for leaving the panel. It is much more costly to acquire a new panelist than extend the lifetime value of a panelist.

Four Phased Approach

Building a social panel begins with a four-phased approach. 032 SURVEY MAGAZINE

MAY 2014

tip

SOCIAL PANELS

Social panels also let companies execute both qualitative and quantitative research

ENGAGE

since you can easily pull together a smaller social group for qualitative and still have a larger panel available for more data-intensive quantitative research. Social panels often exploit the collaborative functions commonly found in online communities, such as forums, blogs and discussion groups. The technology enables the panel manager to listen to the natural dialogue of customers interacting with other customers.

Forums or bulletin boards bridge quantitative and qualitative research

by interacting and engaging in three-way dialogue with panelists. This technique connects clients with customers and also connects panelists to each other. By monitoring these discussions, companies can assess interest in different topics and gauge their audience perceptions of concepts or new product ideas.

The most empowering feature of social panels is the ability for panelists to engage with each other. As panelists react to responses of others, you enable a deeper level of consumer insight. The use of a “star rating” system can allow one panelist to rate another and comment on each other’s reactions. The most valuable comments can be converted into a poll for other panellists to rank.

“

Those listening to the songs are invited to take part in a survey and to discuss what they like and dislike about the songs among one another.

“

SOCIAL

Quant and Qual Research Possible

CASE STUDY

Time Warner Listens to Its Social Panel One company testing concepts through a social panel is Time Warner Music, which recently made the decision that a representative group of music fans should be formed in order to recognize current trends in the music business. The company heavily relies on the opinions of its social community of more than 10,000 music fans who have expressed their willingness to make statements about their music habits and to answer questions about artists and the marketing of their music. The panel members are asked to assess new sales points or advertising campaigns and to help in determining future market trends. Members have access to songs by new HY[PZ[Z PU ]PKLV VY ZV\UK ÄSL MVYTH[ VU [OL WYP]H[L WHULS ^LI-

site. Those listening to the songs are invited to take part in a survey and to discuss what they like and dislike about the songs among one another. As a result of its social panel efforts, Warner Music can develop targeted advertising measures and optimize their advertising expenditures. The socialized customer panel has proven to be a quick opportunity for Warner Music to try out new ideas before the actual market launch – a process that saves the company from making expensive mistakes, The panelists are rewarded for their participation with bonus points that they can view at any time and trade in for prod\J[Z VY MVY WHY[PJPWH[PVU PU YHMÅLZ

MORE > MAY 2014 SURVEY MAGAZINE

033

Leverage Facebook Fan Base Social panel development is made easier by tapping into an online resource you probably already have – your company Facebook page. Facebook has become a de facto social media platform for hundreds of brands. Facebook communities, therefore, are a OPNOS` LMÄJPLU[ HUK JVZ[ LMMLJ[P]L ^H` [V JVUZ[Y\J[ H free, but closely-targeted, demographic database or social panel sample group. Asking Facebook fans for feedback on new products and services is an effective way to gather insights from customers already invested in what you have to say and offer. The real payoff to establishing a Facebook presence comes when you deploy a feedbackbased strategy that engages your fan base regularly through a feedback loop -- or a systematic way of routinely collecting insights about your Facebook fans. You can easily build a private online social panel directly on your Facebook page. Your fans simply can log-in to the panel without ever leaving Facebook. Invite some of your Facebook fans to participate in the panel and you can interact with them in many creative and insightful ways by collecting their data through surveys, online chats, and focus groups. A IPN ILULÄ[ ^PSS IL [OH[ `V\ ^PSS V^U HSS VM [OL KH[H you collect from your Facebook fans. You can better leverage the feedback for insight development that can improve your relationships with your fans and develop more targeted messaging to your Facebook JVTT\UP[` @V\ JHU WYVÄSL ZLNTLU[ HUK [HYNL[ JVTmunications appropriately and manage activities, fan feedback, and their participation.

034 SURVEY MAGAZINE

MAY 2014

The technology to build and manage social panels is typically referred to as an enterprise feedback management suite. This software gives you the ability to develop surveys for computers or mobile devices and analyze data from social media and questionnaires, all in one centralized location. Your business can develop the online social panel, integrate it with your Facebook page, and handle all of the panel management functions in one place. You can create robust panelPZ[ WYVÄSLZ HUK W\SS different respondents together in specialized and highly socialized panel community groups. You can build an online community portal from the ground up HUK OH]L [OL ÅL_ibility of doing many types of feedback research.

& MANAGE How to

DEVELOP

a Social Research Panel

MAY 2014 SURVEY MAGAZINE

035

Asking is an art. Or rather, a science.

The way we ask a datum or an opinion to a person, determines the results we obtain. This is the reason why the creation of a questionnaire must be carried out by an experimented researcher, paying close attention to the effects that the design of the questions can have on the study results.

Carlos Ochoa

Netquest, Marketing Manager

036 SURVEY MAGAZINE

MAY 2014

Different

scales different results The impact of asking about income and the size of the house in three different ways Set up & design In this article, we analyze the results of an experiment done with respondents from an online access panel (Netquest) in Spain, Mexico and Colombia. Respondents were randomly assigned to three groups. Each group got a different version of the questionnaire, where the question itself was kept the same but the scales varied. We compare three ways of asking the respondents’ household and personal net income per month (which are quite sensitive questions) and the size of their house (less sensitive): an open question, a classic radio button and a dropdown menu. Firstly, the distributions in the different groups are compared, with a special focus on the

proportion of “prefer not answering” obtained. >L ÄUK [OH[ T\JO TVYL YLZWVUKLU[Z JOVZL [OPZ option when it is combined with a text-box (open question). Then, the correlations with two external variables (working situation and number of children under 10 years old) are computed. We expect high correlations, mainly for personal income and working situation. Since the highest correlations between these two variables are found for the radio button scale in the three countries, it suggests that the classical radio button scale may be the one performing better for the questions studied.

MORE >

MAY 2014 SURVEY MAGAZINE

037

variables Asking about objective variables in surveys: limits, experiment and data

As everywhere else, in surveys there is no measure without errors. When asking about attitudes, opinions, brand preferences, measurement errors are expected to be quite large. Respondents TH` UV[ OH]L Ă&#x201E;_LK WYLMLYLUJLZ [OL` TH` JOHUNL [OLPY VWPUPVUZ KLWLUKPUN VU [OL JVU[L_[ PU which a question is presented, or on their mood at the moment of answering.

Therefore, the ordering and exact wording of the questions, as well as the conditions in which the survey is conducted (mode of data collection, place, time, presence of other persons) are primordial.

A lot of research gives guidelines about how to design good questions and good questionnaires

038 SURVEY MAGAZINE

MAY 2014

QUESTIONNAIRE DESIGN However, when asking about more objective variables, we often tend to forget about measurement errors and about the rules provided by the literature on questionnaire design. There is this idea that concrete and objective variables (e.g. age, nationality, working situationâ&#x20AC;Ś), for which it exists one true value that could be assessed without the need of asking the individual and for which this true value is stable enough to stay the same during the time of the survey, will not be affected by the ordering, the exact wording or the conditions of the survey. By consequence, we often spend much less time thinking about the best formulation for these variables.

SURVEY

tip

Different

scales different results Nevertheless, there are reasons to expect measurement errors for these variables as well. Firstly, some of these objective variables can suffer from social desirability bias. This means that there are some answers socially seen as preferable and that, therefore, some respondents tend to choose these answers to present themselves in a favorable way. For instance, some respondents may prefer to say that they have the nationality of the country in which they are living, instead of a foreign nationality. Secondly, some respondents do not like to give personal objective information: they feel that these questions are intrusive and do not understand why the researchers need to ask them. An example is a question about the religion of the respondents we asked in a survey: we got several respondents complaining about it because they feel it was “not of our business”. Thirdly, respondents may not know at all or not with exactitude the answers. This holds also for questions we sometimes think are very easy, like the nationality of your father (e.g. if a respondent was adopted). In this study, we focus on three objective variables: household net income per month, personal net income per month and size of the house. All three can suffer from social desirability bias, even if we would expect that the two income questions are more sensitive. All three can be perceived as intrusive, and at least the income questions are. Indeed, in an open ended question at the end of the questionnaire, several respondents complained about these questions being asked. Finally, it is possible for all three questions that the respondents do not know the exact answer. Respondents may not be aware of the income of all the people in the household. Adult children living with their parents, for instance, may have no idea about their parents’ income. Respondents may also not know what their exact net income is: some people have jobs where the income varies from month to month (e.g. H\[VUVTV\Z HUK ^OLYL P[ PZ KPMÄJ\S[ [V JVTW\[L [OL UL[ income from the gross income. Respondents may also not know the size of the house in which they are living.

MORE > MAY 2014 SURVEY MAGAZINE

039

Using three different scales We use three different scales or methods in the experiment in order to measure each of the three ]HYPHISLZ ;OL Ă&#x201E;YZ[ TL[OVK PZ HU VWLU X\LZ[PVU with a text box in which the respondents have to write down the income or the size of the house. However, being asked the exact number may seem more sensitive and intrusive than being asked to select the interval in which one is situated. Besides, asking for intervals instead of exact values also may reduce errors due to the respondents not having the exact knowledge of the numbers asked. Therefore, we expect by proposing a scale with intervals as response categories to get better results. The most common scale used in web surveys for single choice

questions is a radio-button scale. Thus, this is our second method. Finally, we use as third method a dropdown menu scale, because we ^HU[LK [V [LZ[ H ^LI ZWLJPĂ&#x201E;J ZJHSL HUK ILJH\ZL it has been argued that â&#x20AC;&#x153;drop boxes may be useful when the respondent must search a long list of response options (e.g., state or country of residence or make and model of car) for a well-known target answerâ&#x20AC;? (Couper et al., 2004, pp125). This could be the case of our three questions. Methods 2 and 3 propose exactly the same intervals as answer categories, with 4 categories for the income questions and 11 for the size of the house.

For the two income questions, because they are more sensitive, we also included a â&#x20AC;&#x153;prefer not answeringâ&#x20AC;? option in all three scales. For the size of the house, we included it only for the open question. Table 1 provides screenshots of the scales for the income questions. The complete survey is available online:

Different

scales different results 040 SURVEY MAGAZINE

MAY 2014

The impact of asking about income and the size of the house in three different ways Melanie Revilla, Co - Author RESCM, Universitat Pompeu Fabra

This experiment was implemented in a survey completed between the 14th of May and the 18th of June 2013 by the respondents of the Netquest online panel (http://www.netquest.com/en/panel/index.html) in Spain, Mexico and Colombia. Using this data, we show that not only the distributions but also the correlations with other external variables vary depending on the scales used to ask the information about the three objective variables.

Table 1: Distributions Income and size of the house ( 32('''" (( -( )/ '- (''(" (,''" () 0' (+ (0 (,'(" ),''" (, +/ (( -( 4),'(" (- +, + (0

*(' 3$),'' ) -- . 00 $),'( $,,'' - /' (( )+ $,,'( $((''' (* ') (. (- 1$((''' *( *. )' () **/ 3)'' '''$ ) ,- - /+ )'( '''$ -'' '''$ ) /, * +) -'( '''$ ( ''' '''$ . +( 0 0. 4( ''' '''$ )/ .. )( '/

*,(

) * (. (' ! ( *) ., (* )) *, /( ! ( (- ,) (, 0+ (. 0( ! ( (0 ,- ! ( ! ( ) (, '. (, +* (/ /* - */ (, 0/ * ,/ ! ) ). /* ! ( )0 /- ! ( ), '. ! ( ), -) ! ( *+, *-* * 0) (+ .( ! ( * ** (' '' ! ) ! ( ! ( ! ( (* '. (. 0. (+ (. (. )) ! ( ! ( ! ( ! ( ), /) )- (+ )* ' ), '' ! ( ! ( ! ( +( (/ )+ (/ +) ./ ). ./ ! ( (- '( ! ( (- 00 ! ( (- -. ! ( )' '' ! ( *'- *-' ) 0* . 0) ( 0( (( ./ ! ( ) , )/ ! ( (+ *. ! ( (( ./ ! ( ) (. /* ! ( (- 0- ! ( (- (* ! ( (, -( ! ( (. )' ! ( ! ( ! ( ! ( ,/ ' *0 *' ,) /. *) (. ! ( ) ! ( ! ( ! ( (/ .. )) )0 (. /* )( ') ! ( *+( *(+

Table 1 gives the distributions of answers for the household and personal net income per month in the different countries and for the different methods. Respondents are randomly assigned to get one or another method, so ^L L_WLJ[ ZPNUPĂ&#x201E;JHU[ KPMMLYLUJLZ HJYVZZ NYV\WZ [V JVTL MYVT [OL TL[OVK \ZLK Table 1: Distribution of answers for household and personal net income per month 5V[L! 4P TLHUZ [OL JVLMĂ&#x201E;JPLU[ UL_[ [V P[ PZ ZPNUPĂ&#x201E;JHU[S` KPMMLYLU[ H[ SL]LS MYVT [OL VUL \ZPUN 4P

MORE Â > MAY 2014 SURVEY MAGAZINE

041

However, there are differences across countries. On the one hand, in Colombia, these proportions are much higher for M1: around 58% of the respondents prefer not to answer, versus around 45% in Mexico and 42% in Spain. The questions may be perceived as more sensitive or intrusive in Colombia. But, on the other hand, for M2 and M3, the proportions of non answer are quite similar in Mexico and Colombia and even slightly higher in Spain. So, it seems that there is also an interaction between the kind of scale and the country. In Colombia, this leads to very large differences in non-answers across the three methods. 042 SURVEY MAGAZINE

MAY 2014

Thirdly, we see that the radio-button and drop-down menu leads to quite similar answers, especially In Mexico. ( ML^ KPMMLYLUJLZ HYL ZPNUPĂ&#x201E;JHU[ between M2 and M3, for instance in Spain, for household income, the proportion of respondents in the income category â&#x20AC;&#x153;1500-2500â&#x20AC;? is 19.71 in the radio-button group and 27.82 in the drop-down menu group. But similarities prevail.

Table 2: Distribution of answers for the size of the house in m2

These proportions are really similar when the question is about household or personal income, which tend to support more the second reason: more respondents should know their personal income than the household income so if the lack of knowledge is the reason and not the sensitivity, we would expect differences in non answer for both questions.

Secondly, it seems that the respondents that decide not to answer when an open question is used but would have answered if a scale with intervals were used are not concen[YH[LK PU H ZWLJPĂ&#x201E;J PUJVTL JH[LNVY` since the proportions in all categories are higher for M2 and M3 compared to M1, except for a few exceptions. Nevertheless, the differences are not HS^H`Z ZPNUPĂ&#x201E;JHU[

Firstly, we see that when using an open question (M1), the proportions of â&#x20AC;&#x153;prefer not answeringâ&#x20AC;? are much higher in all countries than when using intervals scales (M2 and M3). Therefore, it seems that by not asking the exact information but only the range in which the respondents are situated, much more of them give a substantive answer. This may be because they do not know the exact value or because they feel more comfortable with providing only an approximation.

2*% ' *& ,* ' ,+ .% ' .& &%% ' &%& &*% ' 3&*& ' 2*% *& ,* ,+ .% .& &%% &%& &*% 3&*& 2*% *& ,* ,+ .% .& &%% &%& &*% 3&*&

1 * &+ &- ,& '( ** &' '+ &* )- - (. (&% - '- && ') , &% * +' &) ,. '( .+ ((- &( && &, %. &% '+ . ., && )% && +- (*&

& 1 + &- '' (. '- &. &) +, &- *( &% %) '*. && +, &* -( &% %% , .' '% -( (( ,* ')% &, -( '( '+ &( .* &( *, &* *% &* -. '*-

' 1 - ,% '% '. '& ,) " &! &, ., &- -) ()* &+ %& &) %* && ,+ && ,+ '% '+ '+ &) " &! (%+ &( '% '& && &* *) &( ,- '% -' " &! &* *) ()&

( 1 + () '& ). " '! &) (( &, .& . .' (+( &( %+ &- (( " & '! && (. " & '! '* '- " &! (+% &, *' '' .( '% %+ " &! &% *& &+ *+ &' )' (&)

Size of the house Table 2 gives the distributions for the question about the size of the house. For the sake of clarity, we combined some of the answer categories, so that we have 6 substantive categories instead of 11. Since the option “prefer not answering” was only proposed in the open question format, for this method, Table 2 presents both the distribution of answers including all the respondents of the corresponding group and the distribution excluding the respondents that prefer not to answer. In that way, the distributions for the substantive answers can be better compared.

The proportions of “prefer not answering” in the open question format are much lower for this question in all countries. This supports our hypothesis that this variable was perceived as less sensitive and intrusive. Nevertheless, in Mexico (28.99%) and Colombia (26.5%), still more than ¼ of the respondents chose this option. The lower proportion is found for Spain as in previous questions (16.45%). Comparing the distributions once respondents that prefer not to answer in M1 are excluded, we observe relatively similar results HJYVZZ [OL [OYLL TL[OVKZ /V^L]LY [OLYL HYL ZVTL ZPNUPÄJHU[ differences in proportions mainly between the drop-down menu HUK [OL [^V V[OLYZ PU 4L_PJV 0U *VSVTIPH [OLYL HYL VUS` ZPNUPÄcant differences with the open question and only in two cases. In :WHPU [OLYL HYL HSZV VUS` [^V ZPNUPÄJHU[ KPMMLYLUJLZ

M2 Different

scales different results

MORE > MAY 2014 SURVEY MAGAZINE

043

Correlations with two external variables Income and size of the house are most often used in relation with other variables and not only for descriptive purposes. There are used, for instance, as a control of variables or as a predictor of other variables. Therefore, it is important to consider not only how their distributions are affected by the use of different scales but also how their relationships with other variables are affected. In order to do so, we look at the correlations of our three questions with two external variables: the number of children under 10 years old and the respondentsâ&#x20AC;&#x2122; work situation (measured on a scale from â&#x20AC;&#x153;1-working full timeâ&#x20AC;? to â&#x20AC;&#x153;4â&#x20AC;&#x201C;not workingâ&#x20AC;?). The results can be viewed on table 3. We should notice that we excluded respondents that prefer not to answer and that for the open question, we created a categorical variable with similar categories as in the two other scales in order to compute the correlations. This is to avoid differences due to the nature of the variables.

The external variables are chosen for two main reasons. Firstly, they were asked in a similar format for all the respondents, independently of the group (i.e. independently of the method they got for the income and size of the house questions). This is necessary in order to be sure that variations in correlations come from the method used for the three questions of interest and not from differences in the external variables format. Second, they are expected to correlate highly with our three variables of interest, especially in the case of the working situation and personal income. For the others, this is less obvious. But these were the best options we had in this questionnaire to have common scales for all respondents for the external variables.

Table 3: Correlations of our three variables with the number of children and working situation

COL

MEX

SPAIN

$ $+%( &#,* $)%+ (+&+ $#$+ $#&+ %)&( #&&& ((&+ ##+* #$%) #%,+ #,)& ')+$

% #,)$

$*,* $&%*

#(+$ $ #+&& $)*' #(%, $ #$#' $ #%#$

& % &()+ (&%$ #)+) $ % #%$) $ %$*$ % #,%# '##* $ % #$+* #((% % $ #(') $ (%&# #)$, #$)#

Note: In bold: highest absolute correlation across the three methods

Correlations 044 SURVEY MAGAZINE

MAY 2014

SPAIN

MEXICO

COLUMBIA

Most of the time, the signs of the correlations are similar for the three methods but the size varies quite a lot X\P[L ZVTL ZPNUPĂ&#x201E;JHU[ KPMMLYLUJLZ There are a few cases where even the signs differ: for example, in Mexico, when a radio-button scale is used, we get a positive correlation of household income with the number of children under 10 years old (.0581) but a negative correlation (-.1320) when the open question is used. In Colombia, signs are changing even more often from negative to positive depending on the method used to ask about the income and the size of the house. This may be related to the highest rate of non response (saying â&#x20AC;&#x153;prefer not answeringâ&#x20AC;?) in this country.

In Spain, the highest correlations with the number of children under 10 years old are found when respondents answered the income and size of the house questions using the drop-down menu. However, the highest correlations (in absolute values) with the work situation are found when respondents answered the income and size of the house questions with a radio-button scale. In Mexico and Colombia, the picture is even less clear. The highest correlations are found for different questions using different methods. In Mexico, most of the highest correlations are for the radio-button scale, but in Colombia, there is no systematic pattern.

;OLYLMVYL P[ PZ KPMĂ&#x201E;J\S[ [V ZH` ^OPJO scale performs better. However, if we focus on the correlation between personal income and work situation, which is the one for which we expect ^P[O TVYL JVUĂ&#x201E;KLUJL H OPNO HIZVS\[L correlation, in all three countries, this correlation is higher in absolute value when a radio-button scale is used. This might indicate that the classic radio-button scale is the one performing better.

Different

scales different results

Conclusions This study clearly shows that the choice of the scale is of crucial importance, since it affects both the distributions and the relationships with other variables. This holds even for objective variables, not only for attitudes or opinions. Moreover, this holds also in a self-completed mode as a web survey, even if it is often argue that the level of social desirability bias is lower in the absence of interviewers. Thus, researchers should very carefully design these questions as well. These variables are used in many substantive analyses as control variables (in particular income). Since the relationships between these variables and external variables are affected by the choice of the scale used to measure them, it is of key importance to think deeply about the scale to use. In this experiment, we considered three possible scales. There are of course many more possibilities, but we limit our conclusions to what was studied here. Because it leads to much higher proportions of non-answers, the open question format seems not to be the best option. The radiobutton and the drop-down menu are more similar in terms of distributions and proportions of respondents that prefer not to answer. However, looking at the correlations with two external variables, we observe differences in the size and in one case in the sign of the correlation. If we focus on the correlation between personal income and working situation, which is the one for which we expect high correlation with [OL OPNOLZ[ JVUĂ&#x201E;KLUJL [OLU ^L ZLL [OH[ [OL JSHZZPJ YHKPV button scale leads to the highest correlations in all three countries. This suggests that this may be the scale performing better.

END

MAY 2014 SURVEY MAGAZINE

045

BOB Y AZB E C K

@_Gongos

From Insights to Intelligence: The Next Decade of MROCs

Born from a brand advocacy platform, online research communities were a welcomed disruption in our industry. By 2004, they established themselves at the intersection of practitioner, platform and consumer in providing dynamic means to hear consumers, observe behavior and collect data. One decade later, communities have to become even more sophisticated, accounting for internal challenges and new streams of data. We believe they must disrupt again—moving teams beyond pure insight generation to better fuel intra-organizational intelligence.

Intelligence communities—if we can call them that—stand to become a pivotal platform for future-thinking insights organizations. At their peak performance, they will catalyze cross-functional learning—and decisions large and small—throughout the entire organization.

But this will take a mindset shift on all of our parts. Insights professionals will need to place as much focus on the organizational relationship with the community as the consumer-vendor relationship. To achieve this, end-users must regard communities as a hub of intelligence, rather than an “address” to engage in meaningful chatter with their customers.

In order to break the deeply engrained pattern of primary data collection and insight NLULYH[PVU ^L»]L PKLU[PÄLK MV\Y ZWLJPÄJ areas where communities stand to evolve:

Likewise, insights partners will be prompted to design more intuitive and interactive environments engineered to collect, consume, and curate knowledge within the platform itself. Proportionately, these hubs will have to be crafted to facilitate cumulative learning. 046 SURVEY MAGAZINE

MAY 2014

A LA CARTE LEARNING TO REAL-TIME COLLABORATION 9H[OLY [OHU TV]PUN MYVT VUL Ä_LK Z[HNL of questions and objectives, to another of answers and themed reports, community partners need to work in tandem to see stories unfold. Historically, suppliers and clients diverge onto two paths – suppliers head off to generate insights while clients develop discussions to shape the issues.

Bob Yazbeck Vice President, Digital Methods, Gongos Research

A truly collaborative relationship will be far more iterative. All parties will be as equally able to absorb inbound knowledge as they fuel continued learning initiatives. Community platforms will be constructed to behave more like an intranet of knowledge – where realtime notes, comments, playbooks and next-moves will help facilitate graduated learning within the client organization and with their partners.

COLUMNS M AY 2 0 1 4 SUR VEY MAGAZINE FEATURED COLUMNIST

65305, 9,:,(9*/ *644<50;0,:

FEEDING ORGANIZATIONAL INTUITION For communities to remain relevant there needs to be a clear and explicit linkage between the insights-fed story and the organization’s strategic planning. In a perfect world, this happens as a top-down approach – plans are developed, objectives are clearly stated, and work is initiated. Yet, many of us (yes, clients too) do not operate in a perfect world, so a bottom-up approach often prevails. Visualize H Å\PK UL[^VYR ^P[O H OPLYHYJO` UVKLZ HUK WH[OZ ¶ Z[YH[LNPJ WSHUZ are the framework, objectives and research activity are the nodes, and the story describes the path. It’s through a network where we TVYL LHZPS` \UJV]LY JVUULJ[PVUZ [OH[ ^LYL WYL]PV\ZS` KPMÄJ\S[ [V detect. This network of knowledge needs to reach beyond the initial audience and live on in the organization—not only to provide inspiration, but serve as a blueprint when hypotheses arise and bottom-up approaches are in order.

AN OPEN ECOSYSTEM Environments and systems are often strengthened through the in[YVK\J[PVU VM V\[ZPKL PUÅ\LUJLZ @L[ TVZ[ 496*Z [VKH` HYL IHZLK on private, invitation-only forums. The community environment has to evolve to incorporate qual, quant, social and new layers of system KPHNUVZ[PJZ ¶ H MYLL ÅV^PUN WVY[HS [OH[ PUJVYWVYH[LZ YLSL]HU[ HUK meaningful real-world factors. There should also be an ability to fuse intelligence from outside the environment—from primary, secondary or enterprise research, social media measurements, and timely events— that affect the community alchemy. This membrane effect ^PSS WYVTV[L [OL ÅV^ VM PUMVYTH[PVU [V IYLHR KV^U Z[H[PJ RUV^SLKNL JVYWVYH[L ZPSVZ HUK M\Y[OLY PUÄS[YH[L [OL VYNHUPaH[PVU

HUMANIZATION OF INSIGHTS In a world of data clutter, humanizing members will become ever more important. With a foundation of engagement, trust, and anonymity—where “public privacy” is otherwise constantly challenged—intelligence communities will continue to give consumers a safe space to express themselves. Clients can engage with members face-to-face – through smart devices or PCs – in a scalable fashion. And since self-expression is becoming far more prevalent and accepted among social users, there is no more powerful proof than in personal cameos. They allow for more character-driven narratives in the quest for consumer intelligence, as well as the socialization of that knowledge throughout the organization

MAY 2014 SURVEY MAGAZINE

047

HYBRID QUALITATIVE RESEA RCH Capture the “in-themoment” responses. When conducting an in-home product test, online and mobile research tools give the researcher the chance to be there with the participants.

Uncover Deeper Insights with Hybrid Qualitative Approaches In the past, “hybrid” research studies were usually qualitative studies mixed with a quantitative survey to validate the results. We might ask participants to keep a diary or do some homework before a focus group, but the conversation in a study generally didn’t begin before the researcher met the participant on research day.

;VKH` ^P[O [OL ÅL_PIPSP[` PU our research toolkit, we have an opportunity to get really creative in reaching our research participants. Combining online qualitative approaches with the same participants to be interviewed in-person allows researchers to build a rela[PVUZOPW ^P[O WHY[PJPWHU[Z ÄUK PUZPNO[Z MYVT multiple avenues and deepen understanding. Online qualitative tools have been available for years, but the rise of social media and [OL PUÄS[YH[PVU VM ZTHY[ WOVULZ UV^ THRL P[ L]LU LHZPLY [V UV[ VUS` ÄUK WHY[PJPWHU[Z [V WHY[PJPWH[L I\[ HSZV NP]L ÅL_PIPSP[` PU how they participate and what they share. Photos, videos, mobile experiences are all readily available and add richness to a study. The latest online platforms allow researchers to capture these contributions and organize them into meaningful results as we would in person. 048 SURVEY MAGAZINE

MAY 2014

The following are a few reasons to consider adding an online component to an in-person qualitative study in order to make the results more impactful. Build familiarity and rapport before meeting participants. I like to call it the Facebook effect. If you’re a regular Facebook user you recognize how building an online relationship with new and old friends creates familiarity and comfort so that when you meet in person again, the need for small talk is unnecessary. Similarly, when you work with participants in an online space before meeting them at a shop-along or focus group, you pick up where you left off. The basic introductions are out of the way and you can get into the good stuff. Participants are more comfortable (even excited) working with you in person since you’ve been communicating online.

COLUMNS M AY 2 0 1 4

Harness the power of both methods. Participants tend to give more details online, but the facial expressions and body language revealed in person are missing in a digital interview. When we ask participants to make a video at home, the results are often less than ideal for use in presentation. The sound and lighting might be bad; and, worst of all, we can’t probe. By combining methods, we can start with the details participants describe online and then probe further in person, looking for those non-verbal cues and capturing it on video that we can actually use. Capture the “in-the-moment” responses. When conducting an in-home product test, online and mobile research tools give the researcher the chance to be there with the participants. By asking participants to pick up a product, try it at home and record their experience before, during and after use, we can capture these details before they are forgotten. Then, use the time in the focus group room to get into concept evaluation or other big picture discussions instead of likes and dislikes. You can also re-screen or re-sort participants based on their experience with the product, so that the room isn’t full of “haters” or one particular bad apple doesn’t ruin the entire group experience. Several more reasons for combining online and in-person approaches can be discovered with some creative thinking. Many of today’s tools are affordable and make the additional cost of adding a 2-3 day online discussion to an in-person approach worthwhile, given how much deeper the experience ultimately becomes.

SUR VEY MAGAZINE FEATURED COLUMNIST

C A SE Y B ER N A R D

@QRCA

Casey Bernard QRCA Member President, CB Research Services

Casey Bernard is an active member of the Qualitative Research Consultants Association (QRCA) and serves on national committees within the organization. Her company, CB Research Services, provides customized research solutions using traditional approaches and modern technologies. She is a graduate of The University of Texas Master of Science in Marketing Research program and lives in Austin, Texas. QRCA provides industry-leading resources that are essential to its members and the professionals who use qualitative research. As an association dedicated to advancing the discipline of qualitative research worldwide, QRCA’s nearly 1,000 global members apply their passion, creativity and experience to help clients tap into the power of qualitative marketing research.

MAY 2014 SURVEY MAGAZINE

049

;OL 3PZ[LU 3HK`

A NNIE P ET TI T

@LoveStats

I hereby ban weighting

I have to admit it. I love weighting data. I love putting the raw frequencies into excel, creating equations, and calculating 10 decimal place weights for up to 50 demographic groups. I love watching the frequencies of the ÄUHS KH[HZL[ THNPJHSS` TPYYVY [OL VYPNPUHS intention of my sampling plan. Maybe I’m crazy. Maybe you’re no different.

quite small and the risk of bias will be extremely minimal. It’s easy, it’s quick, it’s accepted industry-wide, and best of all it’s free.

When we develop sampling plans, it is with a clear research objective in mind. We want to collect a small sample of VWPUPVUZ MYVT H ]LY` ZWLJPÄJ NYV\W VM people so that we can generalize our reZ\S[Z IHJR [V [OH[ ]LY` ZWLJPÄJ NYV\W VM people. Not back to a group that kinda, sorta hints at the intended group.

It’s also lazy. The reason you originally planned for 100 completes is because you wanted opinions from 100 people, not 97 people, each of whose opinions you would value just a wee bit more than everyone else. It may just be 3 people but those are 3 people with unique family situations, life experiences, likes, dislikes, opinions, preferences, and perspectives on life. And you’ve just deliberately ignored them for the sake of convenience.

So I have to wonder. If we wanted 100 completes from men aged 18 to 24 and we’ve only got 97, we’ve got two VW[PVUZ :[H` PU ÄLSK HUK NL[ HUV[OLY 3 completes. Well, that costs time and money, and the project is probably a week overdue already. We know that’s what should happen but it’s not practical. The second option is to the weight the data. And hey, since we’re only missing 3 completes, the weights will be

050 SURVEY MAGAZINE

MAY 2014

6WD\ LQ ðHOG *HW PRUH completes. It may not be practical but didn’t you design your research plan that way for a reason?

Annie Pettit, PhD *OPLM 9LZLHYJO 6MÄJLY Peanut Labs

Annie Pettit, PhD is the Chief 9LZLHYJO 6MÄJLY H[ 7LHU\[ 3HIZ Vice President, Research Standards at Research Now, and Editor In Chief of Vue magazine. She specializes in listening research, survey methods, and data quality. Annie is a sought after conference speaker, and has published numerous refereed and industry articles. She won Best Methodological Paper at Esomar 2013, and the 2011 AMA David K. Hardin Award. Annie tweets at @ LoveStats and is the author of The Listen Lady, a novel about social media research. She can be reached at annie@peanutlabs.com.

COLUMNS M AY 2014 SUR VEY MAGAZINE FEATURED COLUMNIST

J I M B R YS O N

@JimBrysonTN

The Looming Smartphone Research Crisis: Meeting the Challenge Mobile. Mobile. Mobile. Everyone wants mobile research. Mobile access can put research in context like never before. It’s the ultimate in meeting the consumer where they are rather than bringing the customer to where we are.

For years, researchers have begged for more mobile. We researchers see the promise and we love the idea of justin-time feedback. There is a beautiful synergy between researchers and respondents since both groups want to utilize mobile devices more. Even so, mobile research is growing but it has never boomed. Why not? Decipher just released a white paper titled, “Update: Participation of mobile users in online surveys.” The Decipher study shows that survey starts for mobile devices are steadily rising but dropout rates for smartphones are almost twice as high as for desktop or laptop computers. Their data is based on very large samples over 1 million surveys per quarter. Respondents are quickly adopting the use of mobile devices to participate in research. The Decipher study shows mobile device survey starts increased approximately 300% in the past two years from about 7% at the beginning of 2012 to about 20% by the end of 2013. Roughly 2/3 of mobile respondents start the survey with a smartphone (13%) and about 1/3 by tablet (7%).

More mobile research is good news, right? It allows us to get that valuable “in context” response. The problem is that dropout rates for smartphones are twice as high as dropout rates for desktop computers. According to the Decipher study, 76% of those who start a survey on a desktop complete it compared to only 59% who start a study on a smartphone. So, only 3 out of 5 respondents using a smartphone actually complete the survey. This dropout rate would be unacceptable standing alone. Fortunately for researchers, the high dropout rate is masked by the 80% of respondents who still respond to online surveys using a desktop. As researchers, we want mobile because we want in-the-moment respondent access. As researchers, we have been largely unwilling to adjust our methods and expectations to take advantage of the contextual promise that mobile offers.

MORE > MAY 2014 SURVEY MAGAZINE

051

Mobile. Mobile. Mobile. Decipher rightly points out that screen size and LHZL VM KH[H LU[Y` HYL ZPNUPÄJHU[ IHYYPLYZ [V TVIPSL research completion. These problems will become more acute as respondents’ smartphone use grows. Extrapolating Decipher’s data indicates that by the end of 2015, smartphones will account for 20-25% of all survey starts, double today’s rate. At that point, today’s 40% dropout rate will become glaring and unacceptable. We are entering an era when researchers must evolve. We must provide dual interfaces that are optimized for the device respondents choose to use. Screen size, data entry functions and length of Z\Y]L`Z T\Z[ HSS IL HKQ\Z[LK [V Ä[ V\Y YLZWVUKLU[Z» situation, not just our need to know.

JIM BRYSON, FOUNDER & CEO, 20/20 RESEARCH

Researchers who want to get in front of this trend can follow three steps.

How does this relate to qualitative research? )LJH\ZL [OLZL ÄUKPUNZ HSS HWWS` [V X\HSP[H[P]L TH`IL more so. At 20|20, we see smartphone login trends very similar to Decipher’s numbers, even for those studies that are not designed to be “mobile studies.” Problematically, typical online qualitative methods do not translate well to a smartphone’s small screen. Text entry and image evaluation are simply much TVYL KPMÄJ\S[ HZ PZ IYV^ZPUN H SHYNL KH[H ÄSL Z\JO HZ a bulletin board conversation. As researchers, we need to understand and appreciate that smartphones are becoming our respondents’ device of choice and we need to design to the trend. The good news is that researchers are doing just that. In the past year, 20|20 has seen a 127% increase in our mobile journal platform (LifeNotes) LU[YPLZ 3PML5V[LZ PZ KLZPNULK ZWLJPÄJHSS` [V IL smartphone friendly. Experienced online qualitative researchers are learning that pictures and video really are worth 1000 words, particularly from a mobile device where those pictures and video provide context for a more nuanced understanding. Researchers are also learning how to squeeze value out of shorter mobile text or audio responses that may be short on words but long on immediacy and context.

052 SURVEY MAGAZINE

MAY 2014

Realize that the world is changing. Don’t resist, respond.

Optimize research for the platforms respondents will use. Provide options for both desktops/laptops and for TVIPSL ZWLJPÄJHSS` smartphones.

Learn how to analyze and interpret the data best collected by smartphones: pic[\YLZ ]PKLV ZOVY[ [L_[ ÄLSKZ and audio

Technology feasibility is about tipping points. In the next two years, respondent mobile access to research will reach a tipping point that researchers will not be able to ignore. The question is, “Will research adapt their methods and expectations or will they ÄUK [OLTZLS]LZ ^P[O \UHJJLW[able dropout rates and frustrated respondents that leaves everyone wanting more?”

www.employeetalk.us P. Â 510-Â397-Â7929 Â Info@employeetalk.us Â

FREE Â OPEN Â ACCESS Time Â Limited Â Offer Â from Â EmployeeTalk

FREE Â open Â access Â to Â experience Â their Â online Â cloud-Âbased Â communication Â engine Â that Â is Â structured Â to Â engage Â stakeholders Â input Â and Â ideas Â for Â better Â

business Â outcomes.

Our Â unique, Â unparalleled Â Pulse Â Communication Â methodology Â empowers Â organizations Â to Â increase Â the Â quality Â of Â internal Â $ ' ! ! & communication Â by Â aligning Â targeted Â dialog Â to Â support Â more Â informed Â decisions. ! # ! % & !, ! # " ! "! ! " !" ! & ! "! * ! # ! # ! ! " ! " &* EmployeeTalk Â pledges Â to Â help Â you Â determine Â how Â to Â use Â the Â value Â of Â feedback Â by Â & " ! & "! ) !" ! $ ! ! ! * " aligning Â consistent Â open Â dialog Â across Â your Â organization. Â $ $ & " ! " # & " ! $ ! " .0/ ! ! # ) $ !) ! ! " ! !& ! * EmployeeTalk Â collates Â and Â delivers Â dynamic Â and Â actionable Â reports. Â Our Â unique Â task-Âbased Â gateway Â allows Â you Â to Â delegate Â

actions Â immediately Â to Â a Â team Â of Â internal Â advisors Â for Â your Â self-Âconsulting Â success, Â all Â within Â one Â framework. Â Â Â

Use Â&#x2021;

! Â&#x2021; Â&#x2021; ! Â&#x2021;

& # $ & " !! " $ % made simple: "& " ! Case examples EmployeeTalkâ&#x20AC;&#x2122;s Free offer Includes: " ! " ! " Unlimited Â pulse Â check Â communications Â&#x2021; Unlimited Â access Â to Â EmployeeTalk Â capabilities " $ # " 360 Â peer Â review Â to Â create Â ownership Â of Â development Â&#x2021; Personal Â implementation Â support Â Reward Â program Â structured Â for Â better Â team Â recognition ' " " "& Â&#x2021; Register Â for Â your Â free Â open Â access Â to Â EmployeeTalk Â Now! Policy Â and Â procedure Â sign-Âoff Â for Â better Â execution Â and Â $ " " $ " assurance Â " ! # "( "% !

B2B

Sample Tradeoffs

and the Power of a Multi-Mode Approach Authored by: Jackie Lorch, Vice President, Global Knowledge Management , SSI

054 SURVEY MAGAZINE

MAY 2014

01 03

The targets are almost always very low incidence within the general population

Many of the most desirable targets are hard to reach â&#x20AC;&#x201C; their time is scarce and they may be protected by gatekeepers who guard access to them.

B2B

;OL YLZWVUKLU[ PZ YLĂ&#x2026;LJ[PUN not only their profession, such as IT developer, but also their company (large corporation or startup, for example) in their answers

B2B

These differences all impact the way we build and manage a quality B2B sample.

Business-to-business research is different from consumer research. That may seem like an obvious point to make, but because consumer research is the predominant research type, many researchers go through their entire careers without doing much if any business-to-business (B2B) research. When faced with a B2B project, we make a mistake in approaching it in the same way as a consumer study.

MAY 2014 SURVEY MAGAZINE

055

0TWHJ[ VM SV^ PUJPKLUJL VU X\HSP[`

B2B 100 interviews at

100%

incidence

( YLHS [OYLH[ JHSSZ MVY

powerful ^LHWVUZ

The low incidence nature of B2B research makes B2B sample susceptible to people getting into a study that they shouldnâ&#x20AC;&#x2122;t be in â&#x20AC;&#x201C; either because of fraud or because genuine respondents sometimes mis-click the answer on a screening question in error. True frauds are a tiny percentage on a well-managed online WHULS I\[ [OLPY PTWHJ[ JHU IL THNUPĂ&#x201E;LK ^OLU the incidence is low.

Here is an example to illustrate that. Imagine that 1% of the panel is fraudulent and they will tend to over-qualify, because they lie. In a project that needs 100 interviews at 100% incidence, of the 100 interviews one will be a fraud. This is unfortunate but it doesnâ&#x20AC;&#x2122;t affect the data. Now assume that the project requires 100 interviews at 5% incidence. This means 2,000 people have to start the survey. Of these 2000, 20 will be frauds (that is 1%) and they lie to qualify. This means that now 20 of the 100 interviews are fraudulent. The data is really at risk now. If the incidence rate is lowered yet further, to 2% for example, 5,000 must start the survey, of which 50 (1%) are frauds, and so on. As the incidence drops, the impact becomes much more serious.

Survey Fraud?

Because the threat of fraud is real, quality controls must be powerful. This requires not only a change in the survey design to make it less obvious how to qualify, but also a change in how business sample is managed. Just combining sources without owning the personal information and relationship with the panelist wonâ&#x20AC;&#x2122;t deliver the reliability and fraud protection weâ&#x20AC;&#x2122;re looking for because we canâ&#x20AC;&#x2122;t implement the many additional layers of protection we need. Recruiting from loyalty programs and communities of interest with their own built-in quality controls is a good foundation, but sample providers can go several steps further when they have the personal information of the respondent and own the relationship by using:

Â&#x2039; + Â&#x2039; + Â&#x2039; +

Â&#x2039; +

056 SURVEY MAGAZINE

MAY 2014

Â %1

+ -Â-Â

+PNP[HS Ă&#x201E;UNLYWYPU[PUN +H[H TPUPUN VM WYVĂ&#x201E;SPUN KH[H ZV MVY L_HTWSL ZVTLVUL ^OV OHZ [^V JVTW\[LYZ HTVUN LTWSV`LLZ TPNO[ [OYV^ \W H YLK Ă&#x2026;HN Matching panel join data with information our panelists have shared on social media like LinkedIn (with their permission of course) (UK [LSLWOVUL ]LYPĂ&#x201E;JH[PVU [V LUZ\YL [OH[ WLVWSL HYL LTWSV`LK where they say they are

The B2B Update Challenge

With an online panel model, we can monitor performance through the respondent’s lifetime on the panel and include things like 72-hour reward claim delays which are used in banking. This data is invaluable in taking away the incentive for cheating and providing an incentive to stay in the panel and continue to get rewarded for good behavior. SURVEY

tip

There is a great deal we can do to ensure quality within the questionnaire itself, beginning with the screener section. For example, are we limited by certain job titles, or is it really decision-making responsibility we are after?

To turn to the second difference between consumer and B2B research, because the respondent is representing not just themselves, but their company, it’s essential that we have HJJ\YH[L ]LYPÄLK PUMVYTH[PVU HIV\[ our respondents and where they work, and that we keep that information up to date, which presents a real challenge. Someone on a consumer panel who plays golf will always be of interest for a golf survey. But a B2B respondent speaks not just for their own area of expertise, like IT development, but also speaks about that role on behalf VM [OL ZWLJPÄJ JVTWHU` [OL` ^VYR MVY If that IT developer switches from a small startup to a large corporation for example, their opinions may no longer be of interest.

“

A B2B respondent speaks not just for their own area of expertise...

“

MAY 2014 SURVEY MAGAZINE

057

<ZPUN [LSLWOVUL ]LYPÄJH[PVU [V JVUÄYT [OL JVTWHU` VM LTWSV`TLU[ provides multiple opportunities to encourage members to keep their information up to date. It’s also important to educate members on the ILULÄ[Z VM KVPUN ZV PU [OL MVYT VM HKKP[PVUHS Z\Y]L` VWWVY[\UP[PLZ

Multi-Mode Methodology: ;YHKLVMMZ HUK )LULÄ[Z

Combining the best of online and telephone research methodology is an attractive solution to combat the particular challenges presented by B2B research: the low-incidence populations and the relative scarcity of B2B respondents’ time. When comparing the two methodologies, there are tradeoffs: The advantage of telephone methodology is direct access to the person or role we’re looking for. If we have a list of companies, there is usually an HR person to be found there. However, business research done on the telephone is usually conducted during the day while the respondent is at work. The targets we need like

B2BB 058 SURVEY MAGAZINE

MAY 2014

C-level executives or medical professionals, have little time to spare in their workday. Online, the respondent can stop and start the interview whenever they want, they can start it at work and complete it at home, or the other way round. Today’s work sched\SLZ HYL ]LY` ÅL_PISL ¶ [OLYL PZ UV SVUNLY H N\HYHU[LL [OH[ ^VYR OV\YZ HYL UPUL [V Ä]L VY LPNO[ [V ZP_ PU [OL VMÄJL" ^P[O VUSPUL ^L KVU»[ have to guess when someone will be there to HUZ^LY V\Y JHSS PU [OL VMÄJL (UK [OL OVTL environment may be a more natural setting to answer some of the questions we pose, such as reactions to advertising, or attitude questions related to the work environment.

There are further tradeoffs when considering representation of the universe. A major difference between online and telephone sample is that an online sample is based on the individual as the sample unit, whereas in the telephone approach, companies are our basic sampling unit. With the company-as-sample-unit approach, there’s a bias towards smaller companies. For example, one company may have 500 IT developers, another only two. With the business as our starting point, developers working at smaller companies are over-represented.

Telephone and the coverage advantage According to the Bureau of Labor Statistics 59% of Americans are employed so could potentially be a B2B research target. But the truth is that the vast majority of working people are not of great interest to B2B researchers. Most researchers want to speak to c-level executives, or people who work in IT or have decision making power in certain segments like HR, or travel or resource procurement. A traditional consumer online panel, however large, may struggle to support very narrow targets. A telephone sample can reach many more people within a very narrow target than even a very large online panel can.

How a multi-mode solution works ( T\S[P TVKL HWWYVHJO SL]LYHNPUN [OL ILULÄ[Z VM IV[O [LSLWOVUL HUK VUSPUL JHU IL H WV^LYM\S VW[PVU [V ÄUK [OL YHYL [HYNL[Z ^L HYL SVVRPUN MVY HUK LUZ\YL ^L HYL LUNHNPUN [OLT ^OLU ^OLYL HUK OV^ [OL` ÄUK TVZ[ JVU]LUPLU[ 0U [OL [YHKLVMM IL[^LLU JVUZPZ[LUJ` VM ZHTWSL ZV\YJPUN HUK [OL ability to reach rare targets reliably – speaking to the target we need is usually more important. A phone invitation, using interviewers who are skilled in bypassing gatekeepers, followed by either a live telephone interview, or an invitation to complete a survey online at their leisure, can provide the best of both worlds in achieving the best coverage and the best respondent experience. This approach can make challenging projects feasible.

B2B

Sample Tradeoffs

MAY 2014 SURVEY MAGAZINE

059

Sample

Tradeoffs

B2B

Best practices for successful B2B

research

Best Practices When choosing a B2B sample provider, it’s PTWVY[HU[ [V LUZ\YL [OH[ [OL` [V [LZ[ ]LYPÄJH[PVU [LJOUPX\LZ KLÄUL ILZ[ WYHJ[PJLZ HUK I\PSK [OLT into their solutions. In conclusion, these are the six key topics that should be part of any conversation with a B2B sample provider.

Source of the Sample Understand the source of the sample you’re using. How diverse are the sources, and is the relationship owned, allowing for enough quality controls?

=LYPÄJH[PVU *VU[YVSZ ,UZ\YL T\S[PWSL X\HSP[` HUK ]LYPÄJH[PVU controls are in place. Understanding the ZV\YJL PZ PTWVY[HU[ I\[ PZ UV[ Z\MÄJPLU[" we are also using techniques like teleWOVUL ]LYPÄJH[PVU ZVJPHS TLKPH TH[JOPUN and data mining of respondent information to minimize any risk of fraud or poor quality response.

060 SURVEY MAGAZINE

MAY 2014

SURVEY

tip

03 Quality Controls Make sure quality controls such as claim delay mechanisms exist at the point of reward redemption, to remove the incentive for fraud.

04 Screening Questions Review the screening questions used to bring people to your study. Screening questions must be very precise to corYLJ[S` KLÄUL [OL WVW\SH[PVU TPUPTPaPUN YPZR of either exclusion bias (excluding people who should be in the sample) or interest bias (revealing the topic of the questionnaire and over-including people with a particular interest in the topic).

% inc

%100 cidence

“

Screening questions must be very precise

“

The Power of a Multi-Mode

Approach

Authored by: Jackie Lorch, Vice President, Global Knowledge Management , SSI

05 +LÄUL @V\Y ;HYNL[ 4HRL Z\YL `V\ JVYYLJ[S` KLÄUL `V\Y [HYNL[ +V `V\ YLHSS` ULLK H ZWLJPÄJ [P[SL VY HYL `V\ SVVRPUN [V ÄUK WLVWSL ^OV OH]L certain responsibilities? Make sure your sample provider clearly understands who you’re looking for.

06 Multi-Mode Approach Finally, consider a multi-mode approach, LZWLJPHSS` MVY KPMÄJ\S[ [HYNL[Z >L OH]L experience the advantages of both live interviewer telephone online – there are many options for combining the two to create just the right solution for your speJPÄJ ) ) WYVQLJ[

When business people are engaged in ^H`Z [OH[ Ä[ [OLPY I\Z` SP]LZ [YLH[LK [V well- designed surveys and rewarded in ways that are meaningful to them, they can be among the most eager and engaged of survey respondents.

B2B

Sample Tradeoffs

MAY 2014 SURVEY MAGAZINE

061

North America

Market Research Mobile World in the

May 27-30, 2014

CHICAGO

Learn, network and interact with senior-level executives from YAHOO, FACEBOOK, KAO, SHELL, MERCK, GOOGLE, QANTAS, AOL, GENERAL MILLS, GIVAUDAN, BANK OF AMERICA AND MANY MORE! REGISTER AT

MRMW.NET

Survey Magazine Subscribers benefit from 15% OFF regular rates Use code: SURVEY15 Organized by

Silver Sponsors

Title Sponsor

Gold Sponsors

Premier Sponsor

Networking Reception Sponsor

062 SURVEY MAGAZINE

MAY 2014

Association & Media Partners

SUBSCRIBE NOW! SUBSCRIBE NOW!

Get your annual subscription to SURVEY Magazine. .L[ `V\Y HUU\HS Z\IZJYPW[PVU [V Your annual subscription includes ten (10) exciting print issues of SURVEY :<9=,@ 4HNHaPUL Magazine and full online access to the @V\Y HUU\HS Z\IZJYPW[PVU PUJS\KLZ [LU L_JP[PUN magazine, special offers, discounts to WYPU[ PZZ\LZ VM :<9=,@ 4HNHaPUL HUK M\SS VUSPUL events and the Market Research Bulletin HJJLZZ [V [OL THNHaPUL ZWLJPHS VMMLYZ KPZJV\U[Z [V L]LU[Z HUK [OL 4HYRL[ 9LZLHYJO )\SSL[PU e-Newsletter. L 5L^ZSL[[LY

Subscribe today for only $10.40/month :\IZJYPIL [VKH` MVY VUS` TVU[O HUK LUQV` [OL TVZ[ [Y\Z[LK THNHaPUL PU YLZLHYJO and enjoy the most trusted magazine MLH[\YPUN PUK\Z[Y` SLHKPUN JVS\TUPZ[ HUK OPNOSPNO[Z in research, featuring industry leading VU THYRL[PUN Z[YH[LNPLZ Z\Y]L` X\LZ[PVUUHPYLZ columnist and highlights on marketing THYRL[ HUHS`ZPZ Z\Y]L` [LJOUPX\LZ HUK strategies, survey questionnaires, market TL[OVKVSVNPLZ MVY YLZLHYJO analysis, survey techniques and methodologies for research.

To subscribe, call +1 484.478.2774 or visit surveymagazine.org/subscribe ;V Z\IZJYPIL JHSS +1 484.478.2774 VY ]PZP[ surveymagazine.org/subscribe

S URVEY

Company Showcase

NEWS AFFECTIVA-20|20 PARTNERSHIP TAKES QUALITATIVE FACIAL CODING GLOBAL Research technology collaboration to support and distribute Affdex Discovery™ in virtually any location

Weeks after Waltham, Mass. technology company Affectiva launched its breakthrough facial coding software for qualitative research -- Affdex Discovery™ -- the company has partnered with U.S.-based market research technology and services ÄYT c 9LZLHYJO [V Z\WWVY[ HUK distribute the technology for face-toface qualitative research in facilities and other locations where research is done. 20|20 will use its technologies, training and executional support to HPK YLZLHYJOLYZ HUK YLZLHYJO ÄYTZ in using the new consumer emotion insights product anywhere the research method is needed. “Because of our global experience delivering online technology applications, we’re able to help researchers implement Affdex Discovery virtually anywhere,” said Jim Bryson, founder and CEO of 20|20 Research. “Our expertise in technology, training and side-by-side support will be invaluable to researchers who want to 064 SURVEY MAGAZINE

MAY 2014

harness facial coding for deeper consumer insight in ad testing efforts... without executional headaches.” Facial coding brings behavioral analysis to advertising, video and media testing so researchers can more thoroughly understand how ads impact consumers emotionally. “Respondents usually struggle to communicate their emotional reactions to advertising,” Bryson explained. “With Affdex Discovery, study participants watch streaming media (ads, trailers, videos, etc) on a computer while their facial expressions are captured via the computer’s webcam. This emotional feedback is gathered in real time on dashboards to moderators, allowing them to immediately and more accurately probe the meaning behind the reaction and uncover raw emotion insights. That knowledge then leads marketers to IL[[LY YLÄUL WYVK\J[ WVZP[PVUPUN VY messaging.”

The association between the two companies optimizes technology from each, as well as 20|20’s extensive capabilities in moderator and facility training, stimuli management and round-the-clock technical support. “We’re excited about this partnership because of 20|20’s proven ability to deliver and support technology solutions,” said Nick Langeveld, president and CEO, Affectiva. ”Their tech support can take Affdex Discovery where the research needs to be done, so researchers are not tethered to a ZWLJPÄJ SVJH[PVU ,]LU IL[[LY [OLPY ^VYRÅV^ WYVJLZZ HUK [YHPUPUN JHU help any researcher dramatically improve implementation, yielding more accurate, cutting-edge wisdom for their clients.”

QUALITATIVE SHOWCASE

FACIAL CODING

Fast delivery, real-time results

HOW AFFECTIVA AND 20|20 RESEARCH MADE FACIAL CODING WORK FOR QUALITATIVE RESEARCHERS

Problem

Solution

A relatively new technology for market research, facial coding brings behavioral analysis to advertising, helping researchers and marketers better understand how messages emotionally impact consumers. Affectiva has long been the global leader in the quantitative use of facial coding, successfully testing thousands of video ads from brands such as Coke, Kellogg’s, Unilever with millions of survey takers over the past few years. Recognizing that self-reported bias extends beyond quantitative research, Affectiva needed to develop their platform to uncover objective key emotion insights for qualitative. Additionally, the processes and ^VYRÅV^Z VM H [`WPJHS X\HSP[H[P]L YLZLHYJO WYVQLJ[ HYL ]HZ[S` KPMMLYLU[ [OHU those found in quantitative. The more individual nature of qualitative meant Affectiva needed to re-factor their reporting console for a oneon-one real-time view. Moderators, on-site facility staff and even clients would require hands-on training and support, none of which had ever been developed for qualitative research.

Over the course of dozens of test groups HUK PU[LY]PL^Z c HUK (MMLJ[P]H ÄLSKLK a number of product design studies to test the boundaries of how Affdex Discovery™ should be structured and how different participant stimuli and processes might increase insight effectiveness. 20|20 provided the venue, participants and moderator access while Affectiva’s technical team observed the real-time use of their prototype product. The two companies tested several approaches to study design, video content length and inMHJPSP[` ^VYRÅV^ [V JYLH[L H [LZ[LK ¸YLZLHYJO friendly” framework for Affdex facial coding in qualitative research.

S U R V E Y M A G A Z I N E : C O M PA N Y S H O W C A S E

When global consumer emotions analytics and insights expert Affectiva wanted to extend their product offerings into the world of qualitative research, they formed a strategic partnership with 20|20, the leader in digital qualitative methodologies, to KL]LSVW YLÄUL HUK THYRL[ H NSVIHSS` H]HPSHISL X\HSP[H[P]L MHJPHS JVKPUN WSH[MVYT

M AY 2 0 1 4

CASE STUDY

The Affdex Discovery™ intuitive moderator solution delivers key consumer emotion insights, in realtime! As participants watch online, streaming media content, their facial expressions are captured via computer webcam. The cloud-based emotionanalytic platform then processes the emotional feedback and reports the ÄUKPUNZ IHJR [V [OL TVKLYH[VY ¶ initiating further discussions with individual participants.

Result Based on the pilot research, Affectiva made minor enhancements to their Affdex Discovery platform. The effort also allowed c [V KL]LSVW H JVTWYLOLUZP]L ^VYRÅV^ process that would help facilities and moderators more effectively implement Affdex Discovery. Affdex Discovery will provide moderators information-rich dashboards that will identify opportunities to probe respondent reactions and better understand emotions to stimuli, such as TV commercials, videos, movie trailers, etc

MAY 2014 SURVEY MAGAZINE

065

S URVEY

Company Showcase

Moderator Dashboard Qualitative Facial Coding

Moderator Training

Said one moderator:

20|20 additionally developed a training program to [LHJO TVKLYH[VYZ HUK YLZLHYJO ÄYTZ OV^ [V ILZ[ \[PSPaL the Affdex Discovery reporting console during a live interview. 20|20’s training covers simple topics such as discussion guide development, tips and tricks for probing on the emotion “tracelines” and more complex topPJZ Z\JO HZ KLHSPUN ^P[O ¸ÅH[SPULYZ¹ HUK ¸J\Y]L I\Z[LYZ¹

“I felt [Discovery] was very easy to use and provided a new layer of data that allowed me to dive more into particular moments that the individual couldn’t recollect. In a typical focus group you’re limited to what the participants say and you can’t really dispute what they say without any factual data to refer to.”

Product Roll-Out

Another observation:

;OL [^V ÄYTZ HSZV KLJPKLK [V WHY[ULY VU WYVK\J[ YVSSV\[ to take advantage of their mutual strengths in launching the insight tool. This partnership today allows a market researcher to utilize the Affdex Discovery™ technology in any facility worldwide and receive training, support and design instruction from 20|20’s product support teams. Any researcher can now leverage powerful facial coding technology for qualitative video testing, with the support and guidance of Affectiva and 20|20.

“Qual gets biased rather quickly so having a roadmap helps act as a ‘framework for conversation’ and provides end clients a point of reference while they’re viewing the qual session in progress.”

066 SURVEY MAGAZINE

MAY 2014

2000 Glen Echo Road Second Floor Nashville, Tennessee 37215 Phone: 615.777.2020 | U.S Toll Free: 800.737.2020 Fax: 615.777.8706 | 2020research.com

NEWS

MARCH 2014 SURVEY MAGAZINE: MARKET RESEARCH INDUSTRY NEWS

Top source for market research news, research topics, research methods, and the resources driving market research best practices.

USAMP UNVEILS NEW SUITE OF IN-CONTEXT MOBILE SOLUTIONS

Marketing Workshop Announces New Team Member Addition Tony Bonn joined the company as a Senior Lead Developer.

Focus Pointe Global launches mobile-friendly respondent website The new focusgroup.com has been designed to provide the ultimate user-friendly experience with improved navigation and functionality throughout, allowing users to access available research opportunities on the go.

TWITTER: @SURVEYMAGAZINE

PINTEREST: PINTEREST/SURVEYMAGAZINE

www.MarketResearchBulletin.com !!

FACEBOOK: FACEBOOK/SURVEYMAGAZINE

MAY 2014 SURVEY MAGAZINE

067

[ DOWNLOAD ] Forrester Report: The Customer Experience Index, 2014. Ever wonder how your customer experience compares to your competitors or similar industries? Now you can see how you stack up in “The Customer Experience Index, 2014 from Forrester Research, Inc. We thought you might like this report because it provides you with case studies and recommendations to beat the CX score of your competitors and be above average in your industry. Download this report to:

Decipher Adds MSG’s PRO-T-S CATI Dialer Functionality to its Beacon Research Platform Computer Aided Telephone Interviewing automated dialing is now integrated into the Beacon market research and reporting platform. Decipher, a comprehensive market research ser]PJLZ ÄYT [VKH` HUUV\UJLK [OH[ P[ OHZ PU[LNYH[LK Marketing Systems Group’s CATI (Computer Aided Telephone Interviewing) dialer application (PRO-T-S) to its industry leading survey and reporting platform, Beacon. This newest integration allows telephone PU[LY]PL^LYZ H[ ÄYTZ SPJLUZPUN [OL 796 ; : KPHSLY Z`Z[LT [V JVUK\J[ *(;0 YLZLHYJO TVYL LMÄJPLU[S` through advanced dialing and sample management functionality. 4:. WYLZPKLU[ 1LYY` 6ILYRVÅLY ZHPK ¸4HYRL[PUN Systems Group is excited about the integration of the PRO-T-S dialer with Decipher’s Beacon data collection and reporting platform. The combination of our ILZ[ PU JSHZZ ZVS\[PVUZ VMMLYZ UL^ LMÄJPLUJPLZ HUK WYVductivity improvements to data collection operations around the globe. We look forward to our partnership with Decipher and delivering another innovative solution to our market research customers.” The PRO-T-S dialing system within Beacon allows project managers access to an automated telephone dialing module, while integrating with Beacon’s existing CATI features. By expanding Beacon’s CATI functionality, Decipher has enabled another fully integrated data collection mode that can be analyzed within the platforms comprehensive reporting module. With its new Research Hub, Beacon serves as a central location where users can view all research initiatives across different data collection methods and unify the research in a single location. 068 SURVEY MAGAZINE

MAY 2014

1. 2. 3.

Understand your company’s competitive environment See if your company or industry has moved up the path to CX maturity Learn how to beat your competitor’s CX score

.L[ [OPZ -VYYLZ[LY YLWVY[ HUK \W `V\Y *? ZJVYL

NEWS MARCH 2014 SURVEY MAGAZINE: MARKET RESEARCH INDUSTRY NEWS

Marketing Workshop Announces New Team Member Addition <: ¶ IHZLK YLZLHYJO HUK JVUZ\S[PUN ÄYT [OL 4HYketing Workshop -MW- has recently announced a new team member addition. Tony Bonn joined the company as a Senior Lead Developer. Tony responsibilities encompasses all web developmental advancement within the organization including new technology service offerings, corporate web presence, company intranets, project planning and architecture of new technology offerings. Tony has over 15 years of professional experience in High Technology, Software Development and Project Management. Most recently, he collaborated with CBS broadcasting and AT&T. “I am excited to become part of MW. I look forward to bringing, supporting and developing innovative technologies in order to facilitate new business, products, and services, in addition to making existPUN ZLY]PJLZ TVYL LMÄJPLU[¹ ZHPK )VUU “Not only is Tony new to the team, his role is new as well. The purpose of this newly created position is to continue our quest to grow while providing scalHISL ÅL_PISL HUK J\Z[VTPaHISL [LJOUVSVN` ZVS\[PVUZ for our clients. We look forward to collaborating on imaginative solutions that will provide unprecedented value. He brings intelligence HUK WYVÄJPLUJ` [V 4> ¹ said Scott Layne, MW President. About The Marketing Workshop: Marketing Workshop is a unique type of market research consultancy. We serve our clients in an advisory role by leveraging our wisdom and imagination while incorporating the use of science and analytical techniques. As a result, we enlighten our clients with researchWISE® business solutions. MW is a strategic market research consultancy that utilizes a fully integrated approach to providing business solutions that are research based and analytically driven. Our work model encourages collaboration, team work and superior client service to empower our clients ^P[O [OL HIPSP[` [V THRL KLJPZPVUZ ^P[O JVUÄKLUJL For more information, visit www.mwshop.com MAY 2014 SURVEY MAGAZINE

069

Critical Mix Strengthens Customer Care Team with New Hire, Brett McCready Critical Mix is pleased to announce the appointment of Brett McCready as Vice President, Account Development. In this role he will be responsible for the development of business strategy, sales distribution, VWLYH[PVUZ Z[HMÄUN HUK 7 3 THUHNLTLU[ The addition of Brett signals Critical Mix’s continued commitment to excellence in customer care and its dedication to simplifying data collection for easier and faster access to global marketing information. Brett comes to Critical Mix with 13+ years of extensive experience in client development and online market research. He is consistently a top individual sales performer and has effectively performed overseas assignments and collaborative efforts between sales and operations teams. “I am excited about the opportunity that exists in the industry and in particular at Critical Mix,” Brett said. “It was an easy decision to join a company that truly cares about their customers and employees, and provides high quality survey data. The proprietary data collection platform that Critical Mix has created, offers the perfect, seamless solution for full-service THYRL[ YLZLHYJO ÄYTZ [V X\PJRS` NH[OLY HUK HUHS`aL survey results. It surpasses everything else I have seen in the industry. I am extremely proud to be a part of the Critical Mix team,” he added. “As we invest and expand our company, we continue to attract the industry’s best and brightest talent,” said Keith Price, Managing Partner. “After having the opportunity to work with Brett in the past at GreenÄLSK 6USPUL 0 YLJVNUPaLK [OH[ )YL[[ ZOHYLZ V\Y WHZsion for excellence and would be a great addition to our team. His proven track record and client development expertise will be leveraged to help strengthen our successful client-centric service model.” About Critical Mix Critical Mix provides easy access to highly-targeted global survey respondents, survey programming and data visualization services for market research and JVUZ\S[PUN ÄYTZ +YP]LU I` H WHZZPVU MVY ZPTWSPM`PUN data collection, the team at Critical Mix is personally invested in giving clients the ultimate customer service experience. Every project, no matter the size or type, is supported by a team of always-available, experienced market research practitioners who anticipate your needs and provide thoughtful customer care. Service options include a complete spectrum of high-quality data collection tools ranging from sample ZV\YJLZ Z\Y]L` WYVNYHTTPUN HUK ÄLSKPUN [V YLWVY[ automation and data dashboards. Critical Mix is headquartered in Westport CT and operates globally.

070 SURVEY MAGAZINE

MAY 2014

iModerate Enhances its Research Suite with Private Boards iModerate, the industry leader in oneon-one research conversations, has expanded its research suite to offer full-service, private bulletin boards to its clients. This new addition to the company’s long-time staple—live, one-on-one conversations—allows [OL ÄYT [V UV^ LUNHNL JVUZ\TLYZ individually in real-time or extended over time. The new research methVKVSVN` HKKZ ÅL_PIPSP[` [V P4VKLYH[L»Z solutions, giving respondents more convenient ways to interact while helping clients tackle a broader range of research objectives. iModerate’s boards are private to keep conversations one-on-one in nature. In contrast to many bulletin board offerings, research respondents will not interact or see each other’s postings, alleviating group bias, and encouraging candor and deeper conversations between our moderators and respondents. iModerate’s boards are powered by the company’s proprietary cognitive framework, ThoughtPath. This qualitative architecture revolves around the cognitive ideas of perception, identity and experience and helps the ÄYT NL[ WHZ[ Z\YMHJL HUZ^LYZ [V OH]L a deeper discussion. The boards will be carried out in-house by iModerate’s team of qualitative specialists. Fueling the decision to offer an asynchronous approach stemmed from a desire by clients to have a more comprehensive, thoughtful solution in the marketplace to engage more diverse audiences, follow the consumer’s journey, and satisfy objectives geared to an extended back-and-forth

conversation. The respondent experiLUJL HSZV WSH`LK H YVSL PU [OL ÄYT»Z expansion into conversations held over time. “Consumers are on the go, communicating on different devices at all times of the day and night,” says Jen Drolet, Managing Partner at iModerate. “By offering asynchronous conversations, we are putting the power of when and how to respond back in their hands for those projects. Keeping the one-on-one interaction is critical for us at this time, but the ÅL_PIPSP[` VM SL[[PUN PUKP]PK\HSZ LUNHNL in this way is a step we embrace.” Despite being an approach conducted over multiple days, online boards nonetheless present a limited opportunity to engage with respondents. Best practices dictate adhering to a few posts a day, so making the most of those opportunities is paramount. The uniqueness of iModerate’s use VM IVHYKZ PZ LTILKKLK PU [OL ÄYT»Z ability to ask the right questions in the right sequence to garner the most relevant insights. The iModerate team has conducted more than 100,000 individual conversations, perfecting its framework for asking questions steeped in consumer psychology. To learn more about iModerate and its one-on-one solutions please visit www.imoderate.com/approach/oneon-ones For more information, please visit www.imoderate.com

MetrixLab Renames MarketTools and CRM Metrix NEWS

AURA Director Harry Mirpuri joins FreshMinds as a Director

MetrixLab acquired Paris-based CRM Metrix in 2010, and San Francisco headquartered MarketTools in 2012 as part of its global expansion strategy. All teams MVJ\Z VU KLSP]LYPUN PUZPNO[Z PU [OL ÄLSKZ VM PUUV]H[PVU brand engagement and customer engagement with a data agnostic approach leveraging survey, social media and enterprise data. CEO Han De Groot says the company will remain locally focused but will now provide expanded access to global expertise and resources. ‘We have continued to proudly use the CRM Metrix and MarketTools names over the past several years because of the long heritage in France and the US respectively. Now it is time to look forward. The proliferation of consumer data PZ KYP]PUN H ZPNUPÄJHU[ JOHUNL PU [OL KL]LSVWTLU[ VM deep, actionable insights, and the speed at which they are delivered. Tomorrow’s complex business decisions require streaming insights derived from many internal and external data sources that become more accessible every day.’ MetrixLab will move from reliance on survey data to integrating ‘always on’ social data, enterprise data from CRM and ERP systems, geospatial data, and data ‘scraped’ from the public web domain such as competitive product and pricing data. ‘Our strategic direction is to become a data source agnostic consumer insights provider through superior data aggregation, integration and analytics,’ adds de Groot. (IV\[ 4L[YP_3HI! MetrixLab is a global provider of analytics and consumer insights. We turn data from surveys, social media and enterprise systems into actionable insights that help leading brands drive innovation, brand engagement and customer retention. Headquartered in 9V[[LYKHT HUK :HU -YHUJPZJV ^P[O VMÄJLZ HJYVZZ the US, Europe and Asia, MetrixLab provides analytics and insights services in more than 50 countries.

Harry joins FreshMinds after 15 years working client-side in a number of prominent insight roles, including Head of Research at RAC and Head of Insight at Which?. Most recently, Harry spent 2 years at BSkyB as Research Director, leading the company’s strategic research to set marketing strategy and the corporate agenda. Harry has also spent over a decade in a leading role within AURA, the prestigious networking organisation for senior insight buyers.

SURVEY MAGAZINE: MARKET RESEARCH INDUSTRY NEWS

MetrixLab, global provider of marketing analytics and research solutions, announced that it has fully integrated US based MarketTools and CRM Metrix in France into its global network. From this moment on HSS VMÄJLZ PU [OL <: ,\YVWL HUK (ZPH ^PSS VWLYH[L under the global MetrixLab brand.

MARCH 2014

Insight consultancy, FreshMinds, continues to attract the industry’s top talent, having appointed Harry Mirpuri as a Director, who until recently served as the Director of AURA.

MORE> www.MarketResearchBulletin.com

Hear it from the experts: Voice of the Customer

As customers increasingly use new technology and interaction channels to engage businesses, the role of Voice of the Customer (VOC) programs has elevated to new heights, and adopting a multi-channel customer engagement strategy is no longer just an option for businesses. Join us for this webinar*, Voice of the Customer: Big Data as a Strategic Advantage, and learn how enterprises deploying more touch-points to capture customer feedback is giving rise to the rapid growth in the volume of structured and unstructured customer data – a trend termed ‘big data.’ Understanding how increasing customer touchpoints results in ever-growing volumes of feedback and operational data for businesses and how the that data is best put into action is critical to an organization’s success. *All participants will receive a copy of the latest Aberdeen Group report, VOC: Big Data as a Strategic Advantage.

MAY 2014 SURVEY MAGAZINE

071

Focus Pointe Global launches mobilefriendly respondent website The new focusgroup.com has been designed to provide the ultimate user-friendly experience with improved navigation and functionality throughout, allowing users to access available research opportunities on the go.

Focus Pointe Global (FPG), a leading provider of marketing research data collection with focus facilities nationwide, today announced the redesign and mobile capabilities of its popular respondent website, focusgroup.com. Now accessible from any web-enabled device, the site connects consumers, wherever they may be, with opportunities to earn extra money by participating in a wide range of paid research studies taking place in their local city or nationally online. Opportunities for participation include focus groups, online communities, webcam interviews, mystery shopping, telephone interviews, and online surveys. In addition to improving the site’s look and feel, FPG has made improvements in infrastructure hardware as well as utilizing cloud-based website accelerators to provide users with speed and security. And, perhaps most importantly, a redesigned and highly secure login interface provides peace-of-mind for consumers concerned about their personal information. FPG-MobileDevice “With close to a million visitors accessing the site on a monthly basis, and a third of those 072 SURVEY MAGAZINE

MAY 2014

coming to us on a hand-held device, the need for a mobile friendly experience was clear. Our panel members can now access research screeners with a simple click on a home page location tile, a Twitter feed, or on an Available Studies dropdown menu. We are already seeing our registration numbers climb,” said Charlotte Daley, Director of Marketing. “With last year’s acquisition of marketing research company, Delve, Focus Pointe Global has not only increased its geographic footprint to 18 locations, it has increased both its client base and with that, the need for fresh participants. With the new mobile-friendly site, we can provide an easy registration and login process for people desiring to participate in our studies no matter where they are,” said Laura Livers, FPG’s President. The launch of the mobilefriendly focugroup.com occurs in conjunction with the rebranding and restructuring of Focus Pointe Global’s online research division, Focus Pointe Online, and FPG’s newly implemented proprietary survey software platform, FPG QualSurvey™.

Is Your Advertising Glass Half Full? Ad Effectiveness Saying Gets An Update Although over a century old, arguably one of the most famous advertising quotes is that by John Wanamaker, a US department store merchant who said “Half my advertising is wasted; I just don’t know which half”. This popular saying seems to strike a chord with our intuitive sense that advertising doesn’t work for everyone all the time. But after 100 years, Vision One Research has undertaken a study to see if this is true, whether things have changed and if advances in our understanding of advertising mean that we can now get a IL[[LY Ä_ VU HK]LY[PZPUN LMMLJtiveness. Over the past couple of years, Vision One have been accumulating data on how consumers react to TV advertising to assess whether or not the 50% rule still holds true today. Using AdProbe, a new proprietary advertising research approach designed for advertising pre-testing and brand tracking studies, it is now possible to understand how people react to advertising in order to measure advertising LMÄJPLUJ` The good news for advertisers, is that consumer reactions to a range of TV ads indicate that the picture isn’t as gloomy as some might have feared – using AdProbe’s overall ‘Impact score’ it was found that only 30% of advertising is actually wasted. This result was taken from consumer reactions to a wide range of TV executions aired from 2013/14 and also highlighted the fact that wastage can vary quite markedly and in some cases can be as little as 10% for the most effective ads. Tony Lewis, Managing Director at Vision One, commented “It is good to know that current TV advertising isn’t as wasteful as some might have us believe.

Most TV advertising appears to be achieving much better results than the 50% effectiveness of the past. These results may also indicate that advances in our understanding are helping advertisers develop more creative and LMÄJPLU[ HWWYVHJOLZ¹ But what is wastage in advertising? Vision One has a unique approach to advertising evaluation born out the fact that brands require different approaches depending on their progress within the brand lifecycle. New or younger brands typically look to generate awareness and highlight their ILULÄ[Z ^OPSZ[ TH[\YL IYHUKZ are looking for ways to reinvent themselves. As such, a one size Ä[Z HSS HWWYVHJO [V HK]LY[PZPUN evaluation is not meaningful. By understanding the range of potential reactions to advertising and the impact it has on people means it is possible to assess how much, or rather how many, are being left unaffected by the advertising. The research concludes that it is also no longer relevant to generalise about advertising performance – the high variation in wastage means that the most effective ads can be up to 4 times, or put another way, 400% less wasteful. The implications of this are clear and suggest that advertisers should ensure their creative ideas and executions are fully optimised with minimal wastage and properly researched before going to market. About the Research Research is based on over 5,000 consumer interviews in the UK reviewing a wide range of advertising executions, including; Food and drink, Household Goods, Public Sector, Retail, Telecoms, and Tourism.

USAMP UNVEILS NEW SUITE OF IN-CONTEXT MOBILE SOLUTIONS

uSamp, the technology leader in providing targeted audiences for global consumer and business insights, today unveiled at the Market Research in the Mobile World conference two mobile research solutions: uSamp Mobile In-Home™ and uSamp Mobile In-Store™. Both WYVK\J[Z VMMLY ÄYZ[ L]LY PU JVU[L_[ HJJLZZ [V consumers on demand and at scale resulting PU NYLH[LY LHZL ZWLLK HUK JVZ[ LMÄJPLUJ` MVY clients. “Mobile has revolutionized the way companies gain insights by granting them access to millions of consumers at the moment of truth when they are interacting with products – whether shopping online or in store,” said Justin Wheeler, VP of Product Innovation & Business Development for uSamp. “The more space there is between the question and the decision, the more rationalized the response. In-context consumers are capable of providing richer detail about their experiences, HUK IL[[LY X\HSPÄLK TLU[HSS` [V HUZ^LYZ X\LZtions. After observing the most successful use cases for our mobile product, we think these moIPSL VMMLYPUN ^PSS WYV]PKL JSPLU[Z ^P[O TVYL LMÄJPLU[ analysis and highly actionable results.”

big-box retailer wanted to determine how path^H`Z HUK WYVK\J[ WSHJLTLU[ PUÅ\LUJLK JVUsumer decisions about its laundry detergent. The CPG client comments that mobile has helped the team “understand what’s in consumers’ cupboards, see how they’re using our products, and capture videos in which they introduce us to their lives.” According to a 2014 study, 41 percent of clients and suppliers are already using mobile as a mode for research, and 41 percent are considerPUN P[ \:HTW»Z ÅL_PISL TVIPSL VMMLYPUNZ [HRL HKvantage of on-the-go accessibility and can easily capture hard-to-reach demographic groups, e.g.,millennials. About uSamp uSamp is the technology leader in market research providing automation, mobile and global sample and insight solutions for marketLYZ )HZLK PU 3VZ (UNLSLZ ^P[O Ä]L VMÄJLZ PU the United States, Europe and Asia, uSamp has been recognized in Inc.’s 500|5000 exclusive ranking of the nation’s fastest-growing private companies. For more information visit www.usamp.com.

uSamp’s mobile offerings provide its customers, including Fortune 200 CPG companies, with unprecedented access to consumer feedback through their highly rated mobile app, in which businesses are able to document the pivotal moments, or the “real-why,” that led up to customer’s purchasing decisions. For example, a leading food manufacturer leveraged the in-home mobile offering for a survey about refrigerated fresh dough products. Another

MAY 2014 SURVEY MAGAZINE

073

-,(;<9,+ :<7730,9:

*P]PJVT 4HYRL[PUN 9LZLHYJO :LY]PJLZ Address: Telephone: Email: Web:

P.O. Box 4689, Greenwich, CT 06831 1-203-618-1811 inquire@civi.com www.civi.com/marketingresearch

::0 :\Y]L` :HTWSPUN 0U[LYUH[PVUHS Address: Telephone: Email: Web:

6 Research Drive, Shelton, CT 06824 (203)567-7200 info@surveysampling.com www.surveysampling.com

7YLJPZPVU 6WPUPVU Address: Telephone: Email: Web:

101 Convention Center Dr., LV, NV 89109 702-483-4000 info@precisionopinion.com www.PrecisionOpinion.com

) ) 0U[LYUH[PVUHS Address: Telephone: Email: Web:

707 Westchester Avenue, White Plains, NY 10604 +1 914-761-1909 info@b2binternational.com www.b2binternationalusa.com

\:HTW Address: Telephone: Email: Web:

16501 Ventura Blvd, Suite 300 Encino, CA, 91436 877-217-9800 info@usamp.com www.usamp.com

;VS\UH Address: Telephone: Email: Web:

21 River Road Wilton, CT 06897 +1 203 834 8585 info@toluna-group.com www.toluna-group.com

DIRECTORY M AY 2 0 1 4 SURVEY MAGAZINE: MARKET RESEARCH SUPPLIER DIRECTORY

8\HS[YPJZ Address: Telephone: Email: Web:

Qualtrics 2250 N. University Ave. Provo, UT 84604 1-800-340-9194 sales@qualtrics.com www.qualtrics.com

5\HUJL *VKPUN Address: Telephone: Email: Web:

520 Avenue H East, Arlington, TX 76011-3100 1-817-640-6170 Lhazen@nuancecoding.com www.nuancecoding.com

(. *VUZ\S[PUN Address: Telephone: Email: Web:

Av.Desembargador, Brasil CEP 88040-400 55 48 32665927 geisa@ag3consulting.com.br www.ag3consulting.com

@V\.V] Address: Telephone: Email: Web:

111 W. 24th Street, New York, NY 10011 +44 (0)20 7012 6000 info@yougov.com www.yougov.com

)\YRL 0UZ[P[\[L Address: Telephone: Email: Web:

500 West 7th Street Cincinnati, OH 45203 800.543.8635 register@BurkeInstitute.com www.burkeinstitute.com

)\YRL 0UJ Address: Telephone: Email: Web:

500 West 7th Street Cincinnati, OH 45203 800.688.2674 info@burke.com www.burke.com

MAY 2014 SURVEY MAGAZINE

075

Market Research Directory The Market Research Directory (www.MarketResearchDirectory.org) connects marketing research buyers and suppliers through online ZVS\[PVUZ [OH[ THRL P[ LHZPLY [V Ă&#x201E;UK HUK KV I\ZPULZZ PU [OL THYRL[ research industry.

Find Your Next Research Partner

FROM ANYWHERE.