Game on qualitative researchers

We believe gamification can be applied in 3 different phases of the

research process; (1) during data collection, (2) during analysis and

What to expect

interpretation and (3) during reporting and presentation of the results. In this paper, we present an approach to gamification

in online qualitative research. There is already ample research with respect to using gamification in quantitative research; however,

a comprehensive approach for online qualitative research is lacking so far.

In this paper we will focus on using gamification during data

collection and will briefly demonstrate how we apply gamification in the last 2 phases. At InSites Consulting, we identified 4 levels in an online community at which

gamification can be applied to increase data quality, participant engagement and impact on the client side. From a question level to a community level, gamification helps, not only to increase participant engagement, but also to increase data quality.

Introduction

In the last decade, we saw a tremendous shift in engagement with brands. Consumers are

looking for brands and companies who engage with them and offer them an experience. Brands and companies which do not

Gen Y is used to constant stimulation and having a say in everything. In order to engage with them, brands need to fulfill these expectations (Van den Bergh and Behrer, 2011 for an elaborate discussion on the characteristics of Generation Y). Finally, the last decade has been marked by disruptive changes in

interact or offer experiences to their consumers tend to fade away, while brands that are inclusive tend to flourish. This has already been showcased plenty of times in the field of marketing and marketers have reacted, by changing their

technology, with the rise of social media and mobile technology evolving at an exponential rate. Everybody is now connected to everybody 24/7. People have become used to an on-demand

marketing plans by including more engaging

lifestyle where everything is available at their fingertips anytime.

marketing actions (Van Belleghem, 2010; 2012).

However, the lower consumer engagement problem has also hit the market research industry. Response rates only go down while straight lining and quickly replying to a question become more and more prevalent. In addition, we

are now also facing a demanding and techsavvy new generation we want to do research with.

The problem of decreasing engagement has been troubling researchers in the industry for a while now and money is not the issue, as increasing monetary

Clearly, market research needs to adopt its

incentives has not been able to solve this problem.

The real problem lies within market research itself and what we offer participants. We know

solution for the problem of waning engagement, some researchers have suggested adopting gamification in market research. The reason behind this is that

these people like to be in an interactive environment, expect anything at their fingertips 24/7 and want to

games are very effective at engaging people with a certain topic or task. Playing games is

feel empowered. However, our offer towards them

popular with everybody, with gaming being popular in different age groups and both for men and women.

36% of gamers are over 36 and 45% of gamers are female (ESA, 2013). However, we believe

consists mainly of surveys which force them to follow a certain question flow. These questions are repetitive, not cognitively stimulating and can be extremely long. With focus groups, people feel more empowered and the questions are normally more cognitively stimulating. However, here also,

participants don’t have the opportunity to interact with the brand or feel like they have a big impact on the brand. In addition, it is not in line with the 24/7 availability to which they have grown accustomed.

offering to participants and, by extension, clients, in order to keep engagement high. As a

gamification is only part of the solution, as it is not sufficient to solve the engagement problem.

To structurally solve the engagement problem, market research needs to empower participants, create a two-way dialogue with them and offer them a stimulating (visually appealing & cognitively stimulating) and 24/7 environment where they can freely speak their mind. Given the broad scope of this, we will focus in this paper on increasing

engagement via gamification.

Definition

Gamification is not the same as turning research into a game. We at InSites Consulting use the following definition from Deterding et al. (2011)

“Gamification is the use of game design elements in non-game contexts.”

The goal of gamification is to improve the data

quality by applying game elements in research, thus motivating participants to participate longer and provide richer answers.

Game elements or game mechanics, as explained

by Donato and Link (2013) are actions, tactics or mechanisms used to create an engaging and compelling experience for consumers. For example, by including levels or using challenges, participants can be stimulated to engage more with the research at hand. These game mechanics work because they tap into game dynamics,

which refers to the motivations people have to engage with a game: (1) achievement, (2) social and (3) immersion (Yee, 2006).

Achievement relates to wanting to become better at something, analyzing and solving problems and outperforming others. Social motivations to engage with a game relate to meeting and interacting

with new people, making friends, building relationships and contributing to team efforts. Finally, immersion relates to discovering new things, taking on new roles and a form of escapism.

Gamification is not a monolithic concept; it rather is a

Secondly, gamification needs to support the

spectrum with different shades of gamification.

On one hand, we identify gamification of research.

research and promote the desired behavior (e.g.

Jon Pulleston is a pioneer in this type of gamification, with a plethora of research into the gamification of surveys. On the other extreme of the spectrum we

answer creatively, give a spontaneous reaction, etc.). It does not have the purpose to entertain people. By tapping into game motivations, the research experience improves for the participant and better data is obtained.

Adamou drew the blueprints for this type of

An improved experience for the participant is a means, but not an end goal of gamification. It is

gamification. Here, the research is embedded in a

important to ask yourself whether the game mechanics

game environment, immersing participants in a game which also includes research questions

you want to introduce will enhance your data or not.

Andy Barker, Engage Research and Lisa Hunt, Heinz (2012) were among thefirstt o explore gamification in offline qualitative research. They operationalized

identify “researchification” of games. Betty

(Adamou, 2011). Although these represent different shades of gamification, both methods help to create more engagement and collect better data. Before applying gamification to your research, there are two important caveats which need to be taken into account.

Firstly, applying gamification can be a costly

endeavor. It costs time and money to implement it and it may not be worthwhile to invest a lot in gamifying a short survey, as the return would not outweigh the investment made.

gamification by setting challenges and using game-like exercises such as making a sentence about soup by asking each member to say one word that followed on that stated by the person before them.

They concluded that gamification has pros and cons

Participants were more enthusiastic and engaged while participating. Answers were similar to those collected on the Heinz Facebook page. On the downside, Barker noted that the games are not introspective in nature and mentioned that the games disrupted the group dynamics. This case highlights the importance of

applying gamification strategically to enhance data quality. As Barker and Hunt explored gamification more broadly and put more emphasis on making the research more game-like (instead of using game motivations to enhance the quality of the data rather than the experience of the research) their results show increased engagement, but not better data. Another example of applying gamification without focusing on the research questions comes from our own experiences. At InSites Consulting, we used to have

leader boards on the homepage of a community that displayed the ranking of the participants. The person with the most posts was on top and the person with the least posts at the bottom.

They concluded that gamification has pros and cons

What we noticed, however, was that gamification was

not improving the research because of two problems. Firstly, the leader board encouraged the more vocal people to become even more vocal while the more timid participants felt discouraged and became more silent. Secondly, the leader board was not rewarding the behavior we wanted to encourage. People posted more replies, but these posts were not richer or adding much to the topic. In this case, the gamification

rewarded posting a lot and not posting richer answers. If you want to benefit from gamification and collect better data, gamification should not only create a more engaging research experience, but should stimulate desired behaviour you want to improve (based on the

research objectives). This could be richer answers

that provide more context, more emotional depth or more creativity.

Current status

Typically, gamification in market research has been studied in surveys. However, looking at the concept of

gamification we would like to argue that there are equally many - if not more - reasons to apply gamification in qualitative research as well.

1

Firstly, from an investment perspective, it makes sense to also focus on

qualitative research. Donato and Link mention that the cost of developing gamification mechanics in surveys might not be worthwhile given the nonlength of a short survey. However, typically qualitative research tends to be longer in nature than quantitative research, making the investment more

worthwhile.

2

Secondly, in qualitative research we use projective techniques and

motivational techniques that are akin to gamification. For example, participants are often challenged to come up with pro and con arguments. We ask participants to take on a different role and think from this perspective; we send them on missions (e.g. shopper missions) and challenge them cognitively by asking analogies. This shows that gamification is ideally suited to be applied in qualitative research.

At InSites Consulting, we believe MROCs - or Consumer Consulting Boards as we call them (De Wulf & De Ruyck ed., 2013) - are powerful qualitative

research tools which help solve a myriad of business questions.

MROCs are composed of a small group of people, brought together on a platform to answer and discuss questions from a moderator for a longer period of time (three weeks or longer). The discussions are organized in different topics and rooms. As this is the main qualitative research method at

InSites Consulting, we investigated the fit between gamification and MROCs.

Looking at MROCs, there are four reasons why this method is Especially suited for applying gamification.

1

Firstly, given that MROCs are used for qualitative research and that we just explained how qualitative research already is a type a gamification, MROCs are well suited for

gamification.

2

Secondly, participants in an MROC need to be engaged for three weeks or longer. As

engagement needs to be built up and maintained, an MROC is more suited than a 30-minute survey. During three weeks, people can be immersed in the research topic and are taken on a research journey, providing more context for gamification and different gamification mechanics can be applied, resulting in more impact.

3

4

Thirdly, the online environment of an MROC makes that some game mechanics can be

automated and hence made more scalable.

Finally, while in surveys a social layer can be added, the social dimension in MROCs

is stronger and more natural. This helps to stimulate the social game dynamics in people.

Gamification in MROCs

As there is a high need to research gamification in qualitative research and there is a high fit between MROCs and gamification, we decided to invest in

this topic. In order to tap into the different

underlying game motivations (see earlier), we decided to apply gamification on four levels.

On the lowest level, the question level, we apply gamification by rephrasing the questions. For example, to understand what Generation Y considers to be the cool places in their city, we posted a challenge. Instead of asking them to list the cool venues and the reason why these places are cool, we challenged the participants to prove their city was the coolest to the client (MTV). We told them that, by the end of a given week and based on their answers, MTV would select the coolest city. This spurred elaborate reactions from our Gen Y participants who were fiercely defending their city.

One level higher, on the individual level, we introduce badges and levels. By answering to topics posted by the moderator, participants gain points. Depending on the quality of the post, participants gain more or less points. Additionally,

participants collect badges (see Figure 1) by performing certain actions such as posting very creative replies on a specific topic, keeping specific topics alive and interactive…

Figure 1. Examples of badges

The third level is the group level. This level is not always applicable, as it requires having at least

Every day, the number of contributions would be counted and the country with most posts would

two different groups in the research. On this level,

score one point (see Figure 2). The effect of this

gamification can be applied by setting a challenge for all groups and then compare the result between them. For example, in a research

was a significant increase in posts by everybody and people encouraging each other to answer to the questions and to take part in the game.

we did with R&D people at Unilever, we split the entire R&D team in several smaller groups. These groups could then compete to show who knows the consumer best. The winning team would then be the first to get access to new and exclusive content, namely new products in the pipeline. Another example of applying gamification on a group level comes from a community we did for Initial. We had participants from both the Netherlands and Belgium on the community, for three weeks. Between both countries there has always been friendly rivalry. So during the community we organized a derby between both countries.

Figure 2. Football derby between The Netherlands and Belgium

Finally, gamification can also be applied on a community level, where the complete group of participants is involved. This is where we set challenges for the community such as completing a certain task by the end of a given week or reaching a given amount of posts in a given timespan. Another example would be the ideation tool which we use to find new insights. Here, the community searches and thinks together to come up with new ideas and solutions. Participants can upload ideas as an answer to a need or problem. Others can rate these ideas and comment on them to further improve them. This way the community

works together on a need or problem that is close to their heart and is personally relevant.

Measuring the results

Looking at MROCs, there are four reasons why this method is

1

Firstly, gamifying a community makes participants think harder. This entails several results. We receive seven times more on-topic arguments compared to a non-gamified community (De Ruyck, Knoops, Schillewaert, Coenen & Rogrigues, 2011). In addition,

participants provide us with more context when answering, enabling us to better understand and frame their answers. Another consequence is that participants give emotional richer answers which allow us to understand them more completely. Additionally, in a gamified community, we get more creative answers.

2

Secondly, a gamified community allows us to make people think differently. When people approach a topic from another perspective, we can again better understand the

topic and come up with different insights. For example, for Chiquita we had to research the potential of a fruit smoothie with both people who eat healthily and people who eat less healthily and who don’t eat fruit regularly. In a first phase, we explored the perception and

attitudes towards eating healthily and eating fruit for both groups. In the second phase, we decided to apply gamification by having the groups switch roles in an activation deprivation exercise. The group of healthy eaters had to decrease their fruit consumption for one week and the group of unhealthy eaters had to eat a certain amount of fruit every day, for one week. Both groups then had to report back to us, providing us in the end with a complete picture on why people eat fruit and why not.

Gamification beyond data collection

As mentioned in the introduction, the market research world needs to

adopt its offering to both participants and clients. So far, we focused on applying gamification during data collection to increase participant engagement to collect better and more data. However, we believe gamification can also

be applied beyond data collection, further increasing participant engagement and also client engagement. Gamification can be applied during the analysis phase and the reporting phase as well.

Analysis

At InSites Consulting we apply gamification during the analysis phase as well, following the principles of crowd interpretation (Surowiecki, 2004; Verhaege, 2011). Crowd interpretation helps us understand the data even better, providing us with initial insights. Imagine for a second that you are doing a research project with people who have a very rare condition. Even though you might be an experienced and well-trained qualitative researcher, if you do not have that rare condition yourself, you will always have some knowledge gaps. This is where crowd interpretation comes into play. By asking our participants who suffer from the condition to interpret certain observations, they can add a different perspective that we - as researcher - are unable to spot. In order to make this even more engaging, we developed the crowd

interpretation game which exists of three rounds during which a selected group of participants (n=10) can collect points. The different rounds are analogous to the analysis process in qualitative research, where we go from observation to interpretation and to interpretation of the interpretation. In a first round, participants receive answers from other participants and are asked to interpret these. In a second round, they receive the same answers, but this time are asked to think deeper and provide more details.

After this, all the interpretations are

collected and sent back to the original poster of the answer. This person reads how others interpret his/her answer and then scores each interpretation on how good it is. Based on this, a ranking is made with the best participant researchers. The crowd interpretation game

has two benefits, one on participant level and one on researcher level. For participants, the crowd interpretation game is rewarding as they are more engaged in the game and because making them part of the analysis makes them feel more empowered. On a researcher

level, having participants interpret their answers means we get richer data and are able to gather 20% to 40% more insights (Verhaege, 2011).

Reporting During the reporting phase, there is another opportunity to apply gamification; this time to increase engagement on the client side, thus creating internal leverage for the research project. The goal of

gamification during this phase is to create positive disruption at the client side by exposing their knowledge gaps. Before presenting the report, different stakeholders at the client side test their knowledge of the topic. They are then given their score. The lower the score (or the bigger the knowledge gap), the more eager people are to read and reflect on the report. The example of the project with the R&D people at Unilever highlights the benefits (De Ruyck, Knoops, Schillewaert, Coenen & Rogrigues, 2011). The R&D people’s goal was to learn as much as possible about their consumer. To show they knew their consumer, they

had to play a game about consumers where they could collect points for each correct answer. There were different quizzes and after each quiz the scores of the different teams were advertised in different places throughout the R&D offices.

The game increased engagement significantly and the report was not only used more frequently, but also reflected upon more. This meant that the R&D people spent more time thinking about the results and their implications. Another example would be the Heineken Open Design Explorations case (De Ruyck, De Boeck, Eising, Troch & Van Hoff, 2012; see Figure 3. The Heineken ODE community platform). In this case we collected data and insights about what going out means to young and trendy people. Heineken’s goal was to take these insights and use them to build the club of the future with upcoming designers from different fields.

To make sure the designers would use the insights, we created an interactive app that immersed the designers in the results, allowing them to interact with the insights. In addition and to further stimulate engagement with the results, the designers had to come up with ideas based on the

results. Every uploaded idea contributed to

the personal score of the designers and the different scores were displayed on the home page.

Figure 3. The Heineken ODE community platform

Conclusion As a result of an outdated offering and changed consumer reality, the market research industry is faced with a declining engagement. Part of the solution to increase engagement is to apply gamification throughout the research process. Other methods to increase engagement include: creating a more engaging environment that is visually appealing and stimulates exploration, empowering people and starting a two-way dialogue as well as offering flexible solutions that bring convenience to participants. This paper presents a first account

of the effects of gamification in qualitative research and more specifically in MROCs. In addition, it also provides a framework on how to apply gamification beyond data collection. We have shown that applying gamification to an MROC can make people think harder (providing more context, more emotional and creative answers and seven times more on-topic arguments)

and differently (the ability to get a 360° perspective on consumers’ perceptions and attitudes), resulting in better and more data. The prerequisite for this being that the gamification is focused on increasing engagement with the research questions (instead of providing a general nice environment). In addition, to further increase the market research offering, we have shown that gamification can also be applied beyond data collection, namely during the analysis and reporting phase.

Finally, because of the nature of MROCs, we are able to develop a scalable model to apply gamification that keeps a healthy balance between the investment made and the benefits gained.

References

Adamou, B. (2011). The Future of Research Through Gaming. CASRO Journal.

Barker, A. & Hunt, L. (2012). Game on to keep consumers engaged. Marketing Week.

De Ruyck, T., De Boeck, F., Eising, H., Troch, T., & Van Hoff, C. (2012). Designing the club of tomorrow. ESOMAR General Congress.

De Ruyck, T., Knoops, S., Schillewaert, N., Coenen, G. & Rogrigues, S. (2011).

Engage, Inspire, Act: 3 Step Stones towards Developing more Impactful Products. ESOMAR General Congress.

Deterding, S., Sicart, M., Nacke, L., O’Hara, K. & Dixon, D. (2011). Gamification. using game-design elements in non-gaming contexts. CHI '11 Extended Abstracts on Human Factors in Computing Systems. 2425-2428.

De Wulf, K. & De Ruyck, T. (2013). The Consumer Consulting Board: Consumers shaping your business. Belgium: InSites Consulting.

.

Donato, P. & Link, M. W. (2013). The gamification of marketing research.

Marketing News. 38-42.Enterntainment Software Association (2013). Essential facts about the computer and video game industry. (source: http://www.theesa.com/facts/pdfs/ESA_EF_2013.pdf) Van Belleghem, S. (2010). The Conversation Manager. Belgium: Lannoo.

Van Belleghem, S. (2012). The Conversation Company: Boost Your Business Through Culture, People and Social Media. London: Kogan Page.

Van den Bergh, J. & Behrer, M. (2011). How Cool Brands Stay Hot: Branding to Generation Y. London: Kogan Page.

Verhaege, A. et al. (2011). Crowd Interpretation: are participants the researchers of the future? ESOMAR, General Congress.

Yee, N. (2006). Motivations of Play in MMORPGs: results from a Factor Analytic Approach. CyberPsychology & Behavior, 9(6): 772-775.

Sébastien Van Laere

Tom De Ruyck

Anouk Willems

Research Consultant InSites Consulting

Head of Consumer Consulting Boards InSites Consulting

Research Innovation Manager InSites Consulting

Thank you!

@InSites marketing@insites-consulting.com www.facebook.com/insitesconsulting www.slideshare.net/InSitesConsulting

www.insites-consulting.com