The journey behind the global evaluation of the 2013 IKEA Catalogue

For IKEA, the yearly Catalogue is the main communication channel

with existing and potential customers globally. This case study show

What to expect?

how the 2013 edition of the Catalogue and possible covers for the 2014 edition were evaluated qualitatively around the world, through ‘Market Research Online Communities’ (or ‘Consumer Consulting Boards’) in five different countries.

The paper also shows how ‘Online Communities’ are becoming a

true ‘Fusion Research’ tool more and more which allows for ‘triangulation’ on different levels (data sources, research methods, research environments, theories and investigators),

leading to more valid research results, fresh inspiration and a deeper understanding of the issue researched. Best practices concerning: moving an existing qualitative project online, creating internal buy-in for emerging research methods, engaging internal audiences with research findings, running communities in different cultures and reactivating an MROC over time, are shared as well.

The IKEA Catalogue

Evaluating an icon of a global brand

IKEA has the vision ‘to create a better

A few weeks after the global distribution, a large quantitative

everyday life for many people’ by ‘offering a wide range of well-designed, functional home furnishing products at prices so low that as many people as possible will be able to afford them’. For IKEA, their yearly

study was conducted to evaluate the new edition on key performance indicators, to measure the impact it had in the market and to benchmark the performance with previous editions. The findings of these studies serve as input for the creative team that works on the latest edition.

Catalogue is one of the main channels of communication with existing and potential

For the 2013 edition, significant changes were made to

customers globally: one Catalogue showcasing

the format (slightly bigger), content (offering more inspiration

one product offering to serve ‘the many people’ around the globe. Every year, IKEA

pages, including more storytelling and a different way of

conducts a ‘global’ qualitative study to understand how people feel and think about the latest edition of the Catalogue and what they do with it (this was done before the actual launch).

1). Furthermore, a complementary mobile application was

picturing the products) and structure of the Catalogue (Figure launched. These structural changes made it even more important to deeply understand people’s emotional and

rational reactions to the latest edition.

From good to great

Over the past years the feeling had grown within IKEA that the qualitative evaluation of the Catalogue, done

through offline focus groups in different countries around the world, could potentially be done differently and better and in a fresher way. Here is a shortlist of the key issues that the business owners and the research team at IKEA were encountering:

1

Focus groups were only giving a snapshot of reality : the first reactions to the new Catalogue. No real insights were gained on people’s second thoughts and their behaviour afterwards (how they are using the Catalogue and how that use evolved over time).

2

Secondly, given the limited time spent with consumers and a limited portfolio of research techniques possible,

focus groups did not bring that much fresh and inspiring information to the table . Every year, a new edition of the Catalogue was only judged partially (due to the limits of the method) and in the same way, leading to conclusions that were very similar year after year.

3

Another issue flagged by internal stakeholders was the limited number of participants per session in a

focus group and the dominance of certain individuals in the discussion. Communities give access to more opinions of people with a wide range of profiles. And due to the longer time period, all participants are given an equal chance to give their opinion.

4

Fourthly, the project owners experienced that the quality of a focus group was largely

determined by the quality and experience of the moderator. There was probably interviewer bias as well. Country differences were possibly due to differences in moderation style, rather than real differences between cultures.

5

Finally, despite all the new technologies available to follow offline focus groups from home as a client,

internal stakeholders do not follow sessions that often.

Conclusion: Focus group transcripts are experienced as not that convenient and pleasant to go back to the real discussion for inspiration or to justify an element when creating or making decisions. The result is that internal stakeholders

are less confronted with the ‘voice of the customers’ during and after the project.

Given this situation, the research team at IKEA was on

Given these objectives, a Market Research Online

the look-out for a method enabling them to

Community (MROC) (De Ruyck et al, 2010) was chosen

understand both the emotional and rational reactions to the 2013 Catalogue and the mobile application. Furthermore, in order to really understand

as the backbone of this project in favour of traditional focus groups (See Figure 2). Online closed platforms

to have a dialogue and work together with

the Catalogue’s impact, it was also important to gain

consumers in five different countries around the

insights into how people use the Catalogue on a daily

world (Germany, Italy, Poland, the US and China)

basis after it landed in their homes.

seemed to be the answer to the above issues. But why

was this really a better option?

A simple, but very useful framework to evaluate a new research method and to demonstrate its effectiveness internally is to check if it is providing ‘automatical’, ‘informational’ and/or ‘transformational’ benefits

1

Automational: doing things faster and more cost-efficient

A research community indeed takes more time to set

Secondly, the set-up cost of a community project is

up, but once created it can be reused without losing

higher than that of an ad hoc qualitative project, but once

the time of a classical set-up phase. This is a plus in

established and in use it becomes a cost-efficient tool:

the process of crafting a new Catalogue: the platform

more and different research methods can be

cannot only be used to evaluate the current issue, it

combined within the same budget.

can also be used as a source of feedback and inspiration during the process of creating the next edition. For this project five communities with a 3-

week duration were created. Eight weeks later the platforms were reopened for one week to get feedback on the next stage - the creation of the cover for the 2014 edition.

The @Home Community

2

Informational: obtaining a better data quality and deeper insights

 Communities are characterised by the fact that you can work with more people of different profiles - in

this case with 50 participants in each of the five countries. We included three different profiles: potential, existing and lapsed customers. The result was five MROCs to hear the opinions of ‘the many people’ in one single project, on one platform.  The ‘research on research’ we did in the past taught us that for most participant profiles and research objectives it is best to run a community in the mother tongue of the participants. Firstly, by doing so they will

discuss more and they will post in a more nuanced and emotional way . Secondly, our experience showed that it is a must for the community to be moderated by a native who knows the language, the

local culture and the local market. All of this will lead to more relevant and to-the-point customer dialogues.

That is why we made the choice to set up a separate MROC in each of the five countries. This approach also allows us to run the whole project at once and in parallel. While being in contact with participants via a local moderator, you still grasp the advantages of having a central/global project team and content overview when

‘connecting the dots’ on a global level.  By working with a single master topic guide (which is adapted to the local reality and culture), interviewer bias is partly eliminated. The project is run by a team of different people who work together both on country and global level.

3

Transformational: doing things which were not possible before

Evaluating the Catalogue in an MROC

Another advantage of having

To obtain a holistic view on a

gives us the opportunity to work

more time with participants is

participant’s use and perception of the

with the same participants over a

‘consecutive learning’. One can

Catalogue, a whole range of

longer period of time (in this case:

build further on what one

research techniques was plugged

three weeks of 24/7 contact) in the

learned previously. Or stimuli

in into the community platform:

comfort and context of their own living

material can be tested, adapted

observational tasks, an online diary, a

environment (home). In this project, we

and tested again in only a matter

collage tool, mini-surveys, creative

followed the evolution in

of days with the same group of

exercises, group discussions and even

perception and usage of the

consumers.

implicit/emotion measurement tools.

Catalogue over time (before the

Communities allowed us to blend

Catalogue arrived, the first flip-through

different research methods, giving

moments and the different reading

us depth and breadth in terms of the

sessions). This is something that is

insights we found.

hard to realise cost-efficiently with traditional methods.

The research team did not have a hard time convincing the internal clients to make the switch, as it was clear that the time

had arrived for something different and better than offline focus groups. The advantages of using a community in the evaluation of the Catalogue (and the IKEA Catalogue App) were very clear as well. The questions arising among the business owners were more about the practical side of things, e.g. how to choose and optimally combine the different

research options available on the platform, how to safely distribute large numbers of Catalogues to different parts of the world before the actual launch and how to make sense of a big pile of information.

A research community as the backbone for fusion research

In order to bring fresh and unique insights to the table and to get a valid, clear and holistic view on people’s perception and use of the 2013 Catalogue, we used the principle of ‘triangulation’ (Guin et al, 2012) to create a true Fusion Research tool: investigating the same subject or issue from different angles and by doing so

create a more adequate and deeper understanding. We applied ‘triangulation’ on five different levels:

1

Data triangulation: by including both existing, lapsed and non-customers in the evaluation, we get data on all three groups and a full view on how different receivers of the Catalogue react to it: what do they feel, think and do?

2

Method triangulation: by combining different observational, qualitative (both interviewing and discussions) and quantitative research techniques (both explicit and implicit measurement) we tackle the same issue from

different angles, leading to a holistic view of it.

3

Environmental triangulation: the mobile application used on the community platform allows participants to not only take part on the main research platform from behind their PC, but to also provide us with more personal and contextual information - in this case on how they use the Catalogue on a day-to-day basis in their homes and over time.

4

Theory triangulation: as we know that ‘people think less than we think they think’, it is important to take into account both ‘System One’ and ‘System Two’ thinking when asking people to evaluate the Catalogue. That is why we included both an implicit and an explicit test of the 2014 cover during the one-week reactivation of the community.

5

Investigator triangulation: by asking the local community moderators, the global project team at agency side, the client-side researchers, the internal client and the research participants to analyse certain parts of the data, we made sure we got everything out of it and our final conclusions included different points of view from people with different backgrounds. The local moderators keep an eye on aspects specific to the local culture and market of their country, the global research team looks for the global consensus, the client-side researchers and internal client frame the results within the business context and participants help us to close the final blind spots we have. In this project we asked ambassadors from the five different country communities to join forces in an English-speaking Global Room where a discussion took place concerning the cover test during the reactivation, to see if and how it was possible to come to one cover for all countries. The view of the participants helped us to see which cultural differences could be won over.

Applying the principle of ‘triangulation’ asks for more time and man-hours.

One needs to make the trade-off between cost and getting additional understanding. In this project, it was a must to get a 360° view on how the Catalogue was perceived and used over time. It was only by making use

of a ‘Research Community’ and applying the principles of ‘triangulation’ to it, that we gained a fresh and complete view on the evaluation of the 2013 Catalogue.

Next we give examples of each of the different tools used to bring Fusion Research into practice

Phase 1: Evaluation of the 2013 Catalogue during three-week communities in five different countries

During the first week the goal was to ‘meet the reader’

After the Catalogue was dropped of at the

and understand the actual and aspirational

participants’ homes (in preview, two weeks before the

behaviour of the participants. Who is the reader

real global launch), we assessed the perception,

and what are his/her expectations towards the brand

satisfaction and level of engagement the 2013

and the Catalogue? To get there we conducted

Catalogue evoked. First impressions and second

amongst other activities a mix of an ethnographic task

thoughts on particular aspects of the Catalogue were

to get a view of their house and insights into how they

researched (structure, pictures, stories, the Ikea

live, a mood board exercise in which we asked the

Catalogue App, etc.). It was important to understand

members to map their feelings about the IKEA brand

how the Catalogue offers both inspiration and

and creative tasks on the forum of the platform (e.g.

information to the reader and if that was done

Tell us ‘Your IKEA Catalogue story’).

to the right extent. Moreover, we wanted to get a grip on the life cycle of the Catalogue. We did so by adding three specific exercises:

A photo safari, in which we asked the participants to spot the Catalogue in their house. We wanted them to imagine that the Catalogue could speak and tell the story of his past, current and future life.

During the last two weeks of the community, a calendar tool was added to the platform to get day-to-day insights in the usage and the emotions it evoked.

This was done in a private part of the community (participants could not read the responses of others), which aimed at avoiding bias.

Finally the Ikea Catalogue App was tested by understanding the expectations of these types of apps, asking them to use the app and to evaluate it afterwards on its relevance, user-friendliness and discuss possible improvements for the next version.

During the final week, we investigated whether the Catalogue was meeting expectations, giving us first clues

on the impact the latest Catalogue had on the brand perception and shopper behaviour. In other words, what impact did it have on the business: was it attracting people to the shop, raising interest in home furnishing and was there a positive impact on the perception of the brand, especially among lapsed customers?

Phase 2: Reactivation of the different communities (after eight weeks of inactivity) to test the overall theme and the first cover ideas for the 2014 Catalogue This was done in a two-step approach during one week: A mini-survey (N=226, response rate > 70%) to test six options for the new cover (illustrating the theme) Each participant randomly evaluated three different covers (resulting in at least 90 participants for each cover tested). The questionnaire was a mix of explicit questions (fit of the covers with brand and mission statements) and emotional/implicit measurement:

• Two-second test: each participant saw a cover for only two seconds. Afterwards, via an open-ended question, the members of the communities were to write down what they saw, which emotions they had felt and what they remembered. This was done three times for each participant in randomised order, to exclude order effect. We were measuring ‘stomach impact’ here explicitly, though spontaneously.

• Implicit measurement: each participant saw the same three cover pages for 10 seconds.

Afterwards, the members had to execute an ‘implicit measurement’ assignment where 15 emotions were shown rapidly (one second) in randomised order. Participants had to press the space bar each time they associated the emotion with the cover they just saw. Here we measured unconscious emotional reactions to seeing the cover page.

By combining the breadth of the ‘stomach impact’ on the X axis (the share of people who associate a given cover with the emotion, via implicit measurement) with the depth on the Y axis (for those who have associated the emotion with the cover, how strong is that association, i.e. how rapidly was the association made). To make

interpretation easier, the scale of the Y axis was reversed (= the higher, the faster) (see Figure 3). Results of these measurements were thrown back into the discussion during phase 2, in order to fully understand them.

Continuing with a discussion on the forum: in the survey, the participants were confronted with different cover ideas After two days they were asked which of the covers they still remembered. We used a tool in which you first have to answer yourself, before the others can see your answer (which ensures that answers are not biased).

We did this to assess the ‘stomach impact’ on the mid-long term. After the spontaneous recall test, we

conducted the spontaneous associations test: upon showing the different covers, participants were asked to share their initial thoughts, without prompting. Via this exercise we got an understanding of the key associations made with each cover. Finally, by asking indirect questions (e.g. tell the story of the cover),

participants were motivated to elaborate on the indirect impact of the cover. Finally, the most remarkable

results of the survey were shared and the participants were invited to comment on them in order to maximise our understanding.

‘Battle of the covers’ across the five countries in order to find creative ideas that would have appeal across the globe All participants of the five communities got the opportunity to take part in this discussion in English

in a central room. This way, cultural differences and local preferences were unveiled.

Reactivating a research community

The previous paragraphs informed us about the fact that a

The answer to the first question is ‘yes’: almost all

community provides one with automational,

participants from the communities in the US, Poland

informational and transformational benefits and that

and Germany took part again. For Italy (78%) and

it is a great tool to apply ‘triangulation’ to. The

China (70%), the reactivation was a bit less

examples provided in the previous paragraph on how we evaluated the 2013 Catalogue and pre-tested the themes

successful, although this was neither expected nor communicated at the start of the community.

and cover for the 2014 edition show that it can be a handy

Cultural differences in commitment are

tool while creating a new Catalogue as well. A community

probably at the basis of the lower numbers in

does not need to be ‘always-on’. You can perfectly align it

both countries. For all the questions on the different

with business planning. In this case the communities were

communities, we had at least 30 posts, which is what

used to draw learnings from the previous edition and

we needed to reach our saturation effect (Schillewaert

to get feedback on first ideas for the new one. We can

et al, 2011). Across the five communities we had

imagine situations later on in the creation process where we

2,807 interactions in a week’s time, which is 38%

might want to reactivate the communities for a short period

of the 7,261 we generated in the initial three weeks.

of time. Is that possible? Are participants indeed willing to

This is perfectly in line with what one may expect.

participate again and what does it take to encourage them to

do so?

How did we get to these results?

By making the participants feel part of the company as real ‘consultants’. Before the first phase of the project a kick-off was organised in a 30-minute chat session in smaller groups, during which we explained who the client was, what the goal of the project was and what was in it for them.

During the first phase, participants received weekly newsletters on the progress of the project

and what the company was learning from the discussion.

After the first three weeks and the workshop at the company side the participants got an update on how the debrief in Sweden went and pictures from the office were the Catalogue is created and a word from the team behind it. It is by doing so

and by keeping your promises in terms of incentives that you create a strong relationship with the members.

Global MROC projects: best practices

The fundamentals of the community approach work on a global scale. Just like the brands we are working for, we need to localise our way of working from country to country. In order to fully understand to what extent localisation of our methodology is required, we conducted several studies with moderators from our ‘Global Community Moderator Network’ (recruited in 30 different countries) and with local research participants. This way we co-created best practices for the different markets we are operating in. We found that it is important to

adapt your community to the local culture on five aspects. The direction the adaptations are made in can almost always be explained by the work of the Dutch academic researcher Geert Hofstede and his five dimensions to explain cultural differences between countries. Next we

explain the different dimensions in more detail and we also add some striking examples.

1

Reason to participate: intrinsic vs. extrinsic motivation From the first e-mail invitation for joining the

We also noticed that some countries are more

community onwards, it needs to be clear

extrinsically motivated than others. This is especially

what is in it for the participants. In

the case in the US and in Eastern European countries, but

almost all countries, the main reason to

for different reasons. Americans consider it normal that

participate is the possibility to have an

influence on the future of a brand or a product.

there is a payment to reward performance. In most Eastern European countries on the other hand an (monetary) incentive is perceived as a nice extra on top of their monthly income. Furthermore, in Poland, it is a must to gain the ‘trust’ of the members - trust in the fact that the agency or company behind the community will not harm them in any way and also in the fact that they will really get their incentive. The preferred type of incentive

differs from country to country. It is an illusion to think that ‘PayPal fits all’. In Asian countries such as China, the intrinsic part is important: they like to be connected with aspirational brands and share their wisdom.

2

Conversation guide: empowering vs. directive

A different culture also means different

Others prefer talking about the group, which is

attitudes and values, leading to a different

considered to be a safer option. See it as a

way of reacting to certain questions, tasks and

projective technique to let people talk about

exercises that moderators want participants to

their own situation, free of any pressure. The

perform. Some cultures, for instance, love to share

same holds for co-creation exercises. It is not a

a lot of details about themselves and their lives.

given in every culture that people are used to taking initiative. They feel better when they are only asked to give feedback about what already exists. It is important to map the country that

one is working in on those two axes (‘me’ vs. ‘we’ and ‘feedback’ vs. ‘co-creation’) and to adapt

the way of writing and (re)mixing topics for the conversation guide to it.

3

Role of the moderator: facilitator vs. authority One does not only need to adapt the way of

But it is also expected from the moderator that he/she

inviting and incentivising the members, the

is steering the discussion in the right direction when it

medium of data collection and the nature of the

is going off topic. In Poland on the other hand, the

topics in the conversation guide. The role of

moderator needs to be strict and almost literally direct

the moderator is also perceived differently

the members to the next question or task they need

from one country to the next. In Italy a moderator needs to facilitate and start the discussion. His/her role lies more in the background.

to look into. At the same time the moderator in Brazil is a social peer, he/she needs to be a formal professional in China and a like-minded person to exchange wisdom with in India. It is crucial to

know and manage all these different

expectations when running (multi-country or global) ‘Consumer Consulting Boards’.

4

Gamification: playful vs. serious Adding elements of ‘gamification’ to the community brings more richness to the table. In our research-on-research among our moderators, we learned that the level of and

the intensity with which you gamify your ‘Consumer Consulting Board’ need to

differ between countries. In Germany, for example, it is wise to limit it to a minimal level as it is culturally less accepted.

The elements described on the previous slides show that in multi-country projects you need to

start from a master conversation guide, which is important to make sure that there is a uniform way of working and that you exclude the effects of interviewer bias. But adaptations both in

content and style of the topics and in the way of moderating will be amongst other key elements in making the community a real success. Furthermore, it is wise to plan several debriefs between the different local moderators facilitated by the global research team: to challenge each other’s conclusions, let them go back to the results of their own country and come back with deeper and richer understanding in several iterative loops.

Creating a dialogue between the internal and external world

Only the business owners were following the

Previous ‘research-on-research’ (De Ruyck, 2011) and

community closely, although it should be convenient

our experience during this project have taught us that

as no travelling is required and one can take a

confronting stakeholders with real stories by real

look at the discussion on the community when

people is very impactful as an illustration of the

one feels like doing so. As a researcher there is a

main conclusions. They become alive. It is also an

clear need to give intermediate updates to the different

advantage that stakeholders can go back to the

stakeholders, as they do not follow the discussion

community based on the final report and read

spontaneously and there is a lot of information to

exactly what and how customers put it during the

digest. Moreover, this gives the business owners the

community. Finally, it is great that a community can be

opportunity to finalise the topic guide for the upcoming

reopened when you need it, perfectly in line with

days: going deeper into certain elements that are

the business planning.

really interesting or pushing the discussion in a new direction.

The research results have led to significant changes to the Ikea Catalogue App. We now know that the new

concept of the Catalogue was a big step into a new and right direction and improvements will be made for the 2014 edition, based on the research.

The future for research communities

The power of ‘Research Communities’ as a methodology lies in the fact that you have the ability

to work with more people, over a longer period of time, and that you get to know a lot from all kinds of different angles by combining different research tools and methods. The latter especially is still untapped potential in most communities. This case study describes how

fusing observational, qualitative and quantitative research methods lead to a deeper understanding and new insights. The case also demonstrates that ‘Structural Collaboration’ with consumers over time is valuable and leads to more impactful communication tools.

References

De Ruyck, T. et al (2010). ‘How Fans Become Future Shapers of an Ice-cream Brand’, Proceedings ESOMAR Qualitative Guin L., Diehl D. and McDonald D. (2012), Triangulation: Establishing the validity of qualitative studies, IFAS Schillewaert, N. et al (2011). ‘The Darkside to Crowd-sourcing in Online Research Communities’, CASRO Journal De Ruyck, T. et al (2011). ‘Engage, Inspire, Act: 3 Stepstones towards Developing more Impactful Products’, Proceedings ESOMAR Congress

Tom De Ruyck

Frédéric Gennart

Pieter De Vuyst

Frank Naessens

Head of Consumer Consulting Boards InSites Consulting

Global Market Research Consultant Inter IKEA Systems

Senior Research Manager InSites Consulting

Senior Research Consultant InSites Consulting

Want to know more about Consumer

Tom De Ruyck Head of Consumer Consulting Boards

Consulting Boards? +32 9 269 14 07 tom@insites-consulting.com

Thank you!

@InSites marketing@insites-consulting.com www.facebook.com/insitesconsulting www.slideshare.net/InSitesConsulting