THE
ESSENTIAL GUIDE TO
All secrets, pros and cons of each methodology to improve survey quality and reduce costs.
The essential guide to CATI CAWI CAPI methodologies.
1
Introduction
3
CATI
4 Description ...............................................................................................................................4 How does CATI methodology work? ........................................................................................4 How is the call made? ..............................................................................................................4 Manual calls vs. Predictive dialing ...........................................................................................5 What are outbound and inbound calls? ...................................................................................7 Costs ........................................................................................................................................9 Conclusion ...............................................................................................................................9 CATI Pros...............................................................................................................................12 CATI Cons ..............................................................................................................................12
CAWI
13
Description .............................................................................................................................13 How does CAWI methodology work? ....................................................................................13 Univocal link distribution. .......................................................................................................13 Univocal link and quotas. .......................................................................................................13 Anonymous distribution. ........................................................................................................14 How is the distribution made? ................................................................................................15 Costs ......................................................................................................................................16 Conclusion .............................................................................................................................17 CAWI Pros .............................................................................................................................18 CAWI Cons ............................................................................................................................18
CAPI
20
Description .............................................................................................................................20 How does CAPI methodology work? .....................................................................................20 Anonymous distribution. ........................................................................................................20 Distribution to an existing list of respondents ........................................................................21 Offline and online CAPI ........................................................................................................21 Mistery shopping ....................................................................................................................22 Costs ......................................................................................................................................22 CAPI Pros ..............................................................................................................................23 CAPI Cons .............................................................................................................................23 CATI vs CAWI vs CAPI .........................................................................................................24
Mixed Mode
25
Description .............................................................................................................................25 How does mixed-mode work? ...............................................................................................26 Questionnaire characteristics for each methodology .............................................................26 Costs ......................................................................................................................................27 Mixed mode Pros ...................................................................................................................27 Mixed mode Cons ..................................................................................................................27 Applied Mixed-Mode use .......................................................................................................28
The essential guide to CATI CAWI CAPI methodologies.
2
Introduction In this ebook we’ll go through all pros and cons of the main methodologies used for research and surveys: telephone (CATI), web (CAWI) and face-to-face (CAPI). You’ll learn how to benefit from them and how to choose the best methodology for your survey. We’ll also talk about mixed-mode, a methodology that mixes multiple modes to optimize your work and minimize cons of the single methodologies. Over the years CATI methodology has lost its crown replaced by the more economic CAWI. But the quality of a web survey will never match a CATI or CAPI one because of the crucial presence of the interviewers. For this reason, mixed-mode can be the winning move to optimize your research. Selecting the right software is really important for any professional in this area. Knowing in detail all characteristics of each methodology can lead you to the best software decision.
After reading “The essential guide to CATI CAWI CAPI methodologies” click on the link you’ll find at the end of the ebook and complete the questionnaire. You’ll receive priceless tips on how to get the best out of your surveys based on your answers.
The essential guide to CATI CAWI CAPI methodologies.
3
CATI Description It’s the acronym of Computer Assisted Telephone Interviewing and it’s the telephone data collection. According to research dimensions, CATI methodology requires one or more telephone operators. The standard is a call center of 20-50 interviewers but it’s quite frequent to find even smaller reality.
How does CATI methodology work? The interviewer makes a phone call to the number that is displayed on the screen. If the respondent wants to join the survey, the interviewer starts reading questions displayed on the screen selecting the answers. The software will automatically proceed to the next question following the logic path. At the end of the questionnaire the interviewer will display a new respondent to call. The system will manage contacts according to rules set by the administrators. For example, contacts with a busy line will be displayed again after 15 minutes. If a contact answers the call but he’s not available for the interview, the interviewer can also schedule an appointment so that the software will automatically display that contact again at the date and time scheduled.
How is the call made? Each interviewer needs a standard phone or a soft phone. In both cases, the interviewer can make the call clicking on the number displayed on the screen. If the line is busy or cannot be reached, the interviewer can call a new respondent that will be automatically displayed by the software.
The essential guide to CATI CAWI CAPI methodologies.
4
Time spent on call attempts can be crucial for survey costs. A solution called “Predictive Dialing” was developed to solve this issue. With Predictive Dialing the dialer automatically manages all calls and forwards only available contacts to interviewers. Predictive Dialing is a feature that saves a lot of time maximizing interviewers performance. Manual calls vs. Predictive dialing
Manual call
Predictive Dialing
Call
On hold
Call
On hold
Waiting to dial number Dialing number Busy line No answer Unreachable Voicemail Wrong number Other
The essential guide to CATI CAWI CAPI methodologies.
5
Work with predictive dialing   PREDICTIVE DIALING (Automatic process) C o d a
Respondents queue
Delay of N mins
Automatic call
No answer
Call outcome Delay of N mins
Wrong number
Busy line
Interviewer available
Date and time
Appointment
Outcome
Refusal
Respondent discarded
Consent
Delay of N mins
Call dropped
Interview
Refusal during the interview
Interview completed
The essential guide to CATI CAWI CAPI methodologies.
6
What are outbound and inbound calls? Everything described above is happening when the call center makes the calls. This method is called Outbound. Some specific surveys may require a different method to allow respondents to reach the call center. This method is called Inbound. When using Inbound, the dialer forwards the call to the first interviewer available. The interviewer can take the call and start the interview after he searched and identified the respondent in the software database. You can also set the survey as mixed mode. This outboundinbound mode is usually used to manage calls that the call center gets after an attempt of call. It’s in fact really frequent that contacts call back the number to know who called them. Mixed mode allows to retrieve these calls optimizing the redemption of the survey.
The essential guide to CATI CAWI CAPI methodologies.
7
Scheme of the outbound-inbound flow.
Outbound
Inbound
Contact automatically selected in the list
Inbound call
Interviewer
Interviewer
Outbound call
Contact searched in the list
Interview
Interview
The essential guide to CATI CAWI CAPI methodologies.
 
8
Costs It’s quite obvious that CATI methodology has high maintenance costs. The software represents just a small portion of the cost: you have to consider costs related to the interviewers, call traffic and the initial investment on stations and software. Despite that, CATI methodology (along with CAPI) is still considered to be the most reliable and accurate mode.
Working with CATI ou'll face different requirements: for some surveys you’ll need all the interviewers you have, for some others you may need half of them. Find a software provider that has license flexibility to optimize the investment!
IdSurvey - leading software for CATI, CAWI e CAPI - offers scalable solutions to earn on the initial investment just paying for real use of the software. No initial cost, also ideal for small organizations to start working with CATI with minimum costs. Cloud dialer solutions, advantageous VoIP rates and soft phones (rather than standard phones) will make Computer Assisted Telephone Interviewing method much more convenient.
Conclusion We've just gone through CATI main characteristics highlighting high maintenance costs, higher than other methodologies. Why to use it then? You have a lot of advantages you'll hardly get with other methodologies. The presence of a real person - that acts like a mediator between the questionnaire and the respondent - has huge benefits: the respondent can understand questions better, a lot of don't know/don't answer can be avoided and, last but not least, the drop out rate is much lower.
The essential guide to CATI CAWI CAPI methodologies.
9
CATI surveys can basically reach the entire population, something that is still impossible to do with CAWI. Let's see a quick recap of CATI pros and cons.
The essential guide to CATI CAWI CAPI methodologies.
10
CATI survey flow  
C o d a
Respondents queue
Interviewer
Call Delay of N mins
No answer
Delay of N mins
Busy line
Wrong number
Respondent discarded
Call outcome
Refusal Date and time
Appointment
Delay of N mins
Call dropped
Consent
Interview
Refusal during the interview
Interview completed
The essential guide to CATI CAWI CAPI methodologies.
11
CATI Pros
• • • • • • • • • • • • • • •
Accurate answers Reliable information on respondents Almost the entire population can be reached You can get an accurate sample Stable redemption Timing can be easily calculated Out of quota interviews can be avoided Full control on the survey progress Realtime feedback The presence of the interviewer discourages the interview dropout Interviews can be longer as the drop out rate is lower Perfect for customer satisfation The interviewer can be trusted for sensitive topic The interviewer can help in questionnaire comprehension Discourage the don't know/don't answer
CATI Cons
• • • • •
High costs for telephone interviewers Costs for call traffic Costs for PC stations Costs for the dialer, software and hardware The number of interviewers is directly proportional to interviews completed per day
The essential guide to CATI CAWI CAPI methodologies.
12
CAWI Description Acronym of Computer Assisted Web Interviewing, it’s the web data collection. The respondent autonomously answers the interview on his computer, tablet, smartphone or any other device with a browser. Low costs make CAWI methodology extremely advantageous but not always the best choice.
How does CAWI methodology work? Respondents log in to the interview clicking on a link. The link can be published in a website or sent via email. The respondent will display a short text introducing the survey and then he will proceed to answer the questionnaire. It’s important that the user interface is as simple as possible and that question texts are perfectly clear. The survey can be distributed anonymously or with a univocal link. Univocal link distribution. Univocal distribution requires a different link for each respondent. The system is then able to link respondents profiles in the database with interviews completed. This procedure is useful to get information as who completed the survey and who started the survey but didn’t complete it. Tracking respondents is important to perform mass actions on the database, for example to send a reminder to all those who didn’t answer the questionnaire.
Univocal link and quotas. Univocal link distribution is also needed for CAWI surveys with quotas based on data already known. Let’s see an example: we want to complete 1000 interviews, half for females and half for
The essential guide to CATI CAWI CAPI methodologies.
13
males. When a quota is full, the software will stop unnecessary interviews (called over quota). If we know the gender in advance, the software will automatically stop respondents of a completed quota with an alert, avoiding useless interviews. When quotas are based on existing data, the univocal distribution is essential for the system to automatically discard irrelevant contacts. Sending email to irrelevant respondents can be also avoided using a unique feature in IdSurvey software. This characteristic optimizes survey distribution and saves you a lot of time and money. Anonymous distribution. In anonymous distribution there’s no connection between the respondent and the interview. This method is used when you have to guarantee anonymity to your respondents. In this case all the contacts will use the same link to log in. It can be published in a website, in social media or sent via email. It’s good to remember that anonymous distribution doesn’t prevent the same respondent to answer the questionnaire multiple times corrupting final data.
The essential guide to CATI CAWI CAPI methodologies.
14
How is the distribution made?
• Email You can send invites to a list of respondents with an email address. A CAWI software with an integrated email sender also gives you information on opened emails or, for example, on people who didn’t receive the email because of a full mailbox or a wrong email, etc…
Carefully selecting the survey software is really important as just a few of them have the integrated email feature. This integration is crucial to easily and consistently manage the entire survey. In fact, in some of these software you can also automatically program reminders to all respondents who didn’t answer the survey.
• Public link You can publish the link in any website or social media. This method is frequently used along with anonymous distribution.
• Panel A panel is a container of outlined respondents that can be p e r i o d i c a l l y i n t e r v i e w e d . We u s u a l l y c o n s i d e r a representative sample of a specific category to be a panel and panels can really useful to make frequent surveys. The term panel is (maybe improperly) used today to define any list of respondents available to join and answer surveys in exchange for prizes or money. There are several services of this kind as Toluna and Research Now that offer their respondents for surveys of any nature. There are also specific software to manage your own panel. For example, a company can create a panel with their own customers to design marketing stategies or to test a new product or brand. Surveys based on panels perfectly integrate with CAWI methodology because they can considerably increase
The essential guide to CATI CAWI CAPI methodologies.
15
response rate. Almost any CAWI software is now able to interact with panel services as that’s more a procedure than a proper feature. But you have to keep in mind that just some of those can really make the procedure simple allowing you to set everything up in a few clicks.
Recent statistics confirm that an high percentage of web respondents answers the interview with a smartphone. It’s essential that your survey software is mobile friendly: ask your provider if they used a responsive design so that the interview template can be correctly displayed on any device.
Costs Costs of CAWI surveys are extremely low and limited to the cost of the software and the tool to send emails. You can find several CAWI software with different policies of purchase. From low cost - with a small monthly subscription - to professional - with a cost per interview. If you just have to administer short questionnaires with simple logic path, then a low cost solution may be enough. To that you have to add a service to send emails like Mailchimp. If you want to administer surveys of medium/high complexity and get an adequate assistance, then you have to purchase a professional software. When you choose a professional software you have to look at the cost of the single interview, the setup and an integrated email sender (other than technical characteristics and ease of use). IdSurvey has a scalable purchase policy with no activation or maintenance cost so it's perfect for both sporadic users and professionals that administer hundreds of thousands interviews per year.
The essential guide to CATI CAWI CAPI methodologies.
16
Conclusion As said, CAWI has lower costs compared to other methodologies. No costs for the interviewers and no cost for the stations mean reducing survey costs a lot. Another benefit of a web survey is how quickly you can collect data. In fact, the user is usually lead to click on the link immediately after he read the email. Unfortunately there are also downsides. In CAWI methodology there’s no interviewer so no intermediary figure - differently than other methodologies. This implies a number of downsides that we have to keep in mind. For example, the user will be tempted to drop the interview if it’s really long or to give random or hurried answers. For this reason it’s important to design short, simple and direct surveys avoiding all text bias. Another peculiarity of CAWI surveys is the low ratio between invites and completed interviews. Depending on the quality of the emails you have, the percentage of completed interviews range between 2% and 5% of the invites sent. Because of prizes and rewards offered, surveys administered to panel respondents have an higher response rate.
The essential guide to CATI CAWI CAPI methodologies.
17
CAWI Pros
• Low costs • Short duration CAWI Cons
• Not suitable for long or complex surveys • Answers not always accurate: respondents can answer • • •
with a random option or with "don't know" answer No feedback on questionnaire design errors People with no internet connection or not familiar with computers/mobile devices cannot be interviewed Low response rate
The essential guide to CATI CAWI CAPI methodologies.
18
Curiosity How much we spend on online research worldwide?
Global Online Research Spend 2011-2015
 
(Source Esomar Industry Report 2016)
The essential guide to CATI CAWI CAPI methodologies.
19
CAPI Description Acronym of Computer Assisted Personal Interviewing, it’s the face-to-face data collection. An interviewer administers the questionnaire to the respondent and marks down answers on his smartphone, tablet or pc. Data are then synchronized with the server and they will be available for the analysis.
How does CAPI methodology work? To collect data you need an application that can interact with the main software. The app can be installed before handing over the device to the interviewer or it can be downloaded directly by the interviewer from the Google Play or App Store. Once the app is installed, the interviewer can log in to work on the survey assigned with credentials provided. They can collect answers from random respondents (anonymous distribution) or from a list assigned by the administrator.
Anonymous distribution. CAPI Anonymous distribution is the one to administer the questionnaire to random people, as supermarket or store shoppers, voters at polling stations, visitors in a museum/event or to passersby in the street. At the beginning (or end) of the interview some personal data are usually requested as the name, surname and telephone number. This procedure is used to create a list of respondents for other surveys or for marketing actions.
The essential guide to CATI CAWI CAPI methodologies.
20
Distribution to an existing list of respondents This distribution mode makes CAPI methodology really similar to the telephone one except for the fact that the respondent is interviewed face-to-face and not by phone. The interviewer has to select the contact from the list before starting the interview.
Offline and online CAPI Before you purchase your CAPI software it’s important to get this difference. An internet connection may not always be available when you go on the field. For this reason it’s crucial to choose a software that has an app able to work in offline mode too. This type of app is called CAPI offline and lets you collect data with no connection at all. It’s possible to collect as many interviews as you want and then synchronize them with the main server when you have a data or WiFi connection. Apps that don’t support offline mode cause a lot of issues for CAPI methodology. Sometimes they’re erroneously called CAPI online and they’re not useful at all for this methodology. Quite often they’re developed using existing material from CATI or CAWI interface: the investment of the software firm is minimal but the result is useless. It’s really important not to fall into a trap and be sure that the software you’re purchasing supports real CAPI - that is to say - an offline app.
The essential guide to CATI CAWI CAPI methodologies.
21
Mistery shopping Mystery Shopping is an activity used by big organizations to check on service quality, customers satisfaction or product quality on their commercial network using fake customers to evaluate any aspect of their stores. This methodology is crucial to collect important information and it’s considered to be essential to keep quality standards high. The fake customer can quietly complete the questionnaire using a smartphone or a tablet and take pictures to verify products display or rate service quality adding notes to any aspect they have evaluate. CAPI apps (those that can take photos or save the gps location in particular) are really useful to save information collected during the mistery shopping activity.
Costs When you estimate costs for a CAPI survey you have to keep in mind those related to the software, the devices and the interviewers. The cost of the software is difficult to calculate as it may differs a lot depending on the license policy. Some firms have a subscription option to pay for each interviewer while others have it only for the interviews. We suggest to opt for the latter as you’ll have definite costs and great flexibility.
The costs of the mobile devices for CAPI surveys can be easily voided if you ask the interviewers to use their own smartphone, tablet or PC. In this case, you need to select a CAPI app that is compatible with all major operative systems as Android, iOS and Windows PC. If you want to opt for company devices, you have to consider around 200/300€ for each interviewer, depending on the model selected.
The essential guide to CATI CAWI CAPI methodologies.
22
CAPI Pros
• • • • • • • • • • • • • •
Accurate answers Reliable information on respondents Almost the entire population can be reached You can interview people you didn't know before: customers of a mall, passersby, etc... You can interview respondents from rural areas The presence of the interviewer discourages the interview dropout With an appointment the interviewer can reach categories difficult to reach as lawyers or MDs Interviews can be longer as the drop out rate is lower Direct contact between the interviewer and the respondent The interviewer can be trusted for sensitive topic The interviewer can help in questionnaire comprehension Discourage the don't know/don't answer Respondent's reactions can be noted No data entry costs in respect to the old PAPI methodology (interviews on paper)
CAPI Cons • Costs for the interviewers • Costs for the mobile devices (if interviewers don't use their own)
The essential guide to CATI CAWI CAPI methodologies.
23
CATI vs CAWI vs CAPI CATI
CAWI
CAPI
★★★
★★
★★★★
Cost per interview
HIGH
LOW
MEDIUM-HIGH
Initial investment
HIGH
LOW
MEDIUM-HIGH
★★★
★★
★★★★★
★★★★
★★
★★★★★
★★★
★★
★★★★
★★
★★★★★
★★★
MEDIUM-HIGH
HIGH
LOW
★★★★
★★
★★★★★
★★★
★
★★★★★
Population coverage
Reliability of collected data
Answers accuracy
Redemption
Fast survey completion
Rate of dropout
Good for long questionnaire
Good for complex questionnaire
The essential guide to CATI CAWI CAPI methodologies.
24
Mixed Mode Description With Mixed-mode you can mix multiple methodologies in the same survey to maximize time and quality of your data. For example, you can start data collection with a phone call and then send a link to complete the survey via web. Or you can invite respondents to join the online survey and then interviewers will interview the respondents who didn’t answer the CAWI invite with a mobile app on their devices. Mixed-mode started as CATI-CAWI only but right now all advanced pro software allow multiple methods (CATI-CAWICAPI) in the same survey. Mixed-mode can be sequential (I start with one method and then move to another) or concurrent (I use multiple methods at the same time). If you use concurrent Mixed-Mode, you have to be sure that your software is really fit for an all-in-one data collection (one questionnaire, one database and one output file) for a faster data analysis.
The essential guide to CATI CAWI CAPI methodologies.
25
How does mixed-mode work? To use Mixed-mode at its best you have to purchase a survey software that is able to work with concurrent methodologies. “All in one” software allow you to create the questionnaire, associate a list of respondent and then export final data once, for all methodologies used.
It’s important to create a questionnaire valid for all methodologies when you use mixed mode. Some survey software allow dynamic texts that vary according to the specific methodology, for example notes for the CATI interviewer. In any case, we suggest to design a questionnaire that is suitable both for interviewers or respondents compiling.
Questionnaire characteristics for each methodology Interviewer Presence
Questionnaire characteristics
yes
Long questionnaires, shorter scales (1-5) and broken into steps (e.g., first asking “satisfied/dissatisfied” then asking “would that be very/somewhat,”), more complex logic as you can ask more questions, ability to hide “don’t know” and “refusal” options.
CAPI
yes
You can use multimedia, long questionnaires, shorter (1-5) or longer (1-10) scales, facilitate open-ended questions due with assistance from interviewer probing, also not verbal answers can be noted, ability to hide “don’t know” and “refusal” options.
CAWI
no
You can use multimedia, short questionnaires, disable answers editing, direct questions (yes/no, radio button and check box).
CATI
The essential guide to CATI CAWI CAPI methodologies.
26
Costs Mixed mode is good to limit costs and increase data quality at the same time. In this case, you have to consider the cost of the software, of the devices (if you use CAPI), of the interviewers and the emails. The list of items to consider might seem longer but you have to remember that the use of each single methodology will be much more limited than using just one, both in terms of timing and quantity.
Mixed mode Pros
• • • • • • •
Costs reduction Higher data quality Shorter timing Maximum reach of the population Each respondent can be reached with his preferred methodology High redemption Useful for multicountries surveys (different methodologies can be more or less effective according to national culture)
Mixed mode Cons • Designing a mixed mode questionnaire can be difficult if texts and logic change according to the methodology • Higher costs compared to CAWI methodology only
The essential guide to CATI CAWI CAPI methodologies.
27
Applied Mixed-Mode use CASE 1: CAWI - CATI PHASE 1: I have a list of respondents with an email address. I invite them to answer a CAWI questionnaire. Using pro software - with integrated email sender - I can schedule one or more reminders to respondents who didn’t answer the survey. PHASE 2: To increase the response rate I activate CATI methodology too after a few days. In this case, telephone interviewers will only contact users who didn’t answer the web survey. Alternatively, you can also use the two methodologies at the same time filtering the list according to specific parameters: for example, I can move to CATI only respondents with a phone number or only those who started and then stopped the interview.
CASE 2: CAWI - CAPI PHASE 1: I have a list of respondents with an email address. I invite them to answer a CAWI questionnaire. For example, the marketing department of a brand invites all their stores to compile the questionnaire. Using pro software - with integrated email sender - I can schedule one or more reminders to respondents who didn’t answer the survey. PHASE 2: To increase the response rate I activate CAPI methodology too after a few days. In this case, respondents who didn’t answer via web will be assigned to the CAPI interviewers. Each interviewer will have his own respondents to go to.
The essential guide to CATI CAWI CAPI methodologies.
28
For example, I assign to each face-to-face interviewer stores of a specific city.
CASE 3: CATI - CAWI PHASE 1: I have a list of respondents with a phone number. CATI interviewers will only administer a first part of the interview asking for personal information, including the email address. This way the respondents become familiar with the survey and are alerted that they'll receive an email invite soon. PHASE 2: I schedule an email just to respondents who provided their account. The interview will be resumed from the last question answered by the telephone interviewer. PHASE 3: In this case, I can plan a third phase when I go back to telephone methodology. After a few days from the email invite, I can contact by phone the respondents who didn’t complete the survey via web.
CASO 4: CAWI - CATI - CAPI PHASE 1: I have to administer the survey to a list of doctors and I have both the email and the phone number. To limit costs I start my research sending an email invite to complete the questionnaire via web. PHASE 2: To increase response rate, I contact the doctors to complete the interview by phone. Direct contact with the operator will encourage respondents to answer the survey.
The essential guide to CATI CAWI CAPI methodologies.
29
PHASE 3: In this situation, it might be difficult to contact doctors by phone too so I decide to move to a third and last phase: CAPI methodology. A part of the respondents list will be assigned to each interviewer so that they can go and administer the interview face-to-face.
The essential guide to CATI CAWI CAPI methodologies.
30
Conclusion Now you know survey methodologies better and you can improve your research and optimize costs. You may need some help to implement them in your working plan. That's why we want to offer you a free session with an expert of the IdSurvey team.
Benefit from our expert free consultancy. Learn how you can cut costs and improve survey quality. Answer the short questionnaire you'll find clicking on the button below to get a practical help on how to improve your research and receive 1 hour of free consultancy.
Go to the questionnaire
The essential guide to CATI CAWI CAPI methodologies.
31