Issuu on Google+

PAT Sampling: Lessons Learned from Haiti and East Timor

March 4, 2009 Natalie Domond Washington

London

Johannesburg

Ramallah


Project Introductions Support to Haiti’s Microfinance, Small and Medium Enterprises Sector (Haiti MSME) ƒ Program focuses on supply side enhancement of financial services to MSME’s ƒ Beneficiary population includes clients of microfinance institutions, commercial banks, and credit unions ƒ PAT first implemented Jan-Mar 2008; preparing for 2nd PAT FebApr 2009

East Timor Dezenvolve Setor Privadu (DSP) ƒ Program focuses on agricultural value chains, rural enterprise, and BDS development ƒ Diverse beneficiary population includes farmers, coffee growers, cattle breeders and microenterprise targeted by DSP and two other USAID programs: Small Grants Program (SGP) and Cooperativa Café Timor (CCT) ƒ PAT implemented June-Sept 2008

Both USAID projects funded under the Accelerated Microenterprise Advancement Program IQC


Haiti MSME: PAT Planning Overview ƒ Overall implementation planned over 8-10 weeks ƒ 18-person team (12 interviewers) ƒ Involved 21 MFI’s and 19 credit unions in all 10 departments of Haiti ƒ MFI’s used solidarity and individual lending, and village banking methodologies Challenges to Sampling Design • Largely rural and geographically dispersed population • Poor road and communications infrastructure • Rugged, mountainous terrain • Involved a number of institutions to get branch and client data


Haiti MSME: Sampling Plan ƒ Used cluster sampling method ƒ Requested active client info by branch from 21 MFI’s and 19 credit unions ƒ Received client info from 15 MFI’s and all 19 credit unions ƒ 164,781 active clients reported (sampling universe/pop.) ƒ Needed 383 surveys for a confidence level of 95%; added 15% for 440 targeted surveys ƒ Clustered branches by geographic location (department) ƒ Conducted three rounds of sampling – Randomly selected districts within departments – Randomly selected branches within districts resulting in 32 branches from 12 different institutions – Randomly selected clients or groups from branch lists


Haiti MSME: Sampling Plan

Department Artibonite Grand Anse Nippes Nord Nord-Est

# of # of clients arrondissement to sample to sample by Total # of % of Total dept.* by dept Clients Clients 16,778 10.18% 45 4,157 2.52% 0 3,175 1.93% 8 9,083 5.51% 24 6,148 3.73% 16

Arondissements Randomly Selected** 1 Gonaives 1 Anse a veau 1 Miragoane 1 Borgne 1 Ouanamenthe Port-de-Paix, Mole St. 2 Nicholas

# of branches to sample 2 0 1 2 2

Nord-Ouest

17,201

10.44%

46

Ouest Plateau Central Sud

56,263 6,468 24,911

34.14% 3.93% 15.12%

150 17 67

Cabaret, Leogane, 3 Port-au-Prince 1 Lascahobas 2 Cayes, Aquin

12 1 5

Sud-Est

20,597

12.50%

55

2 Belle Anse, Jacmel

4

164,781

100.00%

429

Grand Total

*Process for Selecting # of Arondissements to Sample by Dept. 1-10% of total clients - one arrondissement 11-20% of total clients - two arrondissements Greater than 20% - three arrondissements

15

3

32


East Timor: PAT Planning Overview ƒ Expanded scope to include non-DSP beneficiaries therefore further diversifying client characteristics and geographic reach (90% rural) ƒ Planned and implemented over 4 months ƒ 11 member team – most members trained to fulfill multiple roles • 1 PAT Manager & 1 Assistant PAT Manager • 2 field supervisors/trainers • 3 team leaders/interviewers • 4 interviewers and data processors


East Timor: PAT Planning Cont’d Challenges to Sampling Design ƒ Complete lists of beneficiary information not easy to get; had to clearly define what a beneficiary was and rely heavily on cooperation from many organizations to get client data ƒ Beneficiaries geographically dispersed and many sucos (cities/towns) are hard to reach because of road conditions and mountainous terrain ƒ Sampling plan required careful consideration in order to construct a sample that would be feasible to implement within a reasonable timeframe and budget


East Timor: Sampling Design Cont’d Goal: Construct a sample that balances statistical accuracy with administrative practicality Step 1: Determine the PAT Population (Sampling Universe) ƒ Sampling theory requires that a sample be representative of the qualities of the beneficiary population and be of sufficient size in order to achieve valid results ƒ To get PAT population, narrowed down the number of districts by focusing on those that received most support in FY08 and that would be representative of the characteristics of the entire ME population ƒ Over several months took inventory of all DSP, CCT and SGP ME beneficiaries in FY08 ƒ Result: a PAT population of 24,022 Step 2: Determine Sample Size ƒ For a 95% confidence level, a sample of 378 is required ƒ A cushion of about 40% was added to take into a account surveys that would not pass the quality checks or unreachable respondents ƒ Result: a survey sample size of 520


East Timor: Sampling Design Cont’d Step 3: Begin Sampling ƒ ƒ ƒ ƒ ƒ ƒ

Used Probability Proportionate to Size (PPS) Cluster Design Grouped organizations in selected districts by city or village to form a “cluster” Consulted PAT Help Desk who determined 26 clusters of 20 interviews was needed Clusters were allocated proportionately by district Used PPS method to select clusters and random sampling to select respondents within clusters Created final sample list of 20 primary and 10 alternate interviews by cluster Subtotal by district District Aileu 3,774 Baucau & Viqueque 917 Bobonaro 1,408 Covalima 2,947 Dili 489 Ermera 10,629 Liquica 3,858 Total

24,022

% of PAT population 16% 4% 6% 12% 2% 44% 16% 100%

# of clusters by district 4 1 2 4 1 10 4 26

# of interviews by district 80 20 40 80 20 200 80 520


East Timor: Final Sample Characteristics •

7 out of 13 districts selected – Aileu, Baucau, Bobonaro, Covalima, Dili, Ermera, Liquica

Three regions represented– 2 in West, 3 in Central and 1 in East

90% rural; 10% urban (including highlands and lowlands)

Variety of areas of economic activity represented


Examples of Unforeseen Challenges ƒ Two of the 12 MFI’s selected for the sample in Haiti refused to participate. ƒ Five MFI’s/credit unions in three different departments wanted to participate but could not get us the client information electronically. ƒ Selected several MFI branches with solidarity lending or village banks that did not have individual client data. ƒ Other Random Challenges: One client selected lived on an island off the coast; lack of cooperation from branches & loan officers; difficulty finding clients; clients wanted compensation.


Sampling Lessons Learned ƒ Getting client information can be much harder than anticipated! Plan adequate lead time; on-site sampling may be necessary. ƒ Very important to get buy-in/cooperation from partner institutions; if not, can cause extraordinary delays throughout planning and implementation. ƒ Make sure the sample includes a sufficient number of pre-selected alternate respondents (if possible) and that the interview team is clear on when and how to use the alternate respondent list. ƒ Everything will not always go as planned! When not sure what to do, just remember the two basic principles of quantitative surveys: randomness & representation!


General Lessons Learned ƒ Take plenty of time during implementation team training to go through each question and answer choice, especially when surveys were translated. Also, further customization of survey might be necessary. ƒ Dedicating entire team to the capital where training is taking place first provides a cost-effective way to provide feedback and evaluate interviewers’ performance before sending them out to more remote areas. ƒ A flexible team structure with team members trained in multiple roles can prove helpful in filling HR gaps as needed ƒ Always remember: the PAT Help Desk is a valuable resource ready to assist you throughout implementation!


Questions?


DC PAT Training _ Natalie slides