Appendix C Research design for impact evaluation
Development Research in Practice focuses on tools, workflows, and practical guidance for implementing research projects. All research team members, including field staff and research assistants, also need to understand research design and specifically how research design choices affect data work. Without going into too much technical detail, because there are many excellent resources on how to design impact evaluations, this appendix presents a brief overview of the most common methods of causal inference, focusing on their implications for data structure and analysis. This appendix is intended to be a reference, especially for junior team members, for understanding how treatment and control groups are constructed for common methods of causal inference, the data structures needed to estimate the corresponding effects, and specific code tools designed for each method. Research team members who will do the data work need to understand the study design for several reasons. First, if team members do not know how to calculate the correct estimator for the study, they will not be able to assess the statistical power of the research design. This negatively affects their ability to make real-time decisions in the field, where trade-offs about allocating scarce resources between tasks are inevitable, such as deciding between increasing sample size or increasing response rates. Second, understanding how data need to be organized to produce meaningful analytics will save time throughout a project. Third, being familiar with the various approaches to causal inference will make it easier to recognize research opportunities: many of the most interesting projects occur because people in the field recognize the opportunity to implement one of these methods in response to an unexpected event. This appendix is divided into two sections. The first covers methods of causal inference in experimental and quasi-experimental research designs. The second discusses how to measure treatment effects and structure data for specific methods, including cross- sectional randomized control trials, difference-in-differences designs, regression discontinuity, instrumental variables, matching, and synthetic controls.
Appendix C: Research design for impact evaluation
195
