PART TWO
Quantitative Analysis Most often we think of statistical or quantitative analysis when we think about analyzing. Project staff without a background in statistics may be intimidated by quantitative analysis; what we often see is that deference is given to external or internal evaluators because they know how to “do evaluation.” However, there are ways that project staff and an evaluation team without strong statistical backgrounds can analyze collected data. For example, you can begin by converting quantitative findings (e.g., numbers from utilization records, or answers on questionnaires) into percentages or averages. The importance of valuing and seeking multiple perspectives comes into play during this phase of the evaluation. Quantitative data analysis does require interpreting the results and seeing if it makes sense given the project’s contextual factors—factors that staff know better than most. Project staff and the evaluation team should work together and ask: Do these results make sense? What are some possible explanations for findings that are surprising? What decisions were made about categories and indicators of success? Have we missed other indicators? How might what we chose to collect and analyze be distorting the program/ initiative? And most importantly, how will the numbers and results help us decide what actions will improve the program? Remember that we want evaluation to support programs and help them improve. Complex statistical analyses of a well-designed experimental investigation that does not lead to improvements are less desirable than a thorough but simple statistical analysis of existing tracking records that leads to positive changes in both the program and in the tracking system. Qualitative Data Analysis Qualitative data includes information gathered from interviews, observations, written documents or journals, even open-ended survey questions. Information gathered from interviews and observations is often recorded in lengthy narratives or field notes. In some cases, interviews are tape-recorded and then transcribed. Some of these accounts are useful and can stand alone—providing important information about how the program is working. In most cases, however, it is valuable to analyze your qualitative data in more systematic ways. The Foundation, advocating for qualitative methods and analysis as a way to better understand programs, feels that not enough people understand the power and logic of qualitative methods.This is, in large part, because they have not been trained to systematically analyze qualitative data.Too often, qualitative data are
Page 88
Evaluation Handbook