The Quality of Health and Education Systems Across Africa
Improving comparability of SDI surveys over time During the past decade, SDI surveys have evolved steadily. Each survey has been tailored to address specific policy and research questions in its setting of implementation. Although such specificity means that the surveys are well designed to respond to pressing national policy concerns, it also poses challenges for the direct comparability of some surveys from the same country over time. In addition, the SDI surveys are designed to provide just-in-time findings in fragile and conflict-affected settings, which means that logistical and safety considerations may also affect comparability. Experiences from multiple surveys in the same country are now being used to inform the design of future repeated surveys. This section outlines challenges, solutions, and lessons learned from repeated SDI surveys in Kenya, Mozambique, and Tanzania.7 The methods used in the Tanzania SDI education surveys in 2014 and 2016 allow for direct comparisons across the two years. Over this period, student learning outcomes for fourth-grade students, as measured by the English, math, and Swahili SDI tests, improved. The mean student test score across all subjects improved by an average of 11 percentage points. Other SDI indicators for which a significant change was observed in this two-year period included the share of students with pens, pencils, and an exercise book (increased from 84 percent to 92 percent) and the proportion of schools with a functioning blackboard (increased from 74 percent to 83 percent). Interestingly, the proportion of students with a textbook declined significantly (from 25 percent to 19 percent), while the observed student-teacher ratio increased (from 43 percent to 47 percent), perhaps indicating that system resources have not kept pace with enrollment. Smaller changes were observed in other SDI indicators, such as teachers’ absence from the classroom (declined by 5 percentage points) and teachers’ subject knowledge in language (deteriorated slightly) and mathematics (increased significantly, but only on certain tasks, such as Venn diagram and graph interpretation). Pedagogical knowledge also improved modestly. Trends in basic infrastructure showed no noticeable improvement. The 2014 and 2016 education surveys for Tanzania are so far the only SDI surveys that track a complete panel of schools over time. The direct comparability of data is the result of a careful panel sampling strategy and the use of identical survey tools. These methodological and logistical choices allow for accurate measurement of changes over time to track progress in education outcomes and process indicators. Nonetheless, the trends observed in Tanzania also suggest that, whereas certain indicators can be improved substantially in a two-year period, others may take longer. In addition, the measurement error associated with some indicators may make direct comparisons harder to interpret. For instance, many factors may affect the performance of students and 100
