BI1.1 Universes, Samples, and Response Rates, by Country

Box I.1 Some Technical Aspects of the World Bank Short-Cycle Program Survey (continued)

As the survey was conducted, the team learned that some programs had been closed and new ones had opened, so the universe of SCPs was adjusted accordingly. Given universe sizes (see table BI.1.1), random samples were surveyed in Brazil and Colombia, whereas the universe of SCPs was surveyed for the remaining countries. The samples were stratified by location (five regions in Colombia; the two states in Brazil); institution type (three types in Colombia— institución universitaria+escuelas and institutos tecnológicos, institución técnica profesional, and universidades—and four types in Brazil—universidade, centro universitário, faculdade, and instituto or centro federal); and administration type (public or private). Because of the lack of a geographic identifier in Brazil’s universe for online programs, these were not included in the survey. Table BI.1.1 shows the universe and sample sizes, as well as the response rates.

Since information on the surveys is reported by program directors, the team followed best practices to mitigate typical problems in self-reported surveys. First, respondents received a letter from the team leader indicating that their responses would be confidential, anonymous, and reported only in an aggregate fashion, thus favoring truth telling. Second, in the letter, respondents were told that the survey was being conducted by the World Bank and not a unit of their government (such as the Ministry of Education), thus eliminating possible gains from misreporting and also favoring truth telling. Third, questions were designed so as to avoid some common biases. For example, they refer to a specific period (such as the previous academic year) to address memory biases; are not open ended, and, where possible, include drop-down menus with specific options. Where possible, the team cross-checked responses with administrative data.

The team conducted tests to evaluate the representativeness of the samples given the appropriate calibration of sampling weights. In the case of Brazil and Colombia, the issue is whether effective surveys are representative of the universe. In the remaining cases, the issue is whether effective surveys are representative of their samples (which, in turn, are representative of their universes). In all cases, the team found that the effective surveys were representative of their universes (in Brazil and Colombia) or samples (in the remaining countries). In their background paper for this report, Dinarte et al. (2021) provide further information on representativeness tests.

Table BI1.1 Universes, Samples, and Response Rates, by Country

Country

Brazil Colombia Dominican Republic Ecuador Peru

Total

Universe size Sample size

Closed programs Effective surveys Response rate (%)

2,388 1,205 266 603 64 2,130 1,314 207 900 81 209 209 116 80 86 543 543 59 294 61 387 387 9 228 60 5,657 3,658 657 2,105 70

Source: Staff calculations based on data from the World Bank Short-Cycle Program Survey.