Defining and Measuring SCP Quality

determinants—such as providing labor market information and being deemed of high quality by the regulating authorities—as well as some characteristics, such as program size and HEI type, contribute to students’ academic and labor market outcomes.

Defining and Measuring SCP Quality Challenges of Measuring SCP Quality

Measuring the quality of higher education is challenging for a couple of reasons. First, there is little agreement over what is expected of higher education or how to measure quality in a standardized way. Moreover, the measures in a country are usually determined by data availability in its higher education information system.

A second challenge is related to whether quality should be measured through student outcomes or program value added. The distinction between outcomes and value added, outlined in this book’s introduction and chapter 2, helps clarify this challenge. Consider the wage earned by a program’s graduate immediately upon graduation. The wage constitutes the outcome, determined by studentlevel inputs (ability, effort, and other background characteristics), peers’ characteristics, and program-level inputs. The program’s contribution to the student’s wage, net of the contributions made by the student herself and her peers, is the program’s value added.

Estimating the value added of an SCP requires detailed individual-level data on all elements of the production function that could affect the graduate’s wage. Unfortunately, this level of detail in higher education administrative data is difficult to obtain from the countries in the region. Some countries do not collect these data. Others do, but gaining access to these data is an enormous challenge as it usually contains confidential individual level information.1 Collecting the data and facilitating their access remains a key task in LAC.

Due to the lack of data or the complexity of getting access to the data, this chapter follows an alternative approach, which is described in detail in Dinarte et al. (2021). The chapter uses the data reported by program directors to the WBSCPS on program infrastructure, curriculum and training, engagement with industry, costs and financing, faculty, and additional practices, as well as data on other characteristics of the programs, institutions, and students. Further, the chapter uses data collected by the WBSCPS on average academic and labor market outcomes for graduates at the program level, including dropout rates, extra time to graduate, formal employment, and wages.

Throughout, the term “determinant” refers to practices (for example, providing labor market information to students), inputs (for example, workshops for practical training), or input characteristics (for example, the percentage of faculty with more than five years of experience working in industry) that programs can choose and that could potentially affect graduates’ outcomes.

Using WBSCPS data for the five countries covered in the survey—Brazil (the states of Ceará and São Paulo), Colombia, the Dominican Republic, Ecuador, and

Peru (licensed programs)—the chapter estimates the marginal contributions of SCP determinants to academic and labor market outcomes of graduates, net of student characteristics. For example, it estimates how programs’ provision of labor market information to students is associated with improvements in students’ formal employment, after accounting for student characteristics. The analysis focuses on two categories of outcomes: academic performance—measured by dropout rates and time to degree—and labor market outcomes—which include employment in the formal sector and graduates’ salaries.

A couple of remarks are in order. First, the chapter estimates associations without claiming causality. To establish the causal effect of a determinant— for example, availability of an employment center—on an outcome of interest—for example, formal employment—ideally, individuals would be randomized between a group for which an employment center is available and a comparison group for which it is not available. Since individuals would be very similar between the two groups, any difference in formal employment would be attributable to the employment center. However, this approach is not feasible for thousands of programs and a large number of quality determinants.

Second, the program directors reported average outcomes for their programs and average student characteristics, not outcomes for individual students. To facilitate the explanation, imagine that program directors reported one average outcome (graduates’ wage), one average student characteristic (percentage of part-time students), and one program characteristic (whether the program offers remedial education). The estimation answers the following question: If programs A and B have similar student bodies (the same percentage of part-time students), but program A offers remedial education while program B does not, what is the difference in average wages between graduates from A and B? In this sense, the estimation is an attempt to quantify program value added using aggregate data from the WBSCPS.

The rest of this section describes the outcomes of interest. It also documents the average outcomes of the programs using WBSCPS data. The next section describes the quality determinants, and the following one summarizes the main associations between quality determinants and the outcomes of interest. Annex 4A provides summary statistics for the outcomes, quality determinants, and other program characteristics.

Outcomes

Dropout Rate and Extra Time to Graduate

Data were collected on two academic outcomes: the dropout rate and extra time to graduate. To measure the dropout rate, the directors were asked to focus on the cohort that was supposed to graduate the previous academic year. For this cohort, the directors reported the percentage of students who attained each of the following outcomes: graduated on time, dropped out, and were still enrolled in the program. The percentage of students from this cohort that dropped out of the program is the dropout rate measure.