4.3 Quality Determinants and Value Added: The Case of Brazil

graduation, perhaps because programs with a higher reputation might charge higher tuition. Alternatively, higher tuition might provide resources to improve program quality—for instance, by hiring better trained faculty, upgrading infrastructure, or providing more student services, all of which might enhance skill formation and contribute to higher graduates’ wages.

As with formal employment, there are mixed associations between wages and determinants related to engagement with industry. On the one hand, graduates from programs or institutions with an employment center have reported higher salaries than those from programs without this service, a result that is aligned with evidence that job search assistance improves employment outcomes.34 On the other hand, programs that have agreements with firms to hire their graduates report lower wages among their graduates. Those agreements might create a tradeoff: although the firms agree to hire the graduates, they do so at a lower wage.35

The association between the type of administration (public or private) and wages is not statistically significant when accounting for all the other determinants. However, the estimations show that most of the associations between the determinants and wages are driven by private HEIs (see figure 4B.4, in annex 4B). That is, although governance type (public or private) per se is not associated with wages, the relationship between the quality determinants and wages is different for public and private programs.

So far, the chapter has reported findings using program-level data from the WBSCPS. In an ideal setting, the outcomes would be gauged from individuallevel administrative data. At the time of writing this report, Brazil was the only country for which data were accessible. Box 4.3 describes the use of these data to estimate the contributions of program characteristics and practices to student outcomes for Brazil. The estimates show that specific quality determinants— such as providing labor market information and receiving a high grade from the regulating authorities—as well as some characteristics, such as program size and HEI type, are associated with students’ academic and labor market outcomes.

Box 4.3 Quality Determinants and Value Added: The Case of Brazil

As discussed in “Defining and Measuring SCP Quality,” a possible measure of program quality is value added to student outcomes. The estimation of value added requires detailed data at the individual level on all the elements that could affect student outcomes, to disentangle the contributions of all the inputs involved, including student background characteristics and ability, peer background and ability, and others.

Such data were obtained for Brazil, specifically the states included in the World Bank ShortCycle Program Survey, São Paulo and Ceará. Data from several sources were merged: the Annual Reports of Social Information (Relação Anual de Informações Sociais, RAIS), a matched employer-employee data set of all workers and firms in the formal sector; the Higher

box continues next page

Box 4.3 Quality Determinants and Value Added: The Case of Brazil (continued)

Education Census; and the National Educational Entrance Examination (ENEM, Exame Nacional de Ensino Médio), the national assessment taken by students at the end of secondary education. In addition to test scores, ENEM includes student and family characteristics. Thus, the data set contains detailed information on academic readiness for higher education and socioeconomic background for the student and her peers, as well as labor market outcomes (wages and employment) for short-cycle program (SCP) graduates who are employed in the formal sector after graduation.

With these data, a two-stage approach is used to estimate program-level contributions net of the contributions made by the student herself and her peers. Three outcomes are considered for SCP graduates: graduation, employment in the formal sector, and wages. In the first stage, following the background paper by Ferreyra et al. (2020), the following model is estimated:

i R Z Y u ' ' ˆ ijt k k ijt k j k 1 2α α α α= + + + ijt k ∈ ,

(B4.3.1) where ijt kY is the outcome of interest, k = {1,2,3}, for student i, in program j, and cohort t. i kR includes individual characteristics, such as ENEM score, gender, age, socioeconomic status, and parental education. Zijt k is a vector of peers’ characteristics, including average ENEM score, age, socioeconomic status, and parental education of student i’s peers. Finally, uj k is program fixed effects. The vector of fixed effects estimates (û) constitutes the main vector of interest from the first stage—the estimated program-level contributions (the application of this methodology to Colombia is described in chapter 2).

In the second stage, the vector û is merged with the program characteristics collected through the World Bank Short-Cycle Program Survey (WBSCPS). Then the Least Absolute Shrinkage and Selection Operator (LASSO) approach is implemented to identify the determinants that jointly explain the most variation in û (see box 4.2).

The results show that SCP graduation rates in Brazil are associated with one important determinant and two program or higher education institution (HEI) characteristics (see figure B4.3.1). Programs that received a high grade from the regulator the previous year and those provided by universities have higher graduation rates. Moreover, graduation rates are higher for programs with higher enrollment.

Formal employment is higher for programs that provide labor market information for students, aligned with findings from the WBSCPS. Interestingly, offering online classes has a negative association with graduates’ formal employment, aligned with Ferreyra et al.’s (2020) findings for SCPs in large cities in Colombia.

The results for wages show that the single determinant that makes a sizable contribution is having a high grade from the regulator. How can this result be explained? As discussed in chapter 1, the National Institute of Educational Research and Studies (INEP, Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) evaluates programs annually in Brazil. For this evaluation, INEP uses data from the National Higher Education Assessment System, which assigns a Preliminary Course Score (CPC, Conceito Preliminar de Curso) to each program based on multiple indicators. These are related to program inputs and value added to student learning but not labor market outcomes.

box continues next page