Drawing on machine learning in the quest for effective teaching and learning Gabor Fulop (gabor.fulop@oecd.org), Noémie Le Donné (noemie.ledonne@oecd.org), Directorate for Education and Skills, OECD
W
ith most students having experienced remote learning over the past year due to COVID-19 related restrictions, the importance of teachers and schools has become only more evident. Temporary school closures underline the significant benefits students receive from being in school with their teachers and classmates. But what exactly do teachers do that helps students perform well academically, socially and emotionally? Identifying the teacher and school factors that help younger generations to succeed and thrive later in life is a longstanding challenge for education policy. Past education research has shown that how teachers, school leaders and schools shape the quality of instruction and students’ environment is closely related to student academic and social-emotional development (DarlingHammond, 2017[1]; OECD, 2018[2]). Past studies have found that teachers’ value-added accounts for significant variation in student achievement (Chetty, Friedman and Rockoff, 2014[3]; Jackson, Rockoff and Staiger, 2014[4]; Rivkin, Hanushek and Kain, 2005[5]; Rockoff, 2004[6]). There is also evidence that, as with test scores, teachers vary considerably in their ability to support students’ social and emotional development (Jackson, 2018[7]; Kraft, 2019[8]; Ladd and Sorensen, 2017[9]). The literature indicates that teachers and schools matter. However, the evidence is less conclusive as to the specific characteristics and actions of teachers and schools that matter the most for student achievement and social-emotional development. By applying a machine learning technique to a dataset that combines two large international surveys, the OECD report, Positive, Highachieving Students? What Schools and Teachers Can Do (OECD, 2021[10]), pinpoints some of the most effective teacher and school practices. The TALIS-PISA link data The two surveys in question are the OECD Teaching and Learning International Survey (TALIS), which is
the largest international and periodic survey asking teachers and school leaders about their working conditions and learning environments, and the OECD Programme for International Student Assessment (PISA), which provides the most comprehensive and rigorous international assessment of student learning outcomes to date, delivering insights into the cognitive and socialemotional skills of 15-year-old students. We call this the TALIS-PISA link. The TALIS-PISA link 2018 data comprises thousands of variables from more than 30,000 students and more than 15,000 teachers of the same schools from nine countries and sub-national entities: Australia, Ciudad Autónoma de Buenos Aires (referred to as CABA [Argentina]), Colombia, the Czech Republic, Denmark, Georgia, Malta, Turkey and Viet Nam. That being said, the specific survey design of the TALISPISA link data comes with its limitations. First, the data do not allow for matching a teacher and her or his students; rather, the data only permit matching a sample of teachers teaching 15-year-old students in a school and a sample of 15-year-old students at that same school. Information on teachers is therefore averaged at the school level and then analysed together with students’ outcomes. Given that teachers of the same school differ significantly in terms of their characteristics and practices, linking data by averaging teachers’ variables at the school level constitutes a considerable loss of information. Second, the cross-sectional design of the TALIS and PISA studies prevents causal interpretation of the analyses based on the TALIS-PISA link data. Drawing on a machine learning technique to let the data speak Applied education research has yet to tap into the rapidly expanding field of machine learning. Advanced datadriven methods are rarely applied in research looking at the nexus between teaching and learning. The latest OECD report analysing the TALIS-PISA link dataset (OECD, 2021[10]) seeks to break new ground by extracting maximum relevant information from this complex dataset.
Issue No. 74, July 2021 - The OECD Statistics Newsletter 3