8.9 Translation and Adaptation of International Large-Scale Assessments

BOX 8.9. Translation and Adaptation of International Large-Scale Assessments

Every Trends in International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS), and Programme for International Student Assessment (PISA) exercise involves the participation of countries from around the globe. Thus, questions related to translation and adaptation of these assessments commonly arise, particularly for countries interested in taking part in any of these international large-scale assessments for the first time. The TIMSS and PIRLS International Study Center at Boston College produces the international versions of these assessments and related materials in English, along with guidelines for translation and adaptation. National country representatives for each participating country are responsible for the translation of all materials into their languages of instruction and the adaptation to their cultural context while maintaining international comparability. External experts review each assessment translation and adaptation to verify the accuracy and comparability in the translation process. Two international versions of the PISA test and related materials are produced—one in English and one in French. Countries are required to perform two independent translations of either international version of the assessment into their official language of instruction. External reviewers verify the accuracy, equivalence, and fidelity of the translation and identify any discrepancies. After that, experts known as reconcilers produce a final translated assessment by reconciling any discrepancies between the two independent translations. Finally, subject matter experts review the materials for precision in the terminology and content.

Sources: Mullis et al. 2017; OECD 2018.

• Science literacy. Ability to engage with science-related topics and with the ideas of science as a reflective citizen. A scientifically literate person is willing to engage in reasoned discourse about science and technology, which requires the ability to explain phenomena scientifically, evaluate and design scientific inquiry, and interpret data and evidence scientifically (OECD 2019).

Each assessment cycle emphasizes one of the three core domains: reading, mathematics, and science. In 2018, the emphasis was on reading literacy, but mathematics literacy and science literacy were also assessed. In 2015, science received the greatest emphasis. In PISA 2022, mathematics will receive greater emphasis than the other two subject areas.

In addition to the assessment, students are asked to complete a background questionnaire focused on their home context, attitudes to learning, and learning experiences at school. School principals also complete a questionnaire on school management and the school learning environment. In some countries, an elective questionnaire was administered to teachers to gather additional information about instructional practices in the classroom. An elective questionnaire was also given to parents to determine their involvement in their children’s school and learning. Some countries administer additional questionnaires measuring student familiarity with computers, their expectations for further education, and their well-being (OECD 2019).