2.5 Investing in Technical Expertise in Indonesia

• Item writers must be subject matter experts in the domain to be measured and employ best practices for authoring item stems and response options or criteria for evaluating written responses. • Test administrators must ensure that each session is timed and that all testing protocols are adhered to, that criteria for student participation are understood and followed, and that participation and nonparticipation in the assessment are tracked. • Communications strategists must be able to construct clear, consistent messages to communicate key findings to a broad range of nontechnical stakeholders.

Having well-trained, technically proficient staff helps to ensure that the assessment will be administered following appropriate procedures and best practices. For instance, Indonesia invested in specialized training for staff who would be responsible for the design, development, and administration of their national large-scale assessment (box 2.5).

BOX 2.5. Investing in Technical Expertise in Indonesia

Stakeholders in Indonesia expressed interest in participating in the Programme for International Student Assessment and Trends in International Mathematics and Science Study as a way to gain experience and technical expertise that could be applied to their own national large-scale assessment program (Lockheed, Prokic-Breuer, and Shadrova 2015). In addition to participation in these international large-scale assessments, Indonesia invested in developing the technical capacity of its staff in psychometrics and educational evaluation. These specialists have since worked at Indonesia’s Ministry of Education and Culture, managing the country’s participation in these international assessments and the development of their own national large-scale assessments (Lockheed, Prokic-Breuer, and Shadrova 2015).

CLEAR, EFFECTIVE COMMUNICATION OF RESULTS Analysis and use of large-scale assessment results do not take place in a vacuum. It is important to be aware of existing priorities, pressures, and constraints within the education system. Presentation and communication of assessment findings must be sensitive to these concerns and to the intended and unintended consequences of the assessment for students and other educational stakeholders (Kellaghan, Greaney, and Murray 2009).

Assessment results should be summarized in a general report and further analyzed in reports tailored to the information needs of specific stakeholder groups such as policy makers, education managers, teachers, and students. The various ways in which assessment results can be communicated will be discussed in more detail in chapter 7. Findings should be disseminated to each of these audiences in a timely fashion and using clear language that they can understand. Findings from a national large-scale assessment are more likely to be used if they provide a clear indication of the factors affecting different levels of achievement and are diagnostic of problems within the education system. Assessment findings are more likely to be used when teachers understand the practical implications of the results and how information about the performance of the broader education system is relevant to their local classroom and school context.