Team Personnel

TABLE 3.1 Roles and Responsibilities of Key National Large-Scale Assessment Team Personnel

Role Primary responsibility

National coordinator Manage implementation of assessment activities, as guided by the national steering committee; may be assisted by assistant national coordinator, as needed Assistant national Manage and provide technical support to assessment development and coordinator implementation, as needed Regional coordinator Coordinate between the national team and participating local schools within the region

Item writer

Develop new items to measure proficiency against stated learning objectives

Test developer

Ensure that items are aligned with the assessment framework and test the blueprint

Translator

Statistician

Psychometrician

Data manager

Ensure that test instructions and assessment items are as equivalent as possible across languages Develop sampling strategy and support analysis of assessment results by developing appropriate statistical weights Analyze item quality and support item selection in advance of implementation; support interpretation of findings for stakeholder reports Ensure accurate, appropriate labeling of data and labeling and organization of data files to support future and continued analysis Data recorder Perform data entry and quality control Graphic design specialist Design stimulus materials for inclusion in assessment and any charts, graphs, and illustrations used in stakeholder reports

Test administrator

Conduct assessment and ensure that everyone present adheres to testing protocols

Item scorer

Review and score open-ended constructed responses, as needed

Source: Original compilation for this publication.

must be flexible and responsive, especially when faced with technical and political challenges during implementation that must be navigated efficiently and effectively. Team members need to be able to operate with an appropriate degree of independence, particularly when the results of the assessment are unfavorable or potentially sensitive to stakeholders. Team members should be aware of, and demonstrate sensitivity to, local educational contexts in which learning is occurring; this awareness should be reflected in the instrument design, the data analysis process, and the reporting of results (Greaney and Kellaghan 2012).

Figure 3.1 shows the organizational chart for the national agency responsible for developing and administering Chile’s national large-scale assessment (Sistema de Medición de la Calidad de la Educación). At the top of the chart is the agency’s executive secretary, who reports to the NSC. The NSC has five members selected for their experience and knowledge of the Chilean education system and appointed by the minister of education. The agency has regional liaisons who oversee assessment activities and communications in each of Chile’s five macrozonas or regions. The agency also has units in charge of administration, internal and external audits, and legal processes linked to the assessment process (Agencia de Calidad de la Educación 2020) and four work groups focused