38
PRIMER ON LARGE-SCALE ASSESSMENTS OF EDUCATIONAL ACHIEVEMENT
TABLE 3.1 Roles and Responsibilities of Key National Large-Scale Assessment Team Personnel Role
Primary responsibility
National coordinator
Manage implementation of assessment activities, as guided by the national steering committee; may be assisted by assistant national coordinator, as needed
Assistant national coordinator
Manage and provide technical support to assessment development and implementation, as needed
Regional coordinator
Coordinate between the national team and participating local schools within the region
Item writer
Develop new items to measure proficiency against stated learning objectives
Test developer
Ensure that items are aligned with the assessment framework and test the blueprint
Translator
Ensure that test instructions and assessment items are as equivalent as possible across languages
Statistician
Develop sampling strategy and support analysis of assessment results by developing appropriate statistical weights
Psychometrician
Analyze item quality and support item selection in advance of implementation; support interpretation of findings for stakeholder reports
Data manager
Ensure accurate, appropriate labeling of data and labeling and organization of data files to support future and continued analysis
Data recorder
Perform data entry and quality control
Graphic design specialist
Design stimulus materials for inclusion in assessment and any charts, graphs, and illustrations used in stakeholder reports
Test administrator
Conduct assessment and ensure that everyone present adheres to testing protocols
Item scorer
Review and score open-ended constructed responses, as needed
Source: Original compilation for this publication.
must be flexible and responsive, especially when faced with technical and political challenges during implementation that must be navigated efficiently and effectively. Team members need to be able to operate with an appropriate degree of independence, particularly when the results of the a ssessment are unfavorable or potentially sensitive to stakeholders. Team members should be aware of, and demonstrate sensitivity to, local educational contexts in which learning is occurring; this awareness should be reflected in the instrument design, the data analysis process, and the reporting of results (Greaney and Kellaghan 2012). Figure 3.1 shows the organizational chart for the national agency responsible for developing and administering Chile’s national large-scale assessment (Sistema de Medición de la Calidad de la Educación). At the top of the chart is the agency’s executive secretary, who reports to the NSC. The NSC has five members selected for their experience and knowledge of the Chilean education system and appointed by the minister of education. The agency has regional liaisons who oversee assessment activities and communications in each of Chile’s five macrozonas or regions. The agency also has units in charge of administration, internal and external audits, and legal processes linked to the assessment process (Agencia de Calidad de la Educación 2020) and four work groups focused
