3.2 National Large-Scale Assessment Funding Checklist

TABLE 3.2. National Large-Scale Assessment Funding Checklist

Item

Personnel

Facilities and equipment Design of assessment framework Instrument design and development Training (such as item writing and data gathering) Translation

Printing National steering committee Local travel to schools

Data collection

Data scoring (open-ended items) Data recording Data processing and cleaning Data analysis Report writing Printing of reports Press release and publicity Conference on results

Consumables

Communications

Follow-on activities

Source: Greaney and Kellaghan 2012.

Source of funding Dedicated government funds Other funds Not funded

circumstances will vary from country to country, some items may not be relevant for some national large-scale assessment programs.

The absolute and relative costs of assessment activities depend on a range of factors, including the scope of the assessment, item types, administration format (for example, paper and pencil versus computer based), and the number and types of schools selected to participate. The costs will also depend heavily on the local context, including available technology and fees associated with hiring personnel with specialized technical expertise (Greaney and Kellaghan 2012).

PLANNING AND DEVELOPMENT Every assessment design decision has cost implications that should be carefully considered. In general, the more subjects and grades assessed, the more costly the assessment is. In addition, a census-based approach to data collection will, on average, cost more than a sample-based approach (Greaney and Kellaghan 2012).

Moreover, when new items are developed for the assessment, the budget must account for the training of item writers and the costs associated with pilot testing and item calibration procedures. Item scoring protocols and rubrics that will need to be developed for any constructed-response items also have costs involved. There may be costs associated with designing graphic or visual elements to be included in the assessment and printing answer sheets and other materials for paper-based assessments.

DATA COLLECTION Data collection activities typically account for most of the cost associated with conducting a national large-scale assessment. Information must be collected from schools in advance of the assessment; assessment materials (such as test instruments, questionnaires, and manuals) must be printed, packaged, and delivered; test administrators may need to be trained on testing protocols and require support for travel or local accommodations; and completed test booklets and questionnaires must be collected. In each of these undertakings, the number of schools and students participating in the assessment is a direct determinant of cost. Assessment teams must also consider that the costs of conducting an assessment are likely to be greater in schools in remote areas (Greaney and Kellaghan 2012).

DATA PROCESSING Once the assessment has been administered, test booklets and questionnaires must be processed and scored. The costs associated with these processes are frequently underestimated. Funding must be allocated for data entry and quality control and will depend on whether the tests are computer based or paper and pencil. Expenses for computer-based assessments include automatic scoring of student responses and use of data storage systems. For paper-and-pencil assessments, there are costs associated with scanning and machine scoring and scoring by hand for open-ended items (Greaney and Kellaghan 2012).

ANALYSIS AND REPORTING Chapter 2 underscores the importance of producing multiple reports targeted to various stakeholder groups. Policy makers, teachers, and the general public are likely to benefit from differentiated approaches to reporting of assessment results to highlight their relevance for these diverse audiences. The cost of developing and disseminating these reports will depend on the number of reports, whether results are presented numerically or with narrative and graphical support, and whether the results are printed or made available electronically.