Appendix A: developing proficiency descriptors approach allows it to be used to scale descriptors of communicative proficiency as well as test items. In a Rasch analysis, different tests or questionnaires can be formed into an overlapping chain through the employment of ‘anchor items’, which are common to adjacent forms. In the diagram below, the anchor items are shaded grey. In this way, forms can be targeted to particular groups of learners, yet linked into a common scale. Care must, however, be taken in this process, since the model distorts results for the high scores and low scores on each form. Test C Test B Test A No 12. The advantage of a Rasch analysis is that it can provide sample-free, scale-free measurement, that is to say scaling that is independent of the samples or the tests/questionnaires used in the analysis. Scale values are provided which remain constant for future groups provided those future subjects can be considered new groups within the same statistical population. Systematic shifts in values over time (e.g. due to curriculum change or to assessor training) can be quantified and adjusted for. Systematic variation between types of learners or assessors can be quantified and adjusted for (Wright and Masters 1982; Lincare 1989). There are a number of ways in which Rasch analysis can be employed to scale descriptors: No 12.
(a)
No 12.
(b)
No 12.
(c)
No 12.
(d)
Data from the qualitative techniques Nos 6, 7 or 8 can be put onto an arithmetic scale with Rasch. Tests can be carefully developed to operationalise proficiency descriptors in particular test items. Those test items can then be scaled with Rasch and their scale values taken to indicate the relative difficulty of the descriptors (Brown et al. 1992; Carroll 1993; Masters 1994; Kirsch 1995; Kirsch and Mosenthal 1995). Descriptors can be used as questionnaire items for teacher assessment of their learners (Can he/she do X?). In this way the descriptors can be calibrated directly onto an arithmetic scale in the same way that test items are scaled in item banks. The scales of descriptors included in Chapters 3, 4 and 5 were developed in this way. All three projects described in Appendices B, C and D have used Rasch methodology to scale descriptors, and to equate the resulting scales of descriptors to each other.
No 12. In addition to its usefulness in the development of a scale, Rasch can also be used to analyse the way in which the bands on an assessment scale are actually used. This may help to highlight loose wording, underuse of a band, or overuse of a band, and inform revision (Davidson 1992; Milanovic et al. 1996; Stansfield and Kenyon 1996; Tyndall and Kenyon 1996). 211