Learning in Bridge, Public and Private Schools in Lagos
adapted to ensure item difficulty matched pupils’ ability. The level of the assessment and pupils’ abilities were assessed during the piloting and adjusted based on the results. In the case of Lagos, ceiling effects were observed during the DEEPEN baseline evaluation and therefore, more difficult items needed to be added to the instruments in order to ensure the test was targeting to the higher proficiency levels of the pupils in Lagos. This was done using IRT. The tests were placed onto the same scale using link items to link the difficulty of the ESSPIN test to a new version of the test and to place the results onto the same scale (metric).
A.4
Psychometric analysis
The use of Rasch modelling (IRT) increases the amount of analysis required as more sophisticated techniques are used, rather than adding up a total number of items correct in the test and converting the number into a percentage score. In this regard the evaluation team followed the following steps: the first step was to test the psychometric properties of the items to ensure they were useful measures of what pupils know. The second step was to remove any items that did not perform well. In a third step the team ranked the items according to difficulty. This was done by a psychometrician, using fit-for-purpose software. The software then also ranked pupils according to their ability and placed the pupils and the items onto the same metric. This is a probability model as pupils are placed on the scale according to the probability of a pupil answering the corresponding item correctly. A non-parametric IRT analysis was used to see the extent to which items met basic assumptions required by the usual parametrised IRT analysis. For both literacy and numeracy, the items were considered satisfactory. Both the literacy and the numeracy data was analysed using one-parameter IRT model. The one parameter model has an Expected A Priori (EAP) reliability of 0.82 for literacy, and 0.88 for numeracy. These are acceptable values (Allen 2018a, Allen 2018b). Full discussions of the psychometric analysis for literacy and numeracy, in the pilot and in the survey, are available in separate technical reports.
© EDOREN
49
