% of the respondents placed at the lowest level by the test results themselves assess that they are very good or fairly good at reading. For writing and numeracy skills, the corresponding figures are 46% and 67%. The discrepancy may in part result from the fact that the self-assessment questions do not seek to measure skills levels against an external yardstick but against a subjective yardstick of “the needs in daily life”. So in effect, the test and the self-assessment items do not measure the same construct. The discrepancy may also in part be due to measurement errors mentioned in section 3.2.2 above. The social desirability of certain responses as well as self-esteem factors are likely to have been important, and can explain the fact that the discrepancy is much larger at the lower ends of the skills levels than at the higher levels. The questions were not given a form so as to minimise demand effects. In addition, the anchors in the selfassessment items are inadequate, which most likely has contributed to measurement errors. Finally, a part of the discrepancy may result from validity problems in the test items that were used in connection with the survey.16 Since the discrepancy between test results and self-assessments is much larger at the lowest proficiency levels, this argument presupposes that there were particular validity problems with test items for the lower levels. Data from PISA PISA 2000 and 2003 also contained a number of self-assessment items. Thus, in PISA 2003, students’ “self-concept” was an index variable constructed out of 5 different selfassessment questions concerning students’ relationship with mathematics. The relation between the value of the index variable, which can range from -1 to +1 and mathematics performance scores is displayed in Table 6. As can be seen, there is a positive relation between self-concept and performance: The stronger the self-confidence of the student, the higher the performance score. However, the index variable still explains only a little more than 10 per cent of overall variance in student performance scores on average. Moreover, both the average student score on the self-concept variable and the explanatory power of the variable varies considerably between countries. The self-assessment scores of Japanese and Korean students are, for instance, considerably lower than for other countries. Yet the students from these two countries were among the best test performers. One possible methodological explanation is anchoring problems, as students from East Asia may adopt a different scale than students from other countries and cultures, cf. King et al. (2004).17 16
However, the predictive validity of test data from the 1996 NCDS survey, which is to a large extent based on the same test methodology as the BCS70 2004 survey, is good: Low test scores in literacy and numeracy is correlated with formal qualifications, employment status and several other relevant outcome variables (Bynner and Parsons 1997). 17 Differences in student self-assessments may also reflect real differences in educational systems: The Japanese and Korean school systems are extremely competitive and test-based compared to Western systems (DTI 2005: 133-134;
43