Page 104

Interestingly, the actual difficulty (measured as the average number of statements correctly evaluated by the students) was only rather weakly correlated with the difficulty as rated by the jury (ρ = 0.52, p = 0.01, FIGURE 6 .20). Nonetheless, the jury was actually rather accurately predicting the most difficult and most easy questions, but questions of average difficulty were rated with a large degree of uncertainty.

4

ρ = 0.52

# correct answers

● ●

3.5

● ●● ●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●●● ● ●● ●● ● ●● ● ● ● ● ●●●●● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●

3 2.5

2 0.0

0.2

0.4

0.6

0.8

1.0

Difficulty estimated by Jury FIGURE 6 .20  Weak correlation between difficulty estimated by the jury and actual difficulty of questions:

Shown is the average difficulty as rated by the jury (x-axis, 0 corresponding to a too easy and 1 to a too difficult question) against the average difficulty of a question as measured by the average number of correctly evaluated statements across all students (y-axis) for each question, together with a linear fit to the data.

6. 2. 5  Reception of the Theoretical Exams The theoretical exam was rated to be of very high quality as 80% of the jury gave the quality of the exam a mark of 4 or 5 out of 5 with an average rating of 4. 04 in a survey conducted after the IBO (Chapter 7). In contrast, the students ranked the theoretical exam to be too difficult with an average score of 3.82 out of 5 (3 being appropriate difficulty). Astonishingly, the jury was also very happy with the use of computer tablets with 78% choos­ ing a mark of 4 or 5 out of 5 and an average rating of 3.99. This suggests that future organizers should be advised to use similar means of testing. It is to note, though, that 8.7% of the jury members did not approve the use of computer tables. Finally, the jury seemed also very happy regarding how the jury sessions were organized. In total, 78.3% of all jury members appreciated the extended time, with an additional 15.9% be­ ing OK with it. Only 5.8% of the jury members were not happy with the extended time. The best marks, however, were obtained for the overall organization of the jury session, which was marked favorably by 78% of all jury members with an average score of 4.15 out of 5. All results to be found in Chapter 7.

104 | finalreportIBO

IBO 2013 Final Report