Interestingly, the actual difficulty (measured as the average number of statements correctly evaluated by the students) was only rather weakly correlated with the difficulty as rated by the jury (ρ = 0.52, p = 0.01, FIGURE 6 .20). Nonetheless, the jury was actually rather accurately predicting the most difficult and most easy questions, but questions of average difficulty were rated with a large degree of uncertainty.

4

ρ = 0.52

● ●

3.5

● ●● ●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●●● ● ●● ●● ● ●● ● ● ● ● ●●●●● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●

3 2.5

2 0.0

0.2

0.4

0.6

0.8

1.0

Difficulty estimated by Jury FIGURE 6 .20  Weak correlation between difficulty estimated by the jury and actual difficulty of questions:

Shown is the average difficulty as rated by the jury (x-axis, 0 corresponding to a too easy and 1 to a too difficult question) against the average difficulty of a question as measured by the average number of correctly evaluated statements across all students (y-axis) for each question, together with a linear fit to the data.

6. 2. 5  Reception of the Theoretical Exams The theoretical exam was rated to be of very high quality as 80% of the jury gave the quality of the exam a mark of 4 or 5 out of 5 with an average rating of 4. 04 in a survey conducted after the IBO (Chapter 7). In contrast, the students ranked the theoretical exam to be too difficult with an average score of 3.82 out of 5 (3 being appropriate difficulty). Astonishingly, the jury was also very happy with the use of computer tablets with 78% choos­ ing a mark of 4 or 5 out of 5 and an average rating of 3.99. This suggests that future organizers should be advised to use similar means of testing. It is to note, though, that 8.7% of the jury members did not approve the use of computer tables. Finally, the jury seemed also very happy regarding how the jury sessions were organized. In total, 78.3% of all jury members appreciated the extended time, with an additional 15.9% be­ ing OK with it. Only 5.8% of the jury members were not happy with the extended time. The best marks, however, were obtained for the overall organization of the jury session, which was marked favorably by 78% of all jury members with an average score of 4.15 out of 5. All results to be found in Chapter 7.

104 | finalreportIBO

IBO 2013 Final Report