IB Programme Impact Research: Findings and implications of recent research projects

IB Programme Impact Research: Findings and implications of recent research projects Olivia Halic Research analyst IB Global Research Department

2012 IB Conference of the Americas, Mexico July 2012

Presentation objectives

Brief introduction to IB Global Research department Programme impact research agenda Student performance on ISA Title I IB schools Brief overview of in-progress research projects

© International Baccalaureate Organization 2007

Programme impact

Quality Assurance

Programme development

IB Global Research

Assessment research

Professional development School Services

Global coordination, support, and services Š International Baccalaureate Organization 2007

Programme impact research agenda and priorities Learner profile

Student performance

To what extent do learners demonstrate characteristics of the IB learner profile?

What distinguishes IB learners in levels of motivation, values, and attitudes?

How do IB learners perform on external measures of academic achievement?

How do they compare with non-IB peers?

Impact/value-add of IB programmes on learners and schools

Standards

Programme implementation

How do IB standards compare to those at national/ state levels?

To what extent are IB graduates prepared for postsecondary success?

What is the impact/value-add of implementing IB programmes in schools?

What changes, if any, result from the implementation of IB programmes?

What are the enablers/ inhibitors of successful implementation? Š International Baccalaureate Organization 2007

Recently completed studies Project Title

Programme

Date

IB Students’ High School and Postsecondary Experiences in Chicago Public Schools

DP

March 2012

PYP and MYP Student Performance on the International Schools’ Assessment (ISA), Phase 2

PYP & MYP

March 2012

Student Performance and Student Engagement in the IB MYP

MYP

July 2011

International Baccalaureate students studying at UK Higher Education Institutions: How do they fare?

DP

April 2011

Postsecondary Enrollment Patterns of IB Certificate and Diploma Candidates from U.S. High Schools

DP

March 2011

Postsecondary Enrollment Patterns of IB Certificate and Diploma Candidates from International High Schools

DP

March 2011

First college courses taken by Florida IB students

DP

March 2011

© International Baccalaureate Organization 2007

Performance comparison between IB school students (PYP and MYP) and non-IB school students on the International Schools’ Assessment (ISA) – Phase II Australian Council for Educational Research March 2012 © International Baccalaureate Organization 2007

International Schools’ Assessment (ISA) Areas of assessment Mathematical Literacy

Reading

Narrative Writing

Expository Writing

Uncertainty

Retrieving Information

Content

Content

Quantity

Interpreting

Language

ESOL Language

Space and Shape

Reflecting

Spelling

Structure and Organisation

Change and Relationships © International Baccalaureate Organization 2007

International Schools’ Assessment (ISA)

Grade 3 to 10 Two sessions per year (October/ February) Multiple-choice & open-ended items 2 essays Aligned with OECD's Programme for International Student Assessment (PISA) scales for reading and mathematics

© International Baccalaureate Organization 2007

Study design-Sample 2009-10 & 2010-11 ISA sessions 270 participating international schools 117 PYP and 86 MYP schools

50,714 students 34,690 IB students 16,024 non-IB students

© International Baccalaureate Organization 2007

Study design Replication of phase I: 2009-10 & 2010-11 ISA data − Comparison of student performance by: assessment areas & sub-strands region

− PISA benchmark analysis − Multilevel analysis of school variance (new)

Closer examination of particular findings from phase I − Continuum schools compared with single or dual programme schools − Impact of the length of programme authorization

Student Learning and Wellbeing questionnaires (new) © International Baccalaureate Organization 2007

Analysis Significance legend Statistically significant difference: t-test Practical significance: effect size Sign

Meaning

Direction

Cohen’s d

+

Small effect size

Higher IB scores

0.1 ≤ d < 0.2

++

Medium effect size

Higher IB scores

0.2 ≤ d < 0.5

+++

Large effect size

Higher IB scores

d ≥ 0.5

▬

Small effect size

Lower IB scores

0.1 ≤ d < 0.2

▬▬

Medium effect size

Lower IB scores

0.2 ≤ d < 0.5

▬▬▬

Large effect size

Lower IB scores

d ≥ 0.5

© International Baccalaureate Organization 2007

Findings â&#x20AC;&#x201C; Comparison of student performance on ISA

Š International Baccalaureate Organization 2007

Mathematical Literacy Mean

IB S.D.

N

3 4

310 379

84 84

6,455 3,788

322 376

90 89

2,903 1,995

5

425

83

6,872

425

89

2,577

6

467

84

3,167

453

97

2,011

7

499

88

4,767

495

95

2,010

8

517

83

3,653

526

91

1,589

–

-0.10

9

551

85

3,227

535

87

1,699

+

0.18

10

570

91

1,948

529

84

1,085

++

0.46

Grade

Non-IB Mean S.D.

Significance

N

Effect Size

–

-0.13 0.03 -0.01

+

0.15 0.04

© International Baccalaureate Organization 2007

Reading Grade 3 4 5 6 7 8 9 10

IB

Non-IB

Mean

S.D.

N

Mean

S.D.

N

253 323 371 421 464 489 533 568

90 91 87 97 97 86 90 94

6,523 3,771 6,844 3,148 4,868 3,617 3,352 1,924

242 303 363 388 446 489 504 529

100 101 96 108 106 99 94 100

2,888 1,981 2,574 2,015 2,004 1,570 1,694 1,076

Effect Significance Size +

++ +

++ +

++ ++

Š International Baccalaureate Organization 2007

0.11 0.20 0.09 0.31 0.18 -0.01 0.31 0.40

Narrative Writing Grade 3 4 5 6 7 8 9 10

Mean 364 411 452 483 512 535 557 578

IB S.D. 59 62 64 66 69 69 74 74

N 6,540 3,759 6,826 3,160 4,867 3,629 3,337 1,934

Non-IB Mean S.D. 365 62 406 67 456 68 472 74 510 72 539 73 545 77 551 80

N 2,895 1,987 2,565 2,010 2,008 1,586 1,697 1,085

Significance

+ – +

+

++

Effect Size 0.00 0.08 -0.07 0.16 0.03 -0.06 0.16 0.35

© International Baccalaureate Organization 2007

Expository Writing Grade 3 4 5 6 7 8 9 10

Mean

IB S.D.

N

395 433 469 493 524 548 571 594

50 54 57 59 61 62 69 66

6,506 3,771 6,831 3,141 4,858 3,616 3,341 1,921

Non-IB Mean S.D. 394 426 471 486 517 551 553 563

54 60 62 65 66 68 71 81

Significance

N 2,879 1,979 2,565 2,019 2,002 1,570 1,682 1,073

+ + +

++ ++

Effect Size 0.02 0.11 -0.04 0.11 0.10 -0.04 0.26 0.43

Š International Baccalaureate Organization 2007

Findings In general, IB students had significantly higher mean scores than non-IB students Grades 6, 9 &10: IB students outperformed non-IB students in each area; strongest results at grade 10 In reading, IB students had higher scores in 7 out of 8 grade levels In expository writing, IB students had higher scores in 5 out of 8 grade levels

© International Baccalaureate Organization 2007

Europe & the Americas Domain

Mathematical Literacy

Reading

Narrative Writing

Expository Writing

Grade 3 4 5 6 7 8 9 10 3 4 5 6 7 8 9 10 3 4 5 6 7 8 9 10 3 4 5 6 7 8 9 10

IB

Non-IB

Mean

N

Mean

319 389 434 478 501 529 561 559 259 336 383 433 466 503 545 581 363 409 450 483 510 539 560 581 396 434 469 494 522 550 572 593

2,364 1,101 2,630 972 1,891 1,221 1,183 565 2,457 1,100 2,611 968 1,997 1,188 1,316 550 2,462 1,089 2,596 967 1,998 1,198 1,299 551 2,442 1,098 2,597 963 1,991 1,186 1,310 552

322 378 421 432 483 519 530 531 221 297 350 359 436 501 515 544 352 394 445 444 512 547 549 560 390 420 466 463 516 547 550 559

N

Significance

Effect Size

407 -0.04 + 276 0.14 + 367 0.17 +++ 441 0.55 ++ 329 0.22 + 305 0.13 ++ 357 0.42 ++ 424 0.34 ++ 406 0.40 ++ 275 0.44 ++ 363 0.37 +++ 444 0.75 ++ 328 0.31 298 0.02 ++ 354 0.37 ++ 425 0.37 + 406 0.18 ++ 274 0.24 363 0.07 442 +++ 0.51 327 -0.02 304 â&#x20AC;&#x201C; -0.13 + 357 0.16 ++ 424 0.28 + 402 0.12 ++ 273 0.24 359 0.06 447 +++ 0.51 328 0.09 298 0.05 Š International Baccalaureate Organization 2007 ++ 354 0.34 ++ 423 0.43

Comparison with PISA â&#x20AC;&#x201C; Mathematical Literacy, Grade 9/10

570 551

496

OECD countries 9

Partner countries

10

Š International Baccalaureate Organization 2007

Comparison with PISA – Reading, Grade 9/10 568 533

493

OECD countries 9

Partner countries

10

© International Baccalaureate Organization 2007

Student Questionnaire-Primary Years Primary Years (Grades 5 & 6)

Example items

Student and Teacher Interaction

Most of my teachers really listen to what I have to say.

Social Connectedness

I feel like I belong.

Personal Development Outcome

I like helping someone with a problem.

Study Engagement

I try very hard to complete all my work.

Š International Baccalaureate Organization 2007

Student Questionnaire-Secondary Years Secondary Years (Grades 8 & 9)

Example items

Student and Teacher Interaction

If I need extra help, I will receive it from my teachers.

Social Connectedness

I feel like I belong.

Personal Development Outcome

I am good at solving conflicts without fighting.

Learning Goals

I enjoy trying different approaches to see which one will work.

Deep Learning

I am keen to know how the things we do in class are meaningful to me.

Surface Learning

I don’t spend time learning things that I know won’t be in the exam.

Academic Outcome Orientation

I like to work on tasks that I have done well on in the past. © International Baccalaureate Organization 2007

Student Questionnaire Sample size

Primary N = 11,632 Secondary N = 10,058

Response rate

Primary > 80% Secondary > 87%

Findings - High agreement rates • Supportive schools: 88% -PYP students; 80% MYP • Feeling of connectedness: 83% - PYP & MYP • Engaged in their study: 83% - PYP & MYP • Deep learning: 80% MYP • Focus on learning for assessment: 56% MYP © International Baccalaureate Organization 2007

Implications Strongest results at grade 10 – growing trend Sustained results relative to ISA 2007/2009 Performance of PYP & MYP students measured against external benchmarks, with favorable results Comparison with PISA – additional opportunity for evaluation against international benchmarks Questionnaire results inform about student perceptions and non-scholastic outcomes

© International Baccalaureate Organization 2007

Questions?

Š International Baccalaureate Organization 2007

Title I IB Schools IB Global Research May 2012

Š International Baccalaureate Organization 2007

Background IB commitment to expanding access among underserved student populations Title I of the Elementary and Secondary Education Act of 1965 Two types of Title I assistance: Schoolwide programmes: schools with 40% or more of students from low-income families Title I Eligible programmes: schools with less than 40% of students from low-income families © International Baccalaureate Organization 2007

Data 2009–2010 National Center for Education Statistics (NCES) Common Core of Data (CCD) 103,959 public schools in the United States IB school data: 1,389 public schools in the U.S.

© International Baccalaureate Organization 2007

Findings - IB Schools Â&#x192; 56% of IB public schools (IB World Schools or candidate schools) were designated Title I

Š International Baccalaureate Organization 2007

Findings - IB Programmes

All Total IB Programmes

IB World Schools All DP MYP PYP

Candidate schools All DP MYP PYP

1,623

1,196

622

360

214

427

65

192

170

Title I

885

645

277

209

159

240

34

112

94

% Title I

55%

54% 45%

58% 74%

56%

52%

58%

55%

Schoolwide Title I

633

475

174

120

158

15

75

68

% Schoolwide Title I

39%

40% 29%

48% 56%

37%

23%

39%

40%

181

© International Baccalaureate Organization 2007

Findings - IB Programmes 1,623 IB programmes in US public schools Overall, 55% of programmes offered in Title I schools 74% of PYP offered in Title I IB world schools

74%

58%

45%

© International Baccalaureate Organization 2007

Implications IB’s goal to develop a more inclusive IB community by enabling access to education regardless of personal circumstances Understand who IB students and schools are, to provide better services to a increasingly diverse community

© International Baccalaureate Organization 2007

Questions?

Š International Baccalaureate Organization 2007

In-progress studies Project Title

Programme

The Relationship between MYP Student Moderation Performance and DP Student Performance

MYP & DP

DP Extended Essay Series

DP

Enrolment and Achievement of IB DP Graduates in the Australian Tertiary Education Sector

DP

Examining Gender Differences in the IB DP STEM Subjects

DP

Performance and Engagement in the MYP - Continuation study

MYP

MYP UK

MYP

India PYP

PYP

IB Teacher

Continuum Š International Baccalaureate Organization 2007

BRIEF OVERVIEW

The Relationship between MYP Student Moderation Performance and DP Student Performance

Š International Baccalaureate Organization 2007

Purpose To investigate whether DP students benefit from previously completing the MYP

Data MYP moderation scores from 2007-2009 DP exam scores from 2009-2011 6,352 MYP to DP students from 48 countries

© International Baccalaureate Organization 2007

Preliminary results Higher MYP moderation scores ↔ higher DP exam scores One unit increase in the MYP score leads to a 0.4 increase in the DP scores MYP students develop knowledge and skills that prepare them for the DP

© International Baccalaureate Organization 2007

BRIEF OVERVIEW

Performance and Engagement in the MYP - Continuation study

Š International Baccalaureate Organization 2007

Purpose To examine the influence of MYP on later high school performance and course enrollment To further explore student and teacher experiences with the MYP

Data 5 middle schools with MYP & 5 IB high schools Student performance and course enrollment (grades 9 & 10) Student surveys (including global mindedness scale) Teacher surveys & interviews © International Baccalaureate Organization 2007

Preliminary findings Â&#x192; 554 MYP students & 175 non-MYP students Moderate or large amount of time spent learning

MYP

Non-MYP

Issues affecting people around the world 51%

43%

Issues related to the environment

61%

52%

Other cultures

52%

45%

Š International Baccalaureate Organization 2007

Questions?

Š International Baccalaureate Organization 2007

Thank you!

Website Research overview •

http://ibo.org/research/

Programme Impact studies •

http://ibo.org/research/policy/programmevalidation/

Email • •

research@ibo.org olivia.halic@ibo.org

© International Baccalaureate Organization 2007