Issuu on Google+

Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

11

Programme for International Student Assessment

Strong performers and successful reformers The yardstick for success is no longer improvement by national standards alone but the best performing education systems

Andreas Schleicher

Special advisor to the Secretary-General on Education Policy Head of the Indicators and Analysis Division, EDU


PISA 2009 in brief

PISA countries in 2001 2003 2000 2009 2006 1998

Andreas Schleicher Warsaw, 10 February 2011

OECD Programme for International Student Assessment

PISA What students know and can do

22

Coverage world economy 83% Over half a million of students… 81% 77% 86% 85% 87% 

representing 28 million 15-year-olds in 74* countries/economies

… took an internationally agreed 2-hour test… Goes beyond testing whether students can reproduce what they were taught… 65 … to assess students’ capacity to extrapolateRoutine from manual what they Changes in skill demand know and creatively apply their knowledge in novel situations 

60

… and responded to questions on… 

Nonroutine manual

their 55personal background, their schools Routine cognitive and their engagement with learning and school

50 principals and system leaders provided data on… Parents, Nonroutine 

*

analytic school policies, practices, resources and institutional factors 45 that help explain performance differences Nonroutine . interactive

40

Data for Costa Rica, Georgia, India, Malaysia, Malta, Mauritius, Venezuela and Vietnam will be published in December 2011

1960

1970

1980

1990

2002


PISA 2009 in brief

Andreas Schleicher Warsaw, 10 February 2011

OECD Programme for International Student Assessment

PISA What students know and can do

33

PISA countries in 2001 2003 2000 2009 2006 1998

PISA seeks to… of world economy 83% Coverage 81% 77% 86% 85% 87%

… Support governments to prepare students… … to deal with more rapid change than ever before… … for jobs that have not yet been created… … using technologies that have not yet been invented… … to solve problems that we don’t yet know will arise

… Provide a basis for policy dialogue and global collaboration in defining and implementing educational goals, policies and practices – Show countries what achievements are possible – Help governments set policy targets in terms of measurable goals achieved elsewhere – Gauge the pace of educational progress – Facilitate peer-learning on policy and practice .


PISA 2009 in brief

PISA countries in 2001 2003 2000 2009 2006 1998

Key principles

Andreas Schleicher Warsaw, 10 February 2011

OECD Programme for International Student Assessment

PISA What students know and can do

44

world economy ‘CrowdCoverage sourcing’ andof collaboration

83% 81% 77% 86% 85% 87%

– PISA draws together leading expertise and institutions from participating countries to develop instruments and methodologies… … guided by governments on the basis of shared policy interests 

Cross-national relevance and transferability of policy experiences – Emphasis on validity across cultures, languages and systems – Frameworks built on well-structured conceptual understanding of assessment areas and contextual factors

Triangulation across different stakeholder perspectives – Systematic integration of insights from students, parents, school principals and system-leaders

Advanced methods with different grain sizes – A range of methods to adequately measure intended constructs with different grain sizes to serve different decision-making needs – Productive feedback, at appropriate levels of detail, to fuel improvement at multiple levels .


Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do OECD Programme for International Student Assessment

55

What 15-year-olds can do


Shanghai-China

High reading performance

Andreas Schleicher Warsaw, 10 February 2011

Singapore New Zealand Japan Australia Belgium Poland, Switzerland United States Germany, Sweden France, Ireland Hungary, United Kingdom Macao-China Slovenia

OECD Programme for International Student Assessment

PISA What students know and can do

66

Slovak Republic, Czech Republic Luxembourg, Israel Austria Dubai (UAE)

Average performance of 15-year-olds in 540,000 Korea reading – extrapolate Finland Hong Kong-China and apply Canada 520,000

Netherlands Norway , Estonia Iceland 500,000 Liechtenstein Chinese Taipei Denmark Portugal Italy Latvia Greece 480,000 Spain Croatia

Lithuania Turkey 460,000

Russian Federation

Chile Serbia

440,000 55

45

35

25

‌ 17 countries perform below this line

Low reading performance


High reading performance High average performance

Andreas Schleicher Warsaw, 10 February 2011

Large socio-economic disparities

OECD Programme for International Student Assessment

PISA What students know and can do

77

Average performance 15-year-olds Highof average performancein science – extrapolate High social equity and apply

Strong socioeconomic impact on student performance

Socially equitable distribution of learning opportunities

Low average performance

Low average performance

Large socio-economic disparities

High social equity

Low reading performance


High reading performance

2009

High average performance

Andreas Schleicher Warsaw, 10 February 2011

Large socio-economic disparities

OECD Programme for International Student Assessment

PISA What students know and can do

88

Durchschnittliche High average performance Sch端lerleistungen im High social equity Bereich Mathematik

Strong socioeconomic impact on student performance

Socially equitable distribution of learning opportunities

Low average performance

Low average performance

Large socio-economic disparities

High social equity

Low reading performance


School performance and socio-economic background Poland Private school Public school in rural area Public school in urban area

Score 700

Student performance

Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

13 13

593

300 -2

Disadvantage

-1

0

1

PISA Index of socio-economic background

2

Advantage


PISA What students know and can do % 80

70

10

0

Shanghai-China Hong Kong-China Korea Macao-China Singapore Finland Japan Turkey Canada Portugal Chinese Taipei Poland New Zealand Spain Liechtenstein Estonia Netherlands Italy Switzerland Latvia Australia OECD average France Belgium Ireland Iceland Mexico United States Greece Thailand Croatia Tunisia Norway Hungary Sweden Slovenia Indonesia Denmark Chile United Kingdom Israel Colombia Germany Brazil Czech Republic Slovak Republic Luxembourg Lithuania Austria Russian Federation Trinidad and Tobago Uruguay Serbia Jordan Albania Argentina Dubai (UAE) Romania Bulgaria Panama Montenegro Kazakhstan Peru Azerbaijan Qatar Kyrgyzstan

60

50

40

Andreas Schleicher Warsaw, 10 February 2011

30

20

OECD Programme for International Student Assessment

14 14

Percentage of resilient students among disadvantaged students Resilient student: Comes from the bottom quarter of the socially most disadvantaged students but performs among the top quarter of students internationally (after accounting for social background)

Less than 15% resilient students among disadvantaged students

More than 30% resilient students among disadvantaged students Between 15%-30% of resilient students among disadvantaged students


Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do OECD Programme for International Student Assessment

15 15

Quality differences between schools


20

0

20

Argentina Trinidad and Tobago Italy Qatar Turkey Bulgaria Israel Panama Germany Peru Hungary Dubai (UAE) Austria Belgium Luxembourg Netherlands Japan Chile Uruguay Greece Brazil Czech Republic Slovenia Romania Croatia Serbia United States Mexico Singapore Jordan Kyrgyzstan Colombia Montenegro Hong Kong-China Albania Tunisia Slovak Republic Liechtenstein Kazakhstan Macao-China Ireland United Kingdom Chinese Taipei Korea Switzerland Australia New Zealand Portugal Shanghai-China Azerbaijan Russian Federation Canada Sweden Lithuania Indonesia Spain Poland Estonia Latvia Iceland Thailand Denmark Norway Finland

Andreas Schleicher Warsaw, 10 February 2011

What students know and can do PISA Variance 40

60

80

100

OECD Programme for International Student Assessment

17 17

Variability in student performance between and within schools

100

80

60

40

Performance differences between schools

Performance variation of students within schools


Policy

Policies and practices

R

R System

E School

Equity

Andreas Schleicher 7 December 2010

Learning climate

OECD Programme for International Student Assessment

PISA What students know and can do

18 18

Discipline

Teacher behaviour

Parental pressure

Teacher-student relationships

Dealing with heterogeneity Grade repetition

 

Prevalence of tracking

Expulsions

Ability grouping (all subjects)

Standards /accountability Nat. examination


Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do OECD Programme for International Student Assessment

19 19

Does it all matter?


Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

20 20

Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada) after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1) Odds ratio higher education 20 entry

18 16 14 12 10 8 6 4 2 0

Age 19 Age 21 Age 21

Level 2

Level 3

Level 4

Level 5


How poor skills raise the risk of economic and social disadvantage (16-65 year-olds)

Odds ratios

Andreas Schleicher Warsaw, 10 February 2011

Increased likelihood of failure

OECD Programme for International Student Assessment

PISA What students know and can do

21 21

In lowest two quintiles of personal income

3,5

3,0

Unemployed 2,5

2,0

Received social assistance in last year

1,5

1,0

0

1

2

3

4

Number of skills domains with low performance

Did not receive investment income in last year


Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do OECD Programme for International Student Assessment

22 22

What does it all mean?


A commitment to education and the belief that competencies can be learned and therefore all children can achieve Universal educational standards and personalisation as the approach to heterogeneity in the student body… … as opposed to a belief that students have different destinations to be met with different Lessons from PISA expectations, and selection/stratification as the approach to heterogeneity on successful  Clear articulation who is responsible for education systems ensuring student success and to whom

Andreas Schleicher Warsaw, 10 February 2011

OECD Programme for International Student Assessment

PISA What students know and can do

25 25


High reading performance

2009

High average performance

Andreas Schleicher Warsaw, 10 February 2011

Large socio-economic disparities

OECD Programme for International Student Assessment

PISA What students know and can do

27 27

Durchschnittliche High average performance Sch端lerleistungen im High social equity Bereich Mathematik

Strong socioeconomic impact on student performance

Socially equitable distribution of learning opportunities

Low and average performance Early selection institutional differentiation Large socio-economic disparities

Low average performance

High degree of stratification Low degree of stratification

Low reading performance

High social equity


Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

28 28

Clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)  High level of metacognitive Lessons from PISAcontent of instruction 

on successful education systems


Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

29 29

Capacity at the point of delivery Attracting, developing and retaining high quality teachers and school PISA leaders and a work Lessons from organisation in which they can use their on successful potential  Instructional leadership and human resource education systems management in schools  Keeping teaching an attractive profession  System-wide career development 


Incentives, accountability, knowledge management

Andreas Schleicher Warsaw, 10 February 2011

OECD Programme for International Student Assessment

PISA What students know and can do

30 30 Aligned incentive structures For students 

How gateways affect the strength, direction, clarity and nature of the incentives operating on students at each stage of their education Degree to which students have incentives to take tough courses and study hard Opportunity costs for staying in school and performing well

Lessons from PISA For teacherson successful Make innovations in pedagogy and/or organisation systems Improveeducation their own performance 

 

and the performance of their colleagues Pursue professional development opportunities that lead to stronger pedagogical practices

A balance between vertical and lateral accountability Effective instruments to manage and share knowledge and spread innovation – communication within the system and with stakeholders around it A capable centre with authority and legitimacy to act


How much autonomy individual schools have over resource allocation

Andreas Schleicher Warsaw, 10 February 2011

Selecting teachers for hire, OECD average Poland

OECD Programme for International Student Assessment

PISA What students know and can do

31 31

Only "regional and/or national education authority"

Firing teachers, OECD average Poland Establishing teachers’ starting salaries, OECD average Poland

Both "principals and/or teachers" and "regional and/or national education authority"

Determining teachers’ salaries increases, OECD average Poland Formulating the school budget, OECD average Poland

Only "principals and/or teachers"

Deciding on budget allocations within the school, OECD average Poland

0%

20%

40%

60%

80% 100%


How much autonomy individual schools have over curricula and assessment

Andreas Schleicher Warsaw, 10 February 2011

Establishing student assessment policies, OECD average

OECD Programme for International Student Assessment

PISA What students know and can do

32 32

Only "regional and/or national education authority"

Poland

Choosing which textbooks are used, OECD average Poland

Both "principals and/or teachers" and "regional and/or national education authority"

Determining course content, OECD average Poland

Only "principals and/or teachers" Deciding which courses are offered, OECD average

Poland

0%

20%

40%

60%

80%

100%


School autonomy, accountability and student performance

Impact of school autonomy on performance in systems with and without PISA score in reading

accountability arrangements

Andreas Schleicher Warsaw, 10 February 2011

500

OECD Programme for International Student Assessment

PISA What students know and can do

33 33

495 490

School autonomy in resource allocation Schools with more autonomy 480

Schools with less autonomy Systems with more accountability

Systems with less accountability

System’s accountability arrangements


Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

34 34

Local responsibility and system-level prescription Trend in OECD countries

System-level prescription ‘Tayloristic’ work organisation

Schools today The industrial model, detailed prescription of what schools do

Schools tomorrow?

Building capacity

Finland today Every school an effective school

Schools leading reform Teachers as ‘knowledge workers’


PISA What students know and can do

35 35

Public and private schools Observed performance difference

Government schools

Government dependent private

Difference after accounting for socio-economic Government independent private background of students and schools -150 -100 -50 0 50 100 0 20 40 60 80 100

%

Score point difference

OECD Programme for International Student Assessment

Andreas Schleicher Warsaw, 10 February 2011

Australia Austria Canada Chile Czech Republic Denmark Estonia Finland Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Argentina Brazil Hong Kong-China Indonesia Jordan Russian Federation Shanghai-China Singapore Chinese Taipei

Private schools perform better Public schools perform better


Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

36 36

Lessons from PISA on successful education systems

Investing resources where they can make most of a difference 

Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms) Effective spending choices that prioritise high quality teachers over smaller classes


Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do

37 37

A learning system

OECD Programme for International Student Assessment

An outward orientation of the system to keep Lessons from PISA the system learning, international benchmarks as the ‘eyes’ and ‘ears’ on of the system successful Recognising challenges and potential future education systems threats to current success, learning from them, designing responses and implementing these


Andreas Schleicher Warsaw, 10 February 2011

OECD Programme for International Student Assessment

PISA What students know and can do

38 38  Coherence of policies and practices 

 

Alignment of policies across all aspects of the system Coherence of policies over sustained periods of time Consistency of implementation Fidelity of implementation (without excessive control) from Lessons

PISA on successful education systems


Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do OECD Programme for International Student Assessment

39 39

Beyond schooling


Score point difference

-10 Qatar

Panama

Italy

Chile

New Zealand

Hungary

Portugal

Macao-China

Korea

Hong Kong-China

Croatia

60

Denmark

Germany

Lithuania

Andreas Schleicher Warsaw, 10 February 2011

PISA What students know and can do OECD Programme for International Student Assessment

40 Parental support at the beginning of 40 primary school

Score point difference between students whose parents often do (weekly or daily) and those who do not:

"talk about what they had done"

50

40

30

20

10

0


OECD Programme for International Student Assessment

120

20

0

Israel Singapore Belgium Qatar Macao-China Italy France Hong Kong-China Switzerland Denmark United Kingdom Liechtenstein Dubai (UAE) Greece Kyrgyzstan Uruguay Argentina Shanghai-China Germany Spain New Zealand Australia Slovak Republic Sweden Brazil Hungary Luxembourg Mexico Thailand Trinidad and Tobago Canada OECD average Chinese Taipei Indonesia Poland Iceland Kazakhstan Panama Romania Czech Republic Japan Tunisia Peru Austria Jordan Bulgaria Norway Albania Azerbaijan Russian Federation Colombia Portugal Chile United States Lithuania Turkey Serbia Montenegro Netherlands Ireland Slovenia Croatia Finland Korea Latvia Estonia

Andreas Schleicher Warsaw, 10 February 2011

difference pointWhat Score students know and can do PISA

41 41

Beyond schooling

Performance difference between students who had attended preprimary school for more than one year and those who did not

100

80

60

40

Observed performance advantage

Performance advantage after accounting for socio-economic factors


Education reform trajectories

Andreas Schleicher Warsaw, 10 February 2011

The old bureaucratic system

OECD Programme for International Student Assessment

PISA What students know and can do

42 42

Some students learn at high levels

Student inclusion

The modern enabling system

All students need to learn at high levels

Curriculum, instruction and assessment

Routine cognitive skills, rote learning

Learning to learn, complex ways of thinking, ways of working Teacher quality

Few years more than secondary

High-level professional knowledge workers

Work organisation

‘Tayloristic’, hierarchical

Flat, collegial

Accountability

Primarily to authorities

Primarily to peers and stakeholders


High policy value

43 Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

43 Providing countries with effective tools to review the choices and trade-offsMust whichhaves they face as they seek to improve the effectiveness and efficiency of their education systems

Quick wins

Examining individual, institutional and systemic factors associated with quality, equity and efficiency in education Understanding drivers of successful reform trajectories

Monitoring educational progress ‘Democratising PISA’ Extending the range of competencies through which quality is assessed

A brief history of PISA

PISA 2000

More difficult

Less difficult

Proliferation of assessment areas .

Money pits

Electronic delivery of assessments

Measuring student learning outcomes in key subjects

Low-hanging fruits Moderate policy value


High policy value

44 Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

44 Providing countries with effective tools to review the choices and trade-offsMust whichhaves they face as they seek to improve the effectiveness and efficiency of their education systems

Quick wins

Examining individual, institutional and systemic factors associated with quality, equity and efficiency in education Understanding drivers of successful reform trajectories

Monitoring educational progress ‘Democratising PISA’ Extending the range of competencies through which quality is assessed

PISA 2003

More difficult

Proliferation of assessment areas .

Less difficult

Measuring student learning outcomes in Electronic delivery of assessments key subjects and establishing the comparative strengths and weaknesses of education systems

Money pits

Low-hanging fruits Moderate policy value


High policy value

45 Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

45 Providing countries with effective tools to review the choices and trade-offsMust whichhaves they face as they seek to improve the effectiveness and efficiency of their education systems

Quick wins

Examining individual, institutional and systemic factors associated with quality, equity and efficiency in education Understanding drivers of successful reform trajectories

Monitoring educational progress ‘Democratising PISA’ Extending the range of competencies through which quality is assessed

PISA 2006

More difficult

Less difficult

Measuring student learning outcomes in Electronic delivery of assessments key subjects and establishing the comparative strengths and weaknesses of education systems

Proliferation of assessment areas .

Money pits Moderate policy value

Low-hanging fruits


46

High policy value

Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

46 Providing countries with effective tools to review the choices and trade-offsMust whichhaves they face as they seek to improve the effectiveness and efficiency of their education systems

Quick wins

Examining individual, institutional and systemic factors associated with quality, equity and efficiency in education Understanding drivers of successful reform trajectories

Monitoring educational progress ‘Democratising PISA’ Extending the range of competencies through which quality is assessed Affective dimensions of outcomes Assessment of digital literacy More difficult

Proliferation of assessment areas

Money pits Moderate policy value

PISA 2009

Less difficult

Measuring student learning outcomes in Electronic delivery of assessments key subjects and establishing the comparative strengths and weaknesses of education systems

Low-hanging fruits


Andreas Schleicher Warsaw, 10 February 2011 OECD Programme for International Student Assessment

PISA What students know and can do

47 47 Find out more about PISA at…  OECD www.pisa.oecd.org – All national and international publications – The complete micro-level database 

U.S. White House www.data.gov

Email: Andreas.Schleicher@OECD.org

Thank you !

… and remember: Without data, you are just another person with an opinion


PISA 2009 - Warsaw - Poland 10 of February 2011