II Workshop Internacional - EDI Information Community Planning Evaluation Policy ECD

The EDI-Informing community planning, evaluation and policy for ECD outcomes Sally Brinkman: Chief Scientific Officer – Int. EDI Consortium and National Australian EDI Epidemiologist.

Eric Duku: Chief Statistician – Int. EDI Consortium and Chief Statistician – Canadian EDI

“A large number of children at a small risk for school failure may generate a much greater burden of suffering than a small number of children with a high risk�

(Based on Rose 1992, Offord et al. 1998)

What is the EDI? • The Early Development Instrument (EDI) is a community measure of young children’s development.

• The EDI Checklist consists of around 100 questions completed by the child’s teacher based on their knowledge of the child

• It is designed for use with whole populations of children (based on geographical or administrative boundaries) • The checklist was originally designed in Canada to measure child development between 4 and 6 years of age.

What is the EDI • The EDI measures 5 developmental domains: – Physical health and well-being – Social competence – Emotional maturity – Language and cognitive development.

– Communication skills and general knowledge.

The EDI is a measure of how well the community has raised their children before school. It is an index of the outcomes of early child development and of children’s ability to take advantage of the learning environments offered to them by schooling

Why a community or population approach?

• It is important to understand and think about the social and environmental influences on child development • Moving the focus of effort from the individual to the community can make a bigger difference • Allowing communities the opportunity to “shift the curve” of a whole population and therefore improve outcomes for many children

Number of Children

EDI Population curve Children developmentally Vulnerability cut-off vulnerable

Children performing well

Average score

Low

Outcome domain

High

Universal strategies-entire population moves

% of Children

Targeted strategies decrease numbers of vulnerable Gap narrows

Low

Outcome measure

High

Uses of the EDI •

To report on the early childhood development outcomes for whole of populations of children, small geographic areas and special populations

•

To measure progress over time in improving early childhood development outcomes

•

To facilitate community mobilisation around early childhood development through the AEDI implementation process and results

•

To enable schools and teachers to look back at the development of children before entering school and look forward to children’s needs once at school

•

To make state and territory, national and international comparisons

How the AEDI findings are used Local

• Stimulate discussions about early child development among teachers, parents, schools, community groups, and policymakers

Regional

• Identify communities and neighborhoods where children may be at risk developmentally • Plan and evaluate ECD initiatives

State National

• Document the effect, efficacy, and cost-effectiveness of ECD programs.

• Leverage better-informed ECD policies • Match programs with investment opportunities.

Case Studies: Australia Indonesia Canada Jamaica Mozambique Kosovo

Australia

Understanding Early Childhood Development in Australia • Before the AEDI we had no way of consistently understanding how children were developing across Australia as they came to the end of their pre-school years • 2002-2003 North Metro Perth EDI Study (2002 n=200, 2003 n=4500) • 2003 National meeting of experts to consider whether Australia needed an individual measure or a population measure of ECD. Resounding endorsement for a population measure • 2004-2007 Development and piloting of the AEDI, overwhelming success for communities (n=40,000) • 2007-2008 Agreement by COAG for the AEDI as a national progress measure of ECD for Australia • 2009

National AEDI Census in recognition of the need for all communities to have local ECD information to improve outcomes for children

National AEDI Results • 24% of Australian children surveyed were “developmentally vulnerable” on one or more domains of the AEDI 11.9% of Australian children surveyed were “developmentally vulnerable” on two or more domains of the AEDI 67.9% of Australian children surveyed were “performing well” on one or more domains of the AEDI

% Vulnerable on at least one area of ECD

Australia – AEDI Children 5-6 yrs.

30

20

10 Highest 10%

10% -25%

25% -50%

50% -75%

SEIFA disadvantage

75% -90%

Lowest 10%

“Janus” communities – exceptions to the rule (healthy deprived areas & unhealthy wealthy areas)

66th %ile

Social disparity increases over time As children move from year 3 to year 5, the disparity among those meeting literacy standards grows 100 Socioeconomic Status:

Proportion Meeting Standard (%)

90 80 The Wide ning Gap

70 High

60

Low

50 40 30 20 10 0 Year 3

Year 5

WALNA Data, DoE 2006

Map 15: Level of disadvantage according to the SEIFA index

Coolbellup Coolbellup Hamilton Hamilton Hill Hill

Bibra Bibra Lake Lake Spearwood Spearwood

South South Lake Lake

SEIFA Index of disadvantage 1. Most disadvantaged 2.

3.

4.

5. Least disadvantaged

Coogee Coogee

Yangebup Yangebup

Beeliar Beeliar Success Success

SEIFA is derived indicators such as low income and educational attainment, high unemployment, jobs in relatively unskilled occupations and variables that reflect disadvantage. The state quintile categories are referenced to state. City of Cockburn, WA

Atwell Atwell

Prepared by: AEDI National Support Centre Source: ABS Census 2001 Basic Community Profile

Proportion of children vulnerable on one or more domains

Coolbellup Coolbellup Hamilton Hamilton Hill Hill

Spearwood Spearwood

Bibra Bibra Lake Lake

South South Lake Lake

Proportion of children vulnerable

Coogee Coogee

N=Percent 49.1 40 33.3 18.9 11.5

Yangebup Yangebup

to to to to to

53.3 49.1 40 33.3 18.9

Beeliar Beeliar Success Success Atwell Atwell

City of Cockburn, WA

Prepared by: AEDI National Support Centre Source: AEDI Communities Data 2006

Participation in preschool in the year before entering school

Coolbellup Coolbellup Hamilton Hamilton Hill Hill

Bibra Bibra Lake Lake Spearwood Spearwood South South Lake Lake

Preschool participation N=Percent 38.6 48.1 64.3 83.9 90.9

Coogee Coogee

to to to to to

48.1 64.3 83.9 90.9 96.3

Yangebup Yangebup

Beeliar Beeliar Atwell Atwell Success Success

Proportion of children that attended preschool program including in a day care centre .

City of Cockburn, WA

Prepared by: AEDI National Support Centre Source: AEDI Communities Data 2006

AEDI Summary Table Community report – summary table

Proportion of children developmentally vulnerable Suburb

No

Phys

Atwell

121

4.3

Beeliar

57

Bibra Lake

Soc

Average scores

Emo

Lang

Com

Vul 1

Vul 2

Phys

Lang

Com

12.1

9.5

9.5

9.5

20.7

10.3

9.09

8.96

8.65

8.72

8.75

16.4

29.1

20.0

26.0

29.1

49.1

32.7

8.50

8.13

7.12

7.02

6.25

55

1.9

3.8

3.8

11.3

1.9

18.9

3.8

9.55

8.96

8.60

7.69

8.75

Coogee

28

11.5

3.8

0.0

11.5

3.8

11.5

11.5

9.77

9.06

8.65

7.31

10.00

Coolbellup

17

26.7

20.0

13.3

20.0

26.7

53.3

20.0

7.73

8.96

8.46

7.31

6.25

Hamilton Hill

42

23.1

17.9

17.9

15.4

12.8

33.3

20.5

9.09

8.75

7.31

8.08

7.50

South Lake

50

8.3

8.3

14.6

27.1

12.5

35.4

20.8

9.09

9.17

8.46

7.12

8.44

Success

74

2.9

10.0

7.1

13.0

10.0

21.4

11.4

9.09

8.75

7.50

7.69

8.75

Yangebup

62

10.0

11.7

16.9

25.0

21.7

40.0

28.3

8.64

7.60

7.69

6.92

6.88

Abbreviations used: Phys Physical health and wellbeing Soc Social competence Emo Emotional maturity Lang Language and cognitive skills Com Communication and general knowledge Vuln1 Vulnerable on one or more domains Vuln 2 Vulnerable on two or more domains No Total number of children surveyed

Soc

Emo

Prepared by: AEDI National Support Centre Source: AEDI Communities Data 2006

Mobilizing community action for children

The AEDI community planning process 1. Identifying areas of particular need

2. Assessing the local distribution of children’s developmental vulnerability

e.g. Mission Australia funds 3 year play group, language program & mums group at school

3. Community asset mapping

4. Mobilising community action

Governance – National AEDI Census COAG

National AEDI Partnership (CCCH/TICHR) Ultimate responsibility for the AEDI Program

DEEWR

McMaster Int. EDI Consortium

National AEDI Support Centre (NSC) Mapping, analytical, database (ACER), overall project management and support to states State AEDI Coordinators Key tasks: Provide state wide leadership, engage key stakeholders and support communities, schools and teachers to successfully implement the AEDI Resources: State AEDI Coordinator Training and ongoing support from AEDI NSC

Local AEDI Coordinators Key tasks: Raise awareness of the AEDI in the community Resources: Community Preparation & Implementation Guide Training for communities Community Strategies Toolkit

Schools and teachers Key tasks Teachers complete AEDI Training (1 hour) and AEDI Checklist Principals invoice for Teacher Relief Resources: AEDI Guide for Teachers AEDI Teacher Training AEDI Guide for Principals

Indonesia

ECED Project Challenges • 12.7 Million (US$) – 2013 (loan) • Beneficiaries: 738,000 children ages 0 to 6 in 6,000 poor communities (dusuns) in 3,000 villages within 50 districts • an average of 123 children per dusun

Study Objectives •

• • •

•

To what extent the ECED project improves children's development, attendance and readiness for school To what extent the ECED project improves parents awareness and practices To what extent the ECED project increases the availability and utilisation of ECED services How do these impacts differ by gender, wealth, and level of service delivery at baseline What factors contribute to success and failures of the ECED intervention

Challenges • Community driven project • Research design Three components: – Increasing the delivery of ECED services in targeted poor communities using a Community Development district based approach – Developing a sustainable system for ECED quality – Establishing effective program management, monitoring, and evaluation

• Village characteristics • Household survey:

Data to be collected

– Household economy – Parent background – Parent knowledge, attitudes, beliefs child rearing

• Child outcome data : indicators of child development across domains

Instruments • Philippines ECD study (developmental milestones, anthropometrics, nutrition) • IFLS (Rand) • SDQ • EDI

Canada

“EDI data in conjunction with other data can be used to create, maintain, and monitor community support for programs and policies affecting young children…. Analysis can increase public understanding of the factors which contribute to early child development, inspiring a commitment to fundraising, policy development and other initiatives.”

Source: Canadian Association of Principals – Student Readiness to Learn and the School Ready to Teach: an Internet Essay and Collection of Selected Documents: www.schoolfile.com/cap_start/schoolready.html (2003 )

Canadian Normative Data Collected from 2004/5 to 2006/7 (n=176,621) • Percent of children scoring low on 1 more domains = 27% • Percent of children scoring low on 2 or more domains = 13.6%

Factors increasing the vulnerability risk Odds ratios • • • • • • •

Child health (low) Gender (boy) Income (low) Family status (not intact) Age (younger half) Literacy (looking at books) Parent smoking Source: Janus & Duku 2007

2.35 2.32 2.02 1.83 1.36 1.35 1.29

Important neighbourhood variables • Parent-reported neighbourhood quality (includes playground, safety, health, transport, presence of families with children etc.) • Parent-reported frequency of contacts with neighbours (talking, visiting)

Source: NLSCY/UEY/EDI 1998/9

Canada: % vulnerable by SES 60

50

%

40

31.9

29.1 23.1

30

13.7

20

10

0 very poor

poor

Source: NLSCY/UEY 1999-2000; EDI 1999-2000

not poor

well-off

Toronto

Survey of response to EDI results in Ontario and Manitoba • 68.4% report changes have been made • Areas of most frequent change: Areas of least frequent change: Parenting Programs non-profit recreation behaviour programs in schools programs child and family centres ESL programs for children library programs speech/language programs • Educational programs most likely to be implemented • Low school readiness associated with higher percentage of Child & Family Centers, and higher percentage of programs aimed at Aboriginal children with special needs

Kosovo

Kosovo • In collaboration with World Bank • ECD Programme Evaluation Objectives • Assessing using the EDI, whether children who attended ECD centres were more advantaged developmentally over those who did not attend ECD centres

• To gather information on the households of children in the main survey for the assessment of correlates of EDI domains and outcomes

Kosovo Data Collection • 1st wave – Children from 9 municipalities – Teachers filled out forms – Test-retest on smaller sample of 51 ECD children from 2 communities

• 2nd wave – 111 children selected at random using proportional allocation to ensure representativeness from all 11 communities

Kosovo Instruments used • EDI (teacher filled): short version developed in collaboration with the World Bank; translated into Albanian • Teacher demographics • Household Form (2nd wave) • Parent interview

Kosovo – sample descriptives group

Total (n=460)

Variables

categories

ECD (n=219)

non-ECD (n=241)

Gender

Females

42.9%

50.2%

46.7%

Attend pre-school

Yes

3.7%

0.0%

1.7%

Class child attends

One

95%

100%

97.6%

Age

based on EDI

6.71 (0.59)

6.96 (0.60)

6.84 (0.61)*

Kosovo: % vulnerable by SES 60

56.3 43.2

50

40 30.8

%

40 30 20 10 0

Cat 1

Cat 2

Cat 3

Cat 4

Jamaica

Jamaica • In collaboration with World Bank • Part of process for establishing a national monitoring system of children from pre-school through Grade 3 • Over 2,000 schools in Jamaica • School selection using stratified randomization, based on the government’s determinations of rural/urban and small/medium/large.

• All 4 year olds attending the selected schools were included.

• Parental consent eventually determined who participated in the study.

Jamaica Instruments

• Teacher assessments: the EDI (full 2003/4 version) • Parent assessment: Vineland Social-Emotional Early Childhood Scales • Direct assessments : McCarthy Scales of Children’s abilities and PPVT • SES – demographic and household information from parental interview

Jamaica • • • • •

Sample description Sample size, n = 151 % females = 49 # schools = 18 Mean age at completion 4.8 (0.4) Good internal reliability and concurrent validity

JAMAICA - validity

PPVT McCarthy-verbal -perceptual -quantitative -cognitive memory -cognitive general

Language & Communication Cognitive Dev & Gen Knowledge .396 .331 .361 .401 .443

.318 .346 .308

.371 .448

.312 .383

Jamaica % vulnerable by SES 60

50

47.4 34.2

%

40

30

21.1

18.9

20

10

0 Q1

Q2

Q3

Q4

Mozambique

Mozambique • • • • • •

In collaboration with World Bank Pre-pilot Full version of EDI adapted for use Translation into Portuguese Test retest Items removed because…

Mozambique – sample descriptives • • • • • •

Sample size, n = 78 Mean age = 6.3 years % females = 55.1 # schools = 4 Teachers Good test-retest reliability

Early Development Instrument (EDI) Versão em Português para Moçambique Portuguese version for Mozambique Numero de identificação da escola-Número do aluno Copie o número do aluno segundo a marcação após a questão ST20 no questionário para o professor P[__][__] D[__][__] PA[__][__] L[__][__]APC[__][__] UR[__]AE[__][__][__]NAF[__][__][__] ESCOLA [__][__] ALUNO [__][__] Nome da escola: _____________________________________________ Nome do professor: __________________________________________ Nome da criança: ____________________________________________ Data de nascimento da criança: Dia _____/ Mes______/Ano Sexo da criança: M F Data em que completou o questionário : Dia _____/ Mes______/Ano A criança possui necessidades educativas especiais ?

Sim Não Se sim, por favor explique: _________________________________________________________________ _________________________________________________________________ ______________ Língua Materna da criança Changane Chope Ronga Outra? ______________________ A criança se comunica bem em sua língua materna?

Sim Não Não sabe

Reliability and Validity

Reliability and validation studies • Teacher to parent inter rater reliability • Teacher to teacher aid inter rater reliability • Repeat testing intra rater reliability • Construct and concurrent validity • Predictive validity • Rasch psychometric property analyses • Indigenous and minority culture validation studies Publications downloadable from: www.australianedi.org.au and www.offordcentre.com

Adaptation Process

Adaptation • Ethnographic scan (age of schooling etc etc)

• Content validity • Translation / back translation • Translation of the teacher guide • Adaptation to local language, context, deletion/addition of items (cultural relevance, behaviour constructs) • Pilot implementation (inter-rater reliability, test-retest, validity assessment with other instrumentation/assessment) • License agreement with McMaster Uni, Offord Centre.

EDI vs. Traditional Evaluation tools

EDI • Impacts at multiple levels of the ecological model • Can be used as an outcome and predictor • The data is not to be owned by one organisation and as information is aggregated it can be freely shared amongst varying sectors

• When the data is mapped it provides an easily understandable dissemination strategy that facilitates community engagement and thus mobilisation • Communities use the results together with other information such as national and local statistics, community, family, and service provider feedback to develop strategic plans.

EDI

• The data prompts service providers to review existing services and programs and to consider new initiatives that might be needed.

• Prompts universal service provision with the aim to better all children while also reducing future inequality.

• Provides a means to evaluate community development initiatives.

• As more countries utilise the EDI, in the future we will be able to make international, national, jurisdictional and local level comparisons • Quick, cheap • Can be completed by teachers, child care workers or parents - as not a diagnostic does not require referral to allied/medical professionals

• Eric Duku – duku@mcmaster.ca • Sally Brinkman – sallyb@ichr.uwa.edu.au

• www.offordcentre.com/readiness • www.australianedi.org.au • http://www.councilecd.ca/internationaledi/