Assessing Social-Emotional Learning: A Guide for the Perplexed

Assessing Social-Emotional Learning: A Guide for the Perplexed Chase Nordengren Research Scientist, NWEA

About NWEA • Assessment solutions provider • Measures of Academic Progress (MAP) – growth, skills, reading fluency • Nine million+ students, 8500 districts • Professional Learning • Research services

What are the top socialemotional learning priorities at your school?

5 minutes with a nearby partner

What Teachers in One District Picked •Attention

•“Connection to school”

•Happiness

•(Feeling of) Safety

•Attitude toward learning •Diversity

•Intrinsic motivation

•Self-efficacy

•Awareness of one’s own

•Emotion management

•Maturation

•Self-image

emotions

•(Relationships with)

•Mental health and state

•Sense of control

•Behavior

Family members

of mind

•Social media use

•Communication

•Feeling accepted

•“Ownership and

•Stress

strategies

•(Relationships with)

accountability”

•(Relationship with)

•Confidence

Friends and peers

•Positive mindset

Teachers and adults

•Conflict resolution

•Future aspirations

•Readiness to learn

Common Threads • Important

Common Threads • Important • Lots of different themes

Common Threads • Important • Lots of different themes • Impact learning … and so much more

The field urgently requires much greater clarity about how well, at present, it is able to count some of the things that count.

Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. https://doi.org/10.3102/0013189X15584327

Outline • Choices in picking an SEL assessment • Our study – Student Engagement Instrument • Best practice in choosing an SEL instrument

Choices, Choices

Big Picture Considerations • Rigor and quality • CASEL Assessment Guide • School Climate Survey Compendium

measuringsel.casel.org

Big Picture Considerations • Rigor and quality • CASEL Assessment Guide • School Climate Survey Compendium • Cost – financial, staff, time • Access to data

Who should take the assessment Students

Teachers (Observation / Opinion)

• Direct window to mindsets

• Can apply strong professional judgement

• May reveal gaps in teachers’ knowledge of students • May cause discomfort or raise parent concerns

• More complex items / assessments • Bias and equity concerns

(Duckworth & Yeager, 2015; McKown, 2017; Wallace et al., 2016)

How to measure Survey Items

Performance Task

• Simple to administer

• More “authentic” demonstration of application

• More straightforward interpretation • Subject to reference bias, social desirability bias

• May be too removed from day-today practice • Still rare and expensive (Duckworth & Yeager, 2015; West et al., 2016)

What to assess (some considerations) • District / school standards – a “crazy quilt” (McKown, 2017) • Student age and development • College and career readiness • What’s being taught

•

Who should take it?

•

•

How should it measure?

•

•

Students or teachers?

Self-report or performance task?

What should I assess?

•

School, district, community needs?

5 minutes with your partner

SEI: A Case Study

Student Engagement Instrument • Who: Students (grades 3-12) • How: Survey items (33) • What: • Cognitive engagement (feelings about school / future) • Affective engagement (relationships w/ peers, family, teachers)

Example Items • My family/guardian(s) are there for me when I need them.

• Overall, my teachers are open and honest with me.

• My education will create many future opportunities for me.

• Most of what is important to know you learn in school.

Study Design • Given to 3800 students (grades 3-8) 3x • Interviewed 14 teachers, 8 admins • We studied: • Use as an early indicator • Variation across year • Use by teachers and administrators

Relationship to Achievement

Change over Time

Attendance

Barriers to Use by Teachers • Developmental appropriateness in lower grades • Interpreting scores • Aligning measure to curricular focus • Lack of time and tools

How did administrators use it?

• Program placement • Meet mandates • (Limited) work with individual students

• “Having this data piece is just another layer of help”

• Who needs the data from my assessment?

• What do I want them to do with it?

• How can I support them?

5 minutes with your partner

Conclusions

Setting the right stakes • SEL assessments are not for accountability • Unlikely to predict other outcomes • Can support placement and resource allocation • Can provide formative support for SEL interventions

Meeting the right needs • What mandate(s) does the assessment need to meet?

• What does our community want to know?

• What data do our current programs need?

• Placement? Content selection? Effectiveness?

Linking assessment to action • Can students complete the assessment accurately?

• Age groups? Cognitive complexity? SpEd needs?

• How will students develop skills / mindsets?

• Do our teachers have the resources (time, money, training, community) to use this new data?

Chase Nordengren Research Scientist, NWEA chase.nordengren@nwea.org