Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

Date: July 2016 Team: Academics, Bridge International Academies Acknowledgments: Special thanks to:

Submitted by:

https://newglobe.education/

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

2

Contents 1. Introduction 2. Summary Statistics

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

Page 04 Page 05

3

Introduction This memo summarizes the results of a pilot personality test, administered to retroactively study the relationship between a personality test and pupil growth (noted throughout this memo as “PGM”).

While the retroactive nature of this study presents a large caveat, these initial results are encouraging.

The pilot included 167 Kenyan teachers, and showed that the composite score had a 0.25 bivariate correlation with pupil growth, statistically significant. We are investigating the strength of this relationship for a reduced version of this quiz, for ease of implementation.

Beyond simple correlations, we have found that this composite score (combined with other teacher characteristics) can predict up to 30% of pupil growth variation. More details on the pilot study are included below.

Jonathan Lehe (former intern at Bridge International Academies, and current Masters student at the Harvard Kennedy School of Government) will continue to pursue this work as part of his Masters thesis.

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

4

Summary Statistics Three additional metrics were used to compare different forms of each regression with a teacher quality outcome as the dependent variable and the personality test responses as the independent variables. The three metrics are: 1. R-Squared: The percentage of variation in PGM that is explained by variation in responses to the personality test 2. Average Residual: The average difference between each teacher’s actual PGM score and his/her predicted PGM score (in standard deviations) based on the personality test responses 3. Average Improvement: The improvement in average actual PGM score (in standard deviations) that would be achieved by removing the bottom 20% of teachers according to predicted PGM score

12 forms of each regression were analyzed, with the results reported below for Term 2 PGM score as the dependent variable. These 12 forms include with vs. without control variables, with vs. without teacher diligence variables, a full version of the personality test with all 30 questions vs. a reduced version with only 10 questions, and the full personality test responses vs. a single composite score.

For comparison, the regression with only the control variables had an R-squared of 0.07, average residual of 0.68 SDs, and average improvement of 0.06 SDs, and the regression with only the teacher diligence variables had an R-squared of 0.06, average residual of 0.71 SDs, and average improvement of 0.06 SDs.

Regression Personality test only (30 Qs) Personality test (30 Qs) with controls Personality test (30 Qs) with controls and diligence Personality composite score only (30 Qs) Personality composite score (30 Qs) with controls Personality composite score (30 Qs) with controls and diligence Reduced personality test only (10 Qs) Reduced personality test (10 Qs) with controls Reduced personality test (10 Qs) with controls and diligence Reduced personality composite score only (10 Qs) Reduced personality composite score (10 Qs) with controls Reduced personality composite score (10 Qs) with controls and diligence

0.18 0.30 0.43

Average Residual 0.66 0.57 0.51

Average Improvement 0.12 0.12 0.16

0.06 0.12

0.72 0.68

0.06 0.14

0.19

0.63

0.13

0.13 0.20

0.68 0.64

0.09 0.09

0.30

0.59

0.12

0.11

0.70

0.06

0.17

0.64

0.07

0.25

0.60

0.15

R-Squared

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

5

The regression with the reduced personality test with 10 questions and with responses to all 10 questions considered individually (rather than a single composite score) are noted with “10 Qs”. This form of the regression achieves the optimal balance between maximizing R-squared and Average Improvement, minimizing Average Residual, and minimizing the burden to Ambassadors in the field to collect and analyze the personality test responses.

Question List: Below are the 30 personality test questions, ranked by their correlation with Term 2 PGM (positive or negative), and the 10 questions in the reduced personality test.

Question New ideas and projects sometimes distract me from previous ones People should be allowed to drink or smoke, as long as it doesn't affect others I don't like to work in stressful situations Even if I miss a deadline, I would rather not submit an assignment until it's 100% perfect It bothers me when other people cheat on a test I enjoy trying new things that are outside my comfort zone People should be allowed to try new ideas, even if some of them fail In my opinion, rules often reduce creativity I like to stay busy and active throughout the day When I am in a challenging situation, I like to overprepare in advance I like to think through my answer thoroughly before responding to a question If things aren't going as well as I’d like, I tend to get discouraged I like to make my working environment as neat as possible In my opinion, censorship and rules slow down progress I have a consistent daily routine My interests change from year to year If I am running late to an appointment, I may decide not to go I think the interests of the group are more important than the individual I often set a goal but later choose to pursue a different one I like old traditions I need every detail to be perfect in order to work well I like thinking about the small details of a project I believe there is no absolute right and wrong I work better at certain times of the day than others

Correlation with Term 2 PGM

Included in Reduced Personality Test

+0.14

Yes

+0.14

Yes

-0.13

Yes

+0.12

Yes

+0.12

Yes

+0.11

No

+0.10

Yes

-0.10 +0.09

Yes Yes

+0.08

No

-0.08

Yes

-0.07

No

+0.05

No

-0.05

No

+0.05 -0.05

Yes No

-0.04

No

+0.04

No

+0.03

No

-0.03 -0.03 -0.03 -0.02 +0.02

No No No No No

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

6

I become involved in new pursuits or projects every few months I have achieved a goal that took years of work It's important to me that all people get the same opportunities My friends say I am unpredictable I am known to make quick decisions It bothers me when people arrive late to events

+0.02

No

+0.02

No

-0.01

No

-0.01 +0.01 0.00

No No No

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

7

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

Kenya Personality Test Pilot: Summary Statistics and Future Enhancements

8