Page 1

Normalizing Data for Identification of Gifted Students A Recommendation for the Standardization of Gifted Identification with an Identification Matrix CD The measures used for the identification of gifted students vary from state to state and even from school district to school district. During my teaching career I devised a method that I believe more accurately evaluates the multiple types of data used to identify gifted students. Part I of this book provides a summary of the problems inherent in using a gifted identification matrix based on test score percentiles and matrix points. Part II describes an alternative method for evaluating students. Part III gives step-by-step instructions on implementing this alternative identification method using the CD ROM provided. The final section addresses special cases in gifted identification and how to include data which may form a non-normal distribution. Thanks to my wonderful family for supporting me and inspiring me on this journey!

Sharon Ryan A native of Chicago, Sharon Ryan graduated from DePaul University with a B.S. in Elementary Education and an M.B.A. She also holds a M.A. from Northeastern Illinois University in Gifted Education. Her teaching experience ranges from the primary grades to the college level in both public and private schools. She is currently the gifted coordinator in a Chicago suburban school district and is very active in the Gifted and Talented field. Sharon is a regular speaker for the parent speaker series at Northwestern University’s Center for Talent Development and has also been a speaker at both the Illinois and national gifted conferences.


Table of Contents PART I: PROBLEMS WITH EXISTING IDENTIFICATION MATRICES.........................................................................1 Matrix Points............................................................................................................1 Accounting for Students with the Highest Percentiles.............................................2 Using a Range of Percentiles...................................................................................3 Including the 99% in a Range (Ceiling Scores).......................................................4 Considering the Lowest Scores in a Matrix.............................................................6 A Different Matrix in Every District........................................................................6 PART II: A NEW IDENTIFICATION MATRIX....................................................9 The Lowest Scores in a Matrix..............................................................................10 Evaluating Multiple Cycles of Test Data...............................................................11 Identifying Giftedness in a Local Population.........................................................12 Justifying Gifted Programming Using Z-scores.....................................................12 Benefits of Comparing Ability and Achievement Data..........................................13 Standardization of Gifted Identification.................................................................14 Disaggregating Data...............................................................................................14 PART III: USING THE EXCEL SPREADSHEET...............................................16 Choosing Which Tab to Use...................................................................................16 Multiple Test Cycles...............................................................................................16 Double Weighting Scores.......................................................................................18 Screening and Selection of Students......................................................................18 Step-by-Step Instructions for Using the EXCEL Spreadsheet.............................................................................19 PART IV: SPECIAL CASES...................................................................................22 Using Z-scores for Subjective Data or Criterion Based Tests........................................................................................22 Non-normal Distributions......................................................................................23 Using A Teacher Checklist for Nomination of Gifted Students.............................................................................23 Creating a Histogram in Excel using Microsoft Windows (To test for a normal distribution)...............................24 Creating a Histogram in EXCEL using a MAC- Step by Step Instructions (To test for a normal distribution)....................26 Calculating Average and Standard Deviation in EXCEL.............................................................................................29 GLOSSARY...............................................................................................................30 Sample Reports..........................................................................................................31 BIBLIOGRAPHY.....................................................................................................33


PART I: PROBLEMS WITH EXISTING IDENTIFICATION MATRICES Best practice in gifted identification tells us to use as much information as possible when evaluating students for gifted programming. Current national standards for identification of gifted students tell us to use multiple assessments that measure diverse talents and strengths, and as such, a majority of states require the use of multiple criteria in their identification procedures (Houseman, 1987). Standardize testing has historically been a critical component in gifted identification often used as a summative evaluation, conducted once per year. These annual exams, in the form of both achievement and aptitude tests, were used as a snapshot of student ability. In the past, both special education administrators and gifted coordinators used these tests to determine eligibility for services. Recently with changes in special education legislation along with an increased emphasis on accountability in our schools, testing has become a tool for formative instead of summative evaluation. As a result, many school districts have abandoned annual testing in favor of administering multiple test cycles during the school year. Given this change, gifted coordinators who use test data for identification of gifted students must incorporate these multiple cycles of test data into their identification evaluations. The challenge gifted coordinators currently face is how to evaluate multiple cycles and different types of data using gifted identification selection procedures that are statistically accurate.

Matrix Points Most school districts use an identification matrix to evaluate test data for gifted identification. Data are collected from different sources of testing, ability tests and achievement tests, and these data are “normalized” in an identification matrix by awarding matrix points. Matrix points are awarded separately for each test score based on the student’s percentile ranking for each test. These points are then totaled, and gifted identification is assigned to the students with the greatest number of points. Matrix points are used in an identification matrix for two reasons: (1) because percentiles cannot be used in mathematical operations and (2) to ‘normalize’ the data gathered from different tests. Percentiles cannot be used in mathematical operations because a percentile is not the same as a percentage. As teachers we are very adept at averaging percentages when we combine our student’s scores for weekly spellings tests, but arithmetic percentages and standardized test score percentiles are very different measures. Percentages can be averaged because they are a standard, equal interval measure; percentiles, which is the format in which standardized test results are reported, are not the same as percentages and cannot be averaged because percentiles are not equal interval measures. To “normalize” data means to put different types of data on the same scale. Ability and achievement test scores are not on the same scale because ability test scores are in a format that resemble an IQ score. Achievement test score formats will vary depending on the company that administers the test. School districts use matrix points in an attempt to put these two different types of data on the same matrix scale. There are serious problems inherent when using an identification matrix that awards points based on percentile rankings. Five specific problems with using matrix points are documented (1) Students accounting for the highest percentiles (2) Using a range of percentiles in an identification matrix (3) Including the 99th percentile in a range of percentiles (4) Considering the lowest score an a matrix and (5) Having a different matrix in every district.

1


Accounting for Students with the Highest Percentiles ABILITY TEST SCORE Percentile Rank

MATRIX POINTS

ACHIEVEMENT TEST SCORE Percentile Rank

MATRIX POINTS

99

5

99

5

98, 97

4

98, 97

4

96, 95

3

96, 95

3

94, 91

2

94, 91

2

90

1

90

1

89 and below

0

89 and below

0

Figure 1 Ability Score

Ability Test Percentile

Achievement Test Percentile

Total Points

Student 1

140

99%=5 points 95%=3 points

8 points

Student 2

131

97%=4 points 97%=4 points

8 points

Student 3

132

98%=4 points 98%=4 points

8 points

Student 4

127

95%=3 points 99%=5 points

8 points

Student 5

150

99%=5 points 95%=3 points

8 points

Figure 2 Our sample matrix, figure 1, awards the highest scoring students, those in the 99th percentile, five points, and for students with percentiles below the 99th percentile, one additional matrix point is awarded for each two-percentile increase in the student’s test score. Awarding matrix points in this fashion does not accurately account for differences in percentile rank, because, as stated in the online journal of Practical Assessment, Research and Evaluation, “the further a percentile rank deviates from the mean, the more a student’s score must increase for their percentile rank to increase” (Russell, 2000). Sample test data helps to clarify this point, as shown by students 1, 3, and 5 in figure 2. The percentile rank of students in the 99th percentile only differs from students in the 98th percentile by one percentile point, and the matrix recognizes only a one-percentile point difference, but the actual ability scores of these students may vary by as much as eighteen points, as illustrated by the ability score differences between student 3 and student 5. More importantly students with scores beyond the 99th percentile, scores approaching this test’s ceiling of 150, as illustrated by student 5, receive one additional matrix point for their 99th percentile ranking, yet the differences between a ceiling score of 150 and scores of students at the 98th percentile, as illustrated by student 3, could be as significant as eighteen points. Looking at a picture of a standard normal distribution of data helps to illustrate the problem of assigning matrix points to percentile values.

2


Looking at a picture of a standard normal distribution of data helps to illustrate the problem of assigning matrix points to percentile values.

Normal Bell-shaped Curve

Percentage of cases in 8 portions of the curve Standard Deviations -4σ Cumulative Percentages

2.14%

.13% -3σ 0.1%

Percentiles Normal Curve Equivalents

34.13%

13.59%

34.13%

-2σ

-1σ

0

2.3%

15.9%

50%

5

1 10

20

30

40

50

60

.13%

+1σ

+2σ

+3σ

84.1%

97.7%

99.9%

20 30 40 50 60 70 80 90

10

2.14%

13.59%

70

95 80

+4σ

99 90

Figure 3 This image of a standard normal distribution of intelligence data illustrates a distribution of data where results are clustered around the mean. As stated in Kubiszyn and Borich’s book on educational testing “The standard normal distribution is hypothetical. No distribution of scores matches the standard normal distribution perfectly. However, many distributions or scores in education come close to the standard normal distribution and herein lies its value” Kubiszyn, 2007, p. 279). Notice the percentiles in figure 3, which is the second set of scores from the bottom, and observe the distances between the 90th and 95th and the 95th and 99th percentiles. You can easily see that there is a greater distance between the 95th and 99th percentiles than between the 90th to 95th percentiles, which illustrates that percentile ranks are not equal interval measures and should not be awarded incremental values in an identification matrix. The greater distance between the 95th and the 99th percentile shows us that a one-percentile point increase at the highest end of the curve, from the 98th percentile to the 99th percentile, is much more difficult to achieve than a one point percentile increase lower in the curve, from the 90th percentile to the 91st percentile. Going back to our sample matrix figure 1 and considering the increasing difficulty of moving one percentile point at the higher end of the curve, we can see that it is not accurate to award one matrix point when a student moves from the 90th to the 91st percentile and similarly award the same value, one matrix point, for the movement at the highest end of the curve, in the 98th to the 99th percentile, because it is more difficult to move one point at the highest end of the normal curve. In fact using this identification matrix shown as figure 1 puts our highest achieving students, those who have the highest percentiles, at a disadvantage over other lower achieving students, which is exactly the opposite of what we should be doing in our process of identifying students for gifted programming services.

Using a Range of Percentiles For scores below the 99th percentile, our sample matrix figure 1 awarded one additional matrix point for each two-percentile increase in the student’s test score. The problem with an identification matrix that uses a range of percentile ranks is that the student with the lower score in the range of percentiles is unfairly rewarded and receives too many matrix points, while the student with the higher score in the range of percentiles does not receive full credit for his or her performance on the test. 3


ABILITY TEST SCORE Percentile Rank

MATRIX POINTS

ACHIEVEMENT TEST SCORE Percentile Rank

MATRIX POINTS

99

5

99

5

98, 97

4

98, 97

4

96, 95

3

96, 95

3

94, 91

2

94, 91

2

90

1

90

1

89 and below

0

89 and below

0

Figure 1 As illustrated in the sample test data below and using the figure 1 identification matrix, students in the 98th and 97th as well as students in the 95th and 96th percentile, were ranked exactly the same when using an identification matrix that awards points based on a range of percentile values, although the student’s actual test scores varied within this two percentile range, as illustrated in figure 5. Ability Score

Ability Test Percentile

Achievement Test Percentile

Total Points

Student 1

132

98%=4 points 98%=4 points

8 points

Student 2

130

97%=4 points 97%=4 points

8 points

Student 3

129

96%=3 points 96%=3 points

6 points

Student 4

126

95%=3 points 95%=3 points

6 points

Figure 5

Including the 99% in a Range (Ceiling Scores) More troubling are the matrices that include the 99th percentile in the top range. In a different sample matrix shown as figure 6, we see that students in the 99th percentile receive the same matrix points as students in the 95th percentile, which will result in awarding the same matrix points to an even wider range of test scores. Additionally, when the range of percentiles includes the 99th percentile, ceiling scores will also receive the same matrix points as scores that are in the lower end of the 99th percentile. Using the identification matrix in figure 6 and considering sample student data shown in figure 7, we can see that students may have markedly different ability test scores ranging from a score of 126 to 150, yet using the figure 6 matrix these students are awarded the same matrix points for their test scores. ABILITY SCORE Percentile Rank

MATRIX POINTS

ACHIEVEMENT SCORE Percentile Rank

MATRIX POINTS

95-99%

12 points

95-99%

12 points

90-94%

10 points

90-94%

10 points

85-89%

8 points

85-89%

8 points

80-84%

6 points

80-84%

6 points

Figure 6 4


Ability Score

Ability Test Percentile

Achievement Test Percentile

Total Points

Student 1

150

99%=12 points 99%=12 points 24 points

Student 2

132

98%=12 points 98%=12 points 24 points

Student 3

130

97%=12 points 97%=12 points 24 points

Student 4

129

96%=12 points 96%=12 points 24 points

Student 5

126

95%=12 points 95%=12 points 24 points

Figure 7 Some identification matrices include percentile ranks as low as the 90 in their top range of scores. In the identification matrix shown as figure 8 and illustrated with test data shown as figure 9, students in the 90th percentile are awarded the same matrix points as students whose scores are in the 99th percentile, which results in awarding the same number of points to students who have even more discrepant test scores, ranging from an ability score of 121 to an ability score of 150. th

ABILITY SCORE Percentile Rank

MATRIX POINTS

ACHIEVEMENT SCORE Percentile Rank

MATRIX POINTS

90-99%

15

90-99%

15

85-89%

12

85-89%

12

75-79%

8

75-79%

8

74% and below

0

74% and below

0

Figure 8 Ability Score

Ability Test Percentile

Achievement Test Percentile

Total Points

Student 1

150

99%=15 points 99%=15 points 30 points

Student 2

132

98%=15 points 98%=15 points 30 points

Student 3

130

97%=15 points 97%=15 points 30 points

Student 4

129

96%=15 points 96%=15 points 30 points

Student 5

126

95%=15 points 95%=15 points 30 points

Student 6

121

90%=15 points 90%=15 points 30 points

Figure 9

5


Considering the Lowest Scores in a Matrix Every identification matrix has a minimum score for which it awards matrix points. A few school districts have matrices that award matrix points to students in the 50th percentile and below. Most common however are the matrices that award points only to students in the 90th percentile and above. A cutoff score at the 90th percentile seems reasonable until we look again at sample matrix figure 10 and consider samples of student data as shown in figure 11. Considering sample Student 5 in figure 11 whose achievement score is in the 89th percentile, we see that Student 5’s achievement score has ‘fallen off the bottom of the matrix’, because his achievement score is below the 90%. This student has earned only five points in this identification matrix, although his ability score is the highest among all of the sample students. ABILITY SCORE Percentile Rank

MATRIX POINTS

ACHIEVEMENT SCORE Percentile Rank

MATRIX POINTS

99, 98

5

99, 98

5

97, 96

4

97, 96

4

95, 94

3

95, 94

3

93, 92

2

93, 92

2

91, 90

1

91, 90

1

89 and below

0

89 and below

0

Figure 10 Ability Score

Ability Test Percentile

Achievement Test Percentile

Total Points

Student 1

137

99%=5 points

90%=1 point

30 points

Student 2

131

97%=4 points

92%=2 points

30 points

Student 3

127

95%=3 points

95%=3 points

30 points

Student 4

123

92%=2 points

96%=4 points

30 points

Student 5

150

99%=5 points

89%=0 points

30 points

Figure 11

A Different Matrix in Every District It is surprising to find that the matrices used to select students for gifted identification vary from school district to school district. Every school district makes a different decision as to the value of each test percentile, the minimum percentile to which points are awarded, the range of percentiles that should receive similar points, and the final cutoff score for gifted identification. If you search the Internet for samples of gifted identification matrices you will find any number of samples from different school districts across the county each awarding matrix points in a different fashion. Examining three identification matrices used in different school districts (previously figure 1, figure 6, and figure 12) figure 13, illustrates the different values that school districts award the same student scores in the 90th, 95th, and 99th percentiles.

6


Figure 12 Matrix 1 Percentile Ranks

Matrix 1 Points

Matrix 2 Percentile Ranks

99%, 98%, 97%

5

95-99%

12

99%

20

95%, 96%

4

90-94%

10

98%

19

93%, 94%

3

85-89%

8

97%

18

91%, 92%

2

80-84%

6

96%

17

90%

1

70-79%

4

95%

16

94%

15

93%

14

92%

13

91%

12

90%

11

Matrix 3 Matrix 2 Percentile Points Ranks

Matrix 3 Points

Figure 13 Noting the difference between the points awarded in each individual matrix at the 90%, 95%, and 99% allows us to make a comparison of these values among the three school districts. In matrix one a 90% score receives one matrix point, which is 20% of the maximum 5 matrix points possible. In matrix two a 90% score receives 10 points, which is 83% of the total matrix points possible and in matrix 3 a 90% score receives 11 matrix points which is 55% of the total matrix points possible. The figure 14 bar chart provides a visual representation of these differences showing a marked difference among school districts for the value of test scores in the same percentile range. As you can see in the illustration below, each of the three school districts values scores differently. The first school district does not value a 90% score as highly as school district two and three. Scores at the 95% are valued similarly in the first and third school district, but in the second school district a 95% score is valued more highly and is treated exactly the same as a score in the 99%.

7


Comparing the Three Matrices

90th 95th 99th

1

2

3

Figure 14

8

Normalizing Data first ten pages  

Sharon Ryan The measures used for the identification of gifted students vary from state to state and even from school district to school dis...