Education A Portfolio--Scottsdale 2013 by Center for Scholastic Inquiry

Academic Capitalism Academic Integrity Accounting Adolescence and Youth Adolescent Psychology Adult Development Adult Education Aging Alcoholism & Drug Addiction Alternative Education Applied Psychology Applied Research Arts Education Assessment Behavioral Economics Behavioral Sciences Bilingual Education Business Business Administration Business Education Business Ethics Business Law Change Agency Charter Schools Child Development Child Psychology Classroom Technology Clinical Counseling Clinical Psychology Clinical Social Work Cognitive Science Collegiate Education Community Social Work Competitive Intelligence Consumer Psychology Counseling Curriculum & Instruction Decision Sciences Developmental Disorders Developmental Psychology Disparity Research Distance Learning Doctoral Education Early Childhood Education Early Intervention E-commerce Economics Education Administration Educational Diversity Educational Change Educational Interventions Educational Leadership Educational Psychology Elementary Education English as a Foreign Language English as a Second Language Entrepreneurship Environmental Psychology Experimental Psychology Facilitation Families & Parenting Finance Financial Management in Higher Education Foreign Trade Gender Global Economics Graduate Education Guidance Counseling Higher Education Higher Education International Academic Research Conference Higher Education Funding Recruitment Higher Education Retention Holistic Education Home Schooling April Therapy 17-19,Information 2013 Systems Information Management Systems Human Development Human Rights Individual Instructional Delivery Methods Integrative Movements Interdisciplinary Research International Business Scottsdale, Arizona International Education Research International Finance International Politics K-12 Education Knowledge Management Leadership Learning Magnet Schools Management Marketing Marriage & Family Therapy Memory & Learning Middle School Education Music Education New Program Development Non-Traditional Presentation Learners Relapse Prevention Electronic Online Instruction OrganizationalPortfolio Management Organizational Psychology Peace Education Track-Part I & Conflict Political Psychology Pre-School Education Primary Education Private School Education Public Administration Public Policy Public School Initiatives School Counseling School Social Work School Psychology Secondary Education Small Business Social Issues Social Movements Social Psychology Special Education Social Neuroscience Social Structure Social Work Teaching Vocational-Technical Education Undergraduate l

Center for Scholastic Inquiry (CSI) offers electronic presentation portfolios as a courtesy to the attendees of our conferences. Materials are the sole intellectual property of the presenters and are displayed only with their permission. All questions about content should be directed to the presenters. Pages

Table of Contents

4-30

Rural Field Experiences with Urban Pre-Service Teachers – Dr. Miguel Fernández, Dr. Cynthia Valenciano, Dr. Timothy Harrington, Mark Wesolowski, Angela Logwood, and Diana Carrasco

31-44

The Use of Dispositional Assessment in Teacher Preparation as a Means of Preparing Ethically Responsible Teachers – Dr. Sally A. Creasap

45-77

Teaching Critical Skills: The Influence of 3D Virtual World Simulation in a Mock Code Scenario – Dr. Lisa Smith

78-88

An Intuitive Approach for Teaching the Central Limit Theorem – Dr. Brian Huffman

89-121

Student Teaching Self- Efficacy – Dr. Kevin Mackin

122-143

Impact of Instructional Approaches on Achievement of Economically and Learning Disadvantaged Students at Two Similar Ohio Middle Schools – Dr. Sherry Long

144-177

“Soft Skills” and Student Teaching Success: The Predictive Validity of the Group Assessment Procedure for the Selection of Teacher Education Candidates – Dr. Sally A. Ingles

178-192

Exploring Inquiry Principles of Art to Teach Mathematics – Dr. Lisa Douglass, Dr. Mathew Conley, and Rachel Trinkley

193-212

Closing the Achievement Gap: Strategies to Improve Mathematics Achievement of Academically Deficient Students in an Urban School – Dr. Taik Kim

213-230

Improving Preschool Family/Student Achievement through Multicultural Teaching and Learning – Bonnie Sullivan, Dr. Gilbert Dueñas, and Dr. Shelly Bowden

231-253

The Confidence to Reach All Learners: Using Dual Coding Theory to Maximize Learning – Dr. Amy Vizenor

254-286

Faculty and Students Speak Out: When Technology in the College Classroom is Productive or Distractive – Dr. Myrna Olson and Austin Winger

287-310

A Narrative Study of Teacher and Student Perceptions of Corrective Feedback on Speech Production in English – Cynthia Alvarado

311-329

The Professional Exploration Program: An Alternative Admissions Process – Linda Kisabeth

330-359

Urban Charter Schools and Factors that Influence the Achievement of Students from Culturally and Linguistically Diverse (CLD) Backgrounds – Dr. York Williams

360-405

Supporting Value-Added Claims for the Accreditation of an Educational Leadership Program Using a Modified Solomon FourGroup Design – Dr. Glenn Koonce

Rural Field Experiences with Urban Pre-Service Teachers Miguel Fernรกndez, Ph.D. Cynthia Valenciano, Ph.D. Timothy Harrington, Ph.D. Mark Wesolowski, MS.Ed. Angela Logwood Diana Carrasco

Project History O O O

2006 – Initial contact with rural and tribal schools to discuss clinical immersion by urban preservice teachers. 2007 to 2011 – Rural field clinical immersion with a different set of 5 to 8 urban preservice teachers for one week each year. Fall 2011 – Secure IRB approval and Grant Funding to complete research on clinical immersion of urban preservice teachers in rural schools. Spring 2012 – Data collection with 10 urban preservice teachers from 5 programs at 3 rural schools and 1 tribal school conducted by 3 researchers (professors). Summer & Fall 2012 – Data analysis with 3 preservice teacher researchers who continued with the project and 3 researchers (professors). Spring 2013 – International Presentation

Objectives O Prepare pre-service teachers to have field

experiences in rural schools O Present recommendations for effective teacher preparation in the area of diversity O Enhance the ability to describe practices for program approval and accreditation

Methodology O Data Collection Process O Daily Seminars O Asynchronous activities on Moodle

O Instruments O My Stories O Reflection Prompts O Discussion Forums O Field Notes

O Setting O Participants

Research Question 1 What do faculty and students from the urban setting believe about the rural/tribal school setting before the learning experience?

O Experience similar to childhood living in a

rural area outside the United States. O Small school with: O Some grades overpopulated and O Others with just a few students

O Scarce resources

O Prevalent poverty issues. O Tribal school environment similar to a

bilingual classroom where studentsâ&#x20AC;&#x2122; cultures, traditions, beliefs, and their native languages are valued and celebrated. O Tribal schools promote bilingualism and value the importance of preserving studentsâ&#x20AC;&#x2122; native language.

O Expectations O Preconceived

notions O Technology O Student enrollment O One room school

house O Special education setting

Research Question 2 What do faculty and students from the urban setting believe about the rural/tribal school setting as a result of learning and teaching in the rural school setting?

O Similarities O Technology O Resource Room

O Differences O Classroom

management O More family-oriented environment O Security

Research Question 3 In what respects do these preconceived notions facilitate and/or hinder the ability for the participants to have an impact on KK12 student learning?

Student 6 was born in an African country. When she was 6, her mother moved to the United States…she had difficulty with English in school and was often teased by her peers. As a result she began acting out until one day she met a special teacher that created an environment that was safe and supportive.

Research Question 4 Can a weeklong immersion into a diverse setting produce detectible, positive learning that makes it feasible and fiscally responsible to continue offering cultural exchange opportunities to our students?

O 70% found little to no relevance O 30% identified relevance of the experience O Opinions evolve as teaching experience

continues O Weeklong immersion can be beneficial if preservice teacher is willing to learn O For some preservice teachers, a week is a short period of time

Research Question 5 What recommendations emerge for teaching in urban and rural settings as a result of a comparative view of the two settings?

Pre-Service Teacher Recommendations O Let go of any biases O Have an open mind O Donâ&#x20AC;&#x2122;t set expectations too high O Pre-service teachers should have an

experience in a different setting than what they are familiar with

Pre-Service Teacher Recommendations O Be able to adapt to the environment, and

find ways to have students share their concerns and express their fears. O Use techniques which they are comfortable with in an environment that is suitable to them

Pre-Service Teacher Recommendations O Most of all make sure you know what the

experience entails and be a willing party to participate O Preservice teachers should know that the experience will impact them in positive ways and continue to impact them as they grow deeper into their careers

Faculty Recommendations O Help preservice teachers to expect the

unexpected (While schools are highly regulated in their schedules, many unforeseen occurrences happen) O Build strong relationships with people from host schools O Have a required set of seminars scheduled a few weeks and then a few months after the weeklong immersion in order to help pre-service teachers appreciate what they learn from their experiences

Limitations O Participant fall out O Extreme bias to extreme passivity O Reflections in â&#x20AC;&#x153;real timeâ&#x20AC;? O Mixed methods can relay mixed messages-

Participants who were deeply engaged generated the most qualitative evidence that can skew quantitative findings if not fully analyzed.

Conclusions O The participants with greater cultural

experiences and/or openness to learning new things seemed to gain more from the immersion program than those who did not share the same openness O To teach is a human act, to be a humane teacher takes extra “effort” and “experiences” that might not be appreciated by preservice teachers wherever they come from, but that is what we aspire to do.

Final Conclusion O A longer immersion program would be ideal

for a richer learning experience both for the students and the teacher candidates O We expect to improve our immersion

program because we are collecting and analyzing the voices of our preservice teachers “with” them and not “for” them.

DR. SALLY A. CREASAP CAPITAL UNIVERSITY

WHY THE INTEREST IN DISPOSITIONS The story of Rebekah The story of Nick From the department chair to legal counsel

CONFERENCE SESSIONS ASSOCIATION OF TEACHER EDUCATORS ANNUAL WINTER CONFERENCES

Year

# of sessions

Year

# 0f sessions

2005

2010

2006

2011

2007

2012

2008

2013

2009

NCATE ACCREDITATION

At the turn of the century the National Council for the Accreditation of Teacher Educators (NCATE) added the assessment of dispositions to the standards for teacher education programs.

DISPOSITIONS DEFINED the values, commitments, and professional ethics that influence behaviors toward students, families, colleagues, and communities and affect student learning, motivation and development as well as the educatorâ&#x20AC;&#x2122;s own professional growth. Dispositions are guided by beliefs and attitudes related to values such as caring, fairness, honesty, responsibility and social justice. For example, they might include a belief that all students can learn, a vision of high and challenging standards, or a commitment to a safe and supportive learning environment (2001, p. 30).

PROBLEMS WITH OUR CURRENT SYSTEM Assessment Tool with no validity or reliability Inconsistencies with the data collection process Lack of date assessment No data management system

DISPOSITIONS PILOT Based on the work of Rinaldo, Foote, et. al First year: two faculty teaching first field placement course Next year: full implementation across all field experience courses

NEW PROBLEMS Sabbatical Change of Data Management Systems Lack of commitment and ongoing reflection Lack of an Assessment Rubric

FEEDBACK FROM STUDENTS After being shown the disposition assessment: Do you recall completing a self-assessment during your first, second and third field experiences? Did you receive the assessment results from your cooperating teacher and/or university supervisor? Did you reflect on the data (self, group, or written)?

WHERE TO WE GO FROM HERE • More purposeful attention to the disposition assessment each year • Reflection • Faculty support • Ongoing review of the data • Feedback from the students, cooperating teachers and supervisors Based on the work of Koeppen and Davidson-Jenkins (2007)

PROPOSED LEGISLATION Ad Hoc Committee on Educator Candidate Academic Measures Recommending statewide minimum admission requirements for all state approved educator preparation programs in Ohio

PROPOSED PATHWAYS Three Undergraduate Programs Three for Graduate Programs Emphasis on: I) ACT/SAT II) Praxis I cut off scores III) GPA However, Each of the six pathways contain the following: “Earned satisfactory or above ratings in institutionally administered dispositional assessment.”

Dr. Sally A. Creasap screasap@capital.edu (614) 236-6165

Teaching Critical Skills: The Influence of 3D Virtual World Simulation in a Mock Code Scenario Lisa G. Smith PhD, RN, CNE Dissertation Research Findings Center for Scholastic Inquiry conference Scottsdale, Arizona April 17, 2013

Background of the Study Selecting the best educational method and platform is

important to learn, retain, and apply educational concepts Advances in technology teaching and learning

environments different from the traditional classrooms of yesterday Students grown up with technology = Digital Natives (Prensky, 2001)

Individuals born after 1980

New teaching approaches must be investigated

Todayâ&#x20AC;&#x2122;s Classroom Setting

tvsearcher.snapstream.com

Problem Needs teaching methods the 21st century student can connect with If methods are effective = graduate competent, safe nurses Challenged to develop effective teaching methods Achieve competence in critical skills

Other disciplines simulate critical skills using: High fidelity & virtual reality

Unable to justify virtual reality effectiveness Lack of research in nursing

Virtual Reality Different platforms of Virtual Reality

3D Computerized Virtual World Platform Internet Based Free Download Software Program Continuous Environment Social Network Avatars Communicate â&#x20AC;&#x201C; Voice or Text Chat

3D Computerized Virtual Worlds Who uses VW’s in education Librarians Language Arts Sciences Social Sciences Psychology Marketing Health Sciences Anthropology And many more……. Bookmoving.com

Purpose of the Study To determine if a 3D virtual world (VW) simulation was an effective learning method for simulating critical skills. This study: 1. Explored student: satisfaction self-confidence and how important simulation design factors were

with high fidelity and 3D virtual world simulation methods in a mock code scenario

2. Identified relationships between: select demographic variables and nursing student satisfaction with

virtual simulation.

Traditional Simulation Lab

IRSC Nursing Students, 2013

3D Virtual Worlds – Second Life,® OpenSim, ReactionGrid

Virtual Simulation

Virtual Classroom Discussion

Cardiac Simulation Lab Teams of 5-6 Nursing Students Participated in a Mock Code Scenario

Developed by Mark Dubin and Lisa Smith

Review of Literature

HFS - Many Medical & Nursing Studies Validate: Student Satisfaction, in Self-Confidence, & Satisfaction with Design (NLN, 2006; Dobbs et al., 2006; Kuznar, 2007; Smith and Roehrs 2009; Levett-Jones 2011)

Virtual World Studies – Few Medical Studies Validate: As Effective as HF ( Youngblood et al., 2008) Student Satisfaction (Creutzfeldt et al., 2010 , Wiecha et al.,2010, Youngblood et al., 2008 ) in Self-Confidence (Youngblood et al., 2008; Wiecha et al.,2010), Satisfaction with Design – Realistic (Youngblood et al., 2008; Creutzfeldt et al., 2010)

No Published Nursing Studies

Popular Theoretical Frameworks Constructivism Connectivisim Presence Pedagogy Community of Inquiry

Situated Cognition Framework

Situated Cognition Framework by Paige & Daley (2009)

Research Hypotheses Hypothesis 1: Satisfaction for students who are instructed

using virtual simulation will be significantly different than satisfaction for students who will be instructed using highfidelity simulation. Hypothesis 2: Self-confidence for students who are

instructed using virtual simulation will be significantly different than self-confidence for students who will be instructed using high-fidelity simulation. Hypothesis 3: The importance of simulation design factors

for students who are instructed using virtual simulation will be significantly different than the importance of simulation design factors for students who will be instructed using highfidelity simulation.

Research Hypotheses Hypothesis 4:

Select demographic variables (e.g. age, gender, ethnicity) and prior gaming experience, will be related to student satisfaction with virtual simulation.

Protection of Human Subjects IRB approved as an Exempt study Anonymous study

Minimal risk study-no known risks and no direct benefits Student participation - Voluntary Informed consent- Cover letter No direct teaching or grading responsibilities with students

who participated Small token of appreciation to volunteers

Methodology Research Design Descriptive-Quasi-Experimental– single group, nonrandomized Based on a 2006 four-phase national multi-site study National League of Nursing (NLN)

Setting State college in SE Florida 1st time enrollment of the last med-surg course or completion within 3

months of data collection

Sample G Power Analysis 1-β = .80, d = 0.5 (medium) n= 34

α = 0.05 Actual Sample Size = 59 – All subjects met inclusion criteria

Instruments Demographic Questionnaire Researcher Developed Age, Gender, Ethnicity, Previous Computer Gaming Experience Student Satisfaction & Self-Confidence in Learning Scale NLN (2006) 13 items-5 point Likert Scale 5- Satisfaction with current learning 8-Self-Confidence in learning

Simulation Design Scale NLN (2006) 20 items -5 point Likert Scale Part 1 â&#x20AC;&#x201C; Student perceptions of the design features Part 2 - How important that design feature was to the student

Instrument Reliability Cronbach’s Alpha α Analysis

Instruments

NLN (2006)

HFS

Virtual

Satisfaction

α = 0.94

α = 0.768

α = 0.87

Self- Confidence

α = 0.87

α = 0.819

α = 0.82

Importance of Simulation Design

α = 0.96

α = 0.9

α = 0.942

Reliability Results = >0.7

Demographic Data Analysis Gender Age 20-61 0

18.6 Male

33.74

Mean

Female 81.4

n = 59

Age 20-61

Demographic Data Analysis 3.4% 1.7% 5.1% 6.8%

Ethnicity

White Hispanic Other African -American 83.0%

n = 59

Asian

Gaming Experience Data Analysis n= 25 - 42.3%

â&#x2030;Ľ 5 Yrs.

32%

3-5 Yrs.

12%

1-3 Yrs.

Gaming Experience

12%

<1 Yr.

44% 0%

10%

20%

30%

40%

50%

Data Analysis Students were more satisfied performing the mock code

scenario using the HFS than with the virtual simulation Self-confidence levels do not decline when critical skills are

performed in a virtual simulation. Importance of design factors do not vary whether the

method is HF or virtual. Simulation design is important regardless of the environment

Data Analysis - Relationships between Satisfaction with Virtual Simulation Age – No correlation Mean Age 33.7 – Majority were Digital Natives Gender- No correlation Low # of males (11) Ethnicity – No correlation Not a diverse sample Prior Computer Gaming – Correlation More satisfied with virtual simulation than students without prior gaming # of Years of Gaming – no correlation between # years and satisfaction *statistic – negative – indicates the more experience the less satisfied with

the virtual simulation.

Implications of the Study 1.

In this study, age, gender, & ethnicity did not have an effect on student satisfaction with virtual simulation

Therefore, may be appropriate teaching method regardless of age, gender, or ethnicity

Students with prior gaming experience were more satisfied than students with no prior gaming exp. As advances in technology produce more sophisticated and easier

virtual platforms, student satisfaction may exceed more traditional teaching methods

Significance of the Study Education Difficulty finding clinical sites Limited Simulation Scheduling Distance education programs Promotes collaborative learning

â&#x20AC;&#x201C; may enhance critical thinking

Clinical Nursing Practice Students and nurses report a lack

of knowledge & confidence during code situations.

VW = VW allow repeated practice

of critical skills

Students and hospital nurses can

practice code skills repeatedly which can skill level

VW = Self Confidence does not

decline when performing critical skills.

Conclusions Results are promising VW simulation may be an appropriate platform to practice

critical skills Expanse of technology New platforms Meet distance learning needs Simulation Lab Scheduling

Replication of this study on a larger scale is needed with a

more diverse ethnic population

Future Research Recommendations Quantitative Research Effectiveness of VW sim on knowledge learned and retained Self-confidence to perform critical skills in real life after a VW

sim Transferability of skills from a VW environment to a real-life setting

Qualitative Research Satisfaction of students and faculty with VW simulations

Overarching Question How can we use instructional technology to design our learning environments to facilitate: Knowldege Acquisition & Retention Critical Thinking/Reasoning Enhanced Skills Student Learning Outcomes

Key â&#x20AC;&#x201C; Collaborative Learning Environments Students can construct knowledge based

on the social interactions, past experiences, and cognitive knowledge of the group. Interactive teams in classroom, simulation

lab, or virtual environments.

Non-Traditional Teaching Methods that Work in Collaborative Learning Environments Game-based Learning â&#x20AC;&#x201C; 3D Virtual Worlds Simulations Role Playing Unfolding Case Studies Problem-Based Learning Team Based Learning Use of Clickers Concept Mapping

Death By PowerPoint

www.powerltd.com

Questions

An Intuitive Approach for Teaching the Central Limit Theorem

BRIAN J. HUFFMAN & HOSSEIN EFTEKARI COLLEGE OF BUSINESS AND ECONOMICS, UNIVERSITY OF WISCONSIN – RIVER FALLS

The Central Limit Theorem The Central Limit Theorem (CLT): one may assume

sample averages taken from an infinite population are normally distributed, even if the samples come from non-normally distributed populations, provided that the:

samples are independent of one another, the population distribution has a finite mean and standard deviation, the sample size is large enough, â&#x20AC;&#x153;Bla, bla, bla, bla, bla, bla, bla, bla, bla, bla, bla, bla, blaâ&#x20AC;Śâ&#x20AC;?

Surprise!!!: Students DO NOT Understand the CLT (also the world is round)

Pfaff & Weinberg (2009) Roberts & Pierce (1999) Ryan (2006) - depressing Tversky & Kahneman (1974) – really depressing

It's mystery, like those big heads on Easter Island or

the popularity of the teeny-weeny knapsack.— David Sedaris An expert is a man who tells you a simple thing in a confused way in such a fashion as to make you think the confusion is your own fault. — William Castle

As you know, students… What I say: “The sampling distribution can be

assumed to be normal and therefore can be completely described by two parameters: the mean and standard deviation (which in this case is called the standard error).” What the student absorbs: “The new thing…or is it still the old thing can be assumed to be normal, ordinary, or every day; and therefore can be completely described by a pair of meters … metrics …something: the fancy word for average and the other thing (which now has a new name for some reason). I wonder if Zooey Deschanel is like Jess in real life?

Let’s talk strategy… Situation: Orioles/Rays game…Rays are the visiting

team…Top of the 9th…Rodney in to pitch for the Rays…walks first batter…second batter hits RBI double. Showalter then has next batter sacrifice to get runner over to 3rd.

Q: Why give up the first out in the inning in that situation when the runner is in scoring position and the pitcher has yet to prove control of the strike zone? A: Duh! Because it's very easy to score a run from third

base with less than two out. Any fly ball to the outfield or a number of ground balls will score the runner.

Support for the New Approach Ryan (2006) Dyck et al. (1998), and Johnson (1986) Rossman et al. (2000), Lunsford et al. (2006),

delMas et al. (1999), and Velleman et al. (1996) Chi et al. (1994), and delMas et al. (1999) Rossman et al. (2000), and Garfield (n.d.)

The New Approach Review (or re-teach) the concept of a probability

distribution Students develop (predict) 3 population distributions as “input” Students predict the parameters of sampling distributions as “output” Students empirically discover actual parameters of “input” and “output” distributions Students explain their prediction errors Only then are presented with the CLT “details” (and even then the explanation is graphical first)

Intuition/Calculation isn’t understanding… What should the average of 3, 1, 8, 7, 2, 9 look like? What should the standard deviation of a bunch of

numbers look like? Graphical intuition – one picture = 1,000 words

Population Distributions “INPUT” Prob. Distribution for Roll of 2 Dice

“OUTPUT”

Testing MNGT 361 (Operations Management) Fall 2013, and

Spring 2013 My Classes MBA 705 (Operations, Project, and Quality Management) Spring 2013 ECON 226 (Introduction to Statistics) Fall 2012,

Winter 2013, and Spring 2013 Hosseinâ&#x20AC;&#x2122;s Class

Results/Conclusions SUCCESS!

Student Teaching Self- Efficacy

R. Kevin Mackin Ed.D. Upper Iowa University

Factors that impact student teaching self-efficacy: Throwing conventional wisdom out the window.

Why Study Teaching Efficacy? Efficacy is correlated with a plethora of positive outcomes including: 1) Increased student achievement 2) Additional support for low-achieving students 3) Persistence in the face of challenges 4) Job satisfaction and job retention

Why Student Teaching Efficacy? 1) Efficacy is most malleable during the first few years of teaching. 2) Emergence of different avenues to licensure 3) Policy initiatives push increasing licensure requirements. 4) Most studies focused on in-service teachers.

Who Participated Two Public Nine Universities Private Universities

• Augsburg College • Bemidji State University • Bethany Lutheran

271 Student teachers Pre

• Bethel University • College of St. Benedict/St. John’s University

269 Student teachers post

• College of St. Scholastica • Crown College • Northwestern College

11 UG Programs 4 Graduate Programs • St. Catherine’s University • St. Mary”s University • University of Minnesota-Morris

Who Participated • Self-Efficacy Study Sample • Female (246) • Male (86) • Undergraduate • Graduate

74% 26% 77% 23%

• 2011 National Sample of Current Teachers • Female 84% • Male 16% • Undergrad 65% to license • Graduate 18% to license • Alternative 18% path to license

Undergrad Population Compared to Graduate. • • • • • • •

MN Undergraduate Male ( 57) 22% Female (201) 78% Elem. (136) 61% Sec. (111) 39% 25 /Under(200) 77% Over 25 (60) 23%

• • • • • • •

MN Graduate Male (29) Female (45) Elem. (13) Sec. (43) 25/Under (13) Over 25 (61)

39% 61% 23% 77% 18% 82%

Hypothesis 1 1. There will be no difference in perceived teacher self-efficacy before and after student teaching. Your prediction? â&#x20AC;˘ A) Increase B) Decrease â&#x20AC;˘ C) Stay the same

Student Teaching has a Significant Impact on Efficacy Paired Samples Ttest n=205

Mean

Standard Deviation

Standard Error of Mean

Pre-student teaching

85.0

9.28

.65

Post-Student Teaching

90.07

8.02

.56

Difference

5.07

8.76

.61

T-score

8.32 Significant .001

Observations • High Efficacy at Start (85 = 7.1 on 9.0 scale) and even higher afterwards (7.51). • Efficacy increased across all groups. • Consistent with most previous research (conventional wisdom supported).

Hypotheses 2 and 3 • 2. There will be no difference in perceived teacher efficacy between undergraduate and graduate licensure pre-service teachers at the start of student teaching. • 3. There will be no difference in perceived teacher efficacy between undergraduate and graduate pre-service teachers at the end of student teaching. • Your prediction? • A) Difference at start B) Difference at end • C) No difference

No Significant Differences Between Undergraduate and Graduate Student teachers One-way ANOVA of Efficacy Scores

Mean SelfEfficacy

Standard Deviation

Before Undergrad

84.83

10.12

Before Grad

85.47

9.84

Between Group differences

0.64

After Undergrad

90.55

8.5

After Grad

91.57

8.0

Between Group Differences

1.02

Sum of Squares

F-score

18.627

.209 sign. .65

46.32

.70 Sign. .40

Observations • Defies “conventional wisdom” - Why no difference? • No difference in undergraduate GPA between undergraduate and graduate level student teachers. (also contrary to cw) • Trade-off between life experiences and recent school experience. • Slightly better scores for graduate but insignificant difference. Greater difference may be masked by nontraditional undergraduate programs.

Hypothesis 4 • There will be no difference in perceived teacher efficacy between urban, suburban and rural student teachers. • Your prediction? • a) urban public higher • b) suburban higher • c) rural higher • d) no significant difference

No Significant Differences Based on Urban, Suburban, Rural Placements Placement Location

Mean

Standard Deviation

Standard Error

Urban public

91.61

9.07

1.37

Urban private

96.75

8.05

4.02

Suburban public

89.87

8.39

0.88

rural

89.95

6.70

.87

One-Way ANOVA

Sum of Squares

Mean Square

F score

Significance

Between Groups

263.93

87.98

1.35

.26

Observations Defies CW: • Trend is for higher efficacy in urban. • Self selection in placements –not randomly placed -those who seek urban placements may have experience and comfort level in urban schools. • Urban Private efficacy is high but • very few private school placements • Needs further research.

Hypothesis 5 • There will be no difference in perceived teacher efficacy between male and female student teachers. • Your prediction? • a) females higher • b) males higher • c) no significant difference

No Significant Difference in Efficacy Between Genders Gender

Mean

Standard Deviation

Standard Error

Female (201)

90.76

7.54

.53

Male (67)

90.51

9.52

1.16

One-Way ANOVA

Sum of Squares

Mean Square

F score

Significance

3.325

.050

.824

Observations • Mixed previous research • Male efficacy was somewhat (but not significantly) lower in pre-student teaching so they closed the efficacy gap. • Male GPA is lower but efficacy is not.

Hypothesis 6 • There will be no difference in the perceived self-efficacy of elementary and secondary level student teachers.

• • •

Your prediction? a) Elementary higher b) Secondary higher c) No significant difference

Elementary Student Teachers Have Higher Post-Student Teaching Self-Efficacy Level of Placement

Mean

Standard Deviation

Standard Error

Elementary (113)

91.63

7.15

.672

Secondary (87)

88.79

9.07

.972

One-Way ANOVA

Sum of Squares

Mean Square

F-Score

Significance

Between Groups

395.13

6.12

.014

Observations • • • • •

•

Unknown why – Classroom management and age? Required number of field experiences? Challenging content? Smaller difference doubled as a result of student teaching- differential impact worth future study.

Hypothesis 7 There will be no correlation between undergraduate grade point average and perceived teacher efficacy. Your prediction? A) Higher grades correlate with higher efficacy B) Lower grades correlate with higher efficacy C) No significant difference

No Significant Difference in Efficacy Based on GPA SelfReported GPA

Mean

Standard Deviation

Standard Error

Cumulative Percent

2.4-2.69 (3)

92.0

5.29

3.05

2.7- 2.99 (13)

87.54

8.22

2.28

3.0-3.19 (27)

90.44

8.61

1.66

16% (10%)

3.2-3.39 (29)

89.07

7.21

1.34

27% (11%)

3.4-3.59 (51)

91.51

8.47

1.19

46% (19%)

3.6-3.79 (60)

90.62

7.28

0.94

69% (23%)

3.8-4.0

91.70

8.08

0.50

100% (31%)

One-Way ANOVA

Sum of Squares

Mean Square

F score

Significance

Between Groups

327.68

54.613

.833

.545

(83)

(5%)

Observations • Definitely flies in face of CW and public policy. • Pattern was puzzling • Grade Inflation may be a factor – grades may not differentiate as well when average GPA is around 3.5. • May want to take non-EDU grades since high grades are a national norm in Colleges of Education.

Hypothesis 8 • H8: There will be no correlation between age and perceived teacher self-efficacy. • Your prediction? A) Older student teachers have higher self-efficacy B) Younger student teachers have higher self –efficacy C) No significant difference

There was a significant relationship between age and Self-efficacy Age of Student Teacher

Mean

Standard Deviation

Standard Error

20-24 (173)

89.87

8.02

.61

25-34 (66)

91.55

7.70

.95

35-44 (22)

96.05

7.32

1.56

45+

89.12

10.51

3.72

One Way ANOVA

Sum of Squares

Mean Square

F Score

Significance

Between Groups

814.64

271.55

4.28

.006

(8)

Observations • Relationship appears to be curvilinear but small n for oldest group makes conclusions difficult • Significance shades the graduate/undergraduate comparison • Older undergrads (non-traditional) should be separate group.

Significant Findings Mean SelfEfficacy Scores

Mean SelfEfficacy Scores

Level of Significance

Pre/Post student Teaching

Pre: 85.00

Post: 90.07

.001

Elementary/ Secondary

Elem: 91.63

Sec.: 88.79

.014

Age

20-24: 89.87

35-44: 96.05

.006

Significant â&#x20AC;&#x153;Insignificantâ&#x20AC;? Differences 1) No significant difference in efficacy based on GPA. (policy implications) 2) No significant difference based on placement location (urban, suburban, rural). 3) No difference between Graduate and Undergraduate. (salary schedule implications)

Next Steps • Include Alternative Path (TFA) • Reexamine GPA • Examine non-traditional undergraduate programs as separate group.

Your Questions or Comments

Thank You! â&#x20AC;˘ The Efficacy Rap Contact: R. Kevin Mackin (mackinr@uiu.edu) Upper Iowa University 563-425-5759

IMPACT OF INSTRUCTIONAL APPROACHES ON ACHIEVEMENT OF ECONOMICALLY AND LEARNING DISADVANTAGED STUDENTS AT TWO SIMILAR OHIO MIDDLE SCHOOLS

Dr. Sherry Long Alice Lloyd College

Background for Study •Achievement gaps among diverse populations of students exist in schools across the nation. •Achievement scores are low for economically disadvantaged and/or learning disabled students in Ohio. •The instructional approach employed by educators affects student performance.

Problem and Purpose Problem: Low academic achievement in reading and mathematics by economically disadvantaged and/or learning disabled students is resulting in achievement gaps in schools. Purpose: The purpose of this quantitative crosssectional survey study was to determine if instructional approach affects reading and math achievement scores of economically disadvantaged and/or learning-disabled students at 2 demographically similar middle schools in Ohio.

QUICK WRITE

Conceptual Framework Theories •Brain-based learning •Constructivism •Influence of theory on instructional approaches •Teacher effect

Broad theoretical area •Closing the reading and mathematics achievement gap of economically disadvantaged and/or learning disabled students Theoretical gap •Determining specific instructional approaches that significantly affect achievement scores for disadvantaged middle school students

Research Questions •How do teacher-centered and student-centered instructional approaches affect student scores? •Which instructional approach is more effective for specific learning-disabled students? •Which instructional approach is more effective for economically disadvantaged students? •Which instructional approach is more effective for economically disadvantaged and learning-disabled students?

Population Grade 6-8 regular and special education teachers in Ohio

Grade 6-8 regular and special education reading and math teachers in similar public school districts as described by the Ohio Department of Education

10 similar Ohio districts with grade 6-8 regular and special education reading and math teachers of record from different regions of the state

13 reading teachers, 9 math teachers, and 9 reading/math teachers at 2 middle schools

Design Appropriateness Why use a quantitative approach? •Asks specific, narrow questions •Collects numeric data from participants •Analyzes these numbers using statistics •Conducts the inquiry in an unbiased, objective manner Why use a cross-sectional survey design? •One point in time •Examines current attitudes, beliefs, opinions, or practices

Data Collection Quantitative •Principals of Adult Learning Scale (PALS) (Conti, 1979; 1983; 1985)

– – –

Online survey 44 item Likert Scale 146 mean score; 20.0 SD

•Demographic information –

Online survey

•Ohio Achievement Test Scores –

Electronic file • • •

student demographics test scores teacher of reading and math for each student

Data Analysis •Descriptive – Teacher – Student

•T-test – Teacher-centered •

Reading and Math

– Student-centered •

Reading and Math

Data STUDENTS

•272 Total Students •107 Reading IEP •106 Math IEP •215 Economically Disadvantaged

TEACHERS •26 Teacher Centered •5 Student Centered

T-test Affect ED students

Teacher Centered

ED/LD students LD students Affect Student Centered

ED students ED/LD students LD students

Results •H1A: A statistically significant difference exists between the mean achievement scores of students and instructional approach.

– Reading: Supported (p < .001) – Math: Higher scores, but rejected (p > .05)

•H2A: A statistically significant difference exists between the

instructional approach and the achievement scores of specific learningdisabled students at two demographically similar middle schools in Ohio.

– Reading: Supported (p < .05) – *Math: Supported for reading scores (p < .05)

Results •H3A: A statistically significant difference exists between the instructional approach and the achievement scores of economically disadvantaged students at two demographically similar middle schools in Ohio.

– Reading: Supported (p < .01) – Math: Slightly higher, but rejected (p > .05) •H4A: A statistically significant difference exists between the instructional approach and the achievement scores of economically disadvantaged and learning -disabled students at two demographically similar middle schools in Ohio. – *Reading: Rejected (p > .05) Mann-Whitney U test

– *Math: Rejected for reading scores (p > .05) Mann-Whitney U test

Research Questions Answered •How do teacher-centered and student-centered instructional approaches affect student scores? – Student-centered reading teachers significantly affect reading OAT scores

•Which instructional approach is more effective for specific learningdisabled students? – Student-centered reading teachers significantly affect reading OAT scores

•Which instructional approach is more effective for economically disadvantaged students? – Student-centered reading teachers significantly affect reading OAT scores

•Which instructional approach is more effective for economically disadvantaged and learning-disabled students? – T-test: student-centered reading teachers significantly affect reading OAT scores; however, Mann-Whitney U indicated no difference

7 Factors of PALS •Learner Centered Activities •Personalizing Instruction •Relating to Experience •Assessing Student Needs •Climate Building •Participation in the Learning Process •Flexibility for Personal Development

Highest Student-Centered Factors •Personalizing instruction – variety of materials, methods, and assignments based on needs and abilities

•Relating to experience – real-world problems, experiential background, and reflection

•Assessing student needs – Conferencing, goals, and strategies

Meaningful Connections Challenge Socialization Unique Brain

Learner

Meaningful Connections

Challenge

Socialization

Unique Brain

Authentic, RealWorld Problems

Creative and Critical Thinking

Cooperative Groups

Interests

Problem-Based Learning

Analyze, Defend, or Explore Viewpoints

Small Groups

Learning Styles

Project-Based Learning

Higher Order Thinking

Large Groups

Multiple Intelligences

Self-Explanation

Flexible Groups

Scaffolding

Memorization Techniques

Varied Levels of Difficulty

Significance General Importance •Provides educators with instructional approaches that are effective with specific disadvantaged groups at the middle school level to close the achievement gap •Provides an understanding of appropriate strategies to create a conducive learning environment for students •Provides an understanding of the impact of instructional approaches on student achievement

Leadership Importance •Leaders need to know what instructional practices promote higher achievement scores among economically disadvantaged and/or learning disabled students in order to support teachers

Recommendations •Additional studies – Lack of studies focused on specific disadvantaged groups – Support or refute findings

•Additional teachers – Increase number of student-centered teachers

•Additional middle schools – Increase generalizability

QUICK WRITE

â&#x20AC;&#x153;Soft Skillsâ&#x20AC;? and Student Teaching Success: The Predictive Validity of the Group Assessment Procedure for the Selection of Teacher Education Candidates Presentation at the International Academic Research Conference Scottsdale, Arizona April 18, 2013

Dr. Sally A. Ingles Assoc. Professor of Education Spring Arbor University ingles@arbor.edu

Many researchers and practitioners purport the following are requisite skills of effective teachers: Verbal Communication Skills (thinking, speaking, writing) Human Interaction (expressions of warmth, rapport, listening) Leadership (initiative, self-assuredness, providing direction to a group)

Simple Definitions Hard Skills

Soft Skills

“include job skills like typing, writing, math, reading and the ability to use software programs”

“personality-driven skills like etiquette, getting along with others, listening and engaging in small talk”

tend to be specific skills that are more easily taught than soft skills Source: Invtestopedia.com

“less tangible and harder to quantify” than hard skills Dispositions “habits of action and mind” (DarlingHammond)

What does the literature indicate? Traditional academic criteria • •

GPA required course work competency tests

are among the most commonly used selection criteria for teacher education candidates, and minimum passing scores for each have been raised significantly in recent years. (Laman & Reeves, 1983; Petersen & Speaker, 1996)

Though these traditional academic selection criteria are predictive of general academic success in the program, they are poor predictors of student teaching performance and teaching success.

(Byrnes, Kiger, & Shechtman, 2003; D’Agostino & Powers, 2009; Metzger & Wu, 2008; Michiels Hernandez, Ward & Strickland, 2006; Mikitovics & Crehan, 2002; Shechtman & Godfried, 1993; Shechtman & Sansbury, 1989)

Why don’t we measure dispositions at admission?

Perhaps programs do not attempt to select candidates based upon dispositions because these soft skills—by definition—they are difficult to measure.

In essence, teacher preparation programs are selecting teacher candidates based upon evidence of hard skills . . . although many researchers and practitioners contend that soft skills are equally vitally in the repertoire of an effective teacher.

Background of the Study Teacher

preparation programs commonly use quantitative indices for admission selection criteria.

Recent

studies suggest that traditional academic indicators, such as GPA and standardized test scores, are poor predictors of student teaching performance.

The

Group Assessment Procedure is one teacher candidate selection tool that has demonstrated validity in predicting student teaching performance scores for teacher candidates attending large public institutions of higher education in both Israel and Utah.

Group Assessment Procedure: A Brief Description

90 minutes structured “interview”

Approximately 8 teacher candidates are interviewed at the same time

Two trained faculty/staff “interviewers/raters” facilitate a four-stage discussion

Four Distinct Stages in the interview that elicit participant responses and interactions with one another.

The Group Assessment Procedure (GAP) is a “Group Interview” designed to measure individual’s performances in three areas:

Oral Communication

Human Interaction

Leadership

Scores range from 6 (exceptional) to 1 (poor) based upon trained interviewers’ analyses of student behavior. Additionally, interviewers (assessors) record a “global impression” score of each participant’s Overall Fitness for the Teaching Profession. This score is denoted as Overall Rating

Rationale & Significance of Study Prior

to this research, the Group Assessment Procedure (GAP)had been studied exclusively at public universities in Israel and Utah.

Two

recent studies of the Group Assessment Procedure in those locales have raised questions regarding its validity and reliability

The

validation of the GAP at a small, private university is of significant value because the majority of institutions of higher education preparing teachers are independent, non-profit institutions

Purpose of the Study

to examine the validity of the Group Assessment Procedure as a teacher candidate selection tool at a small, private university in the Midwest

to examine the relationship between teacher candidatesâ&#x20AC;&#x2122; performance scores in the Group Assessment Procedure, GPA at the time of admission to the teacher preparation program, and student teaching performance scores

Study Variables Criterion

variable:

student

teaching performance rating (measured by cooperating teachers using the Group Assessment Rating Scale)

Predictor

variables:

GPA calculated

from select general education courses that were required for admission to the School of Education Group Assessment score (overall rating) participants earned in the Group Assessment Procedure

Conducting the Study

Non-experimental, correlational research design was used

Existing academic records were accessed to collect data used to calculate each candidateâ&#x20AC;&#x2122;s GPA on select general education courses at the time of admission

Student teaching performance scores and Group Assessment scores were collected concurrently during the student teaching semester

A linear regression analysis was used to measure the strength and direction of the relationship between variables

Study Participants

31 teacher candidates at a small, private university in the Midwest who were enrolled in student teaching

65% of participants were female

77% were seeking elementary certification

16% were racial/ethnic minorities

65% were between the ages of 21-24

Research Question 1 Is there a relationship between the overall rating score a teacher candidate receives from a cooperating teacher during student teaching and a teacher candidateâ&#x20AC;&#x2122;s overall rating score on the Group Assessment Procedure?

Research Question 2 Is there a relationship between the overall rating score a teacher candidate receives from a cooperating teacher during student teaching and a teacher candidateâ&#x20AC;&#x2122;s GPA at the time of admission to the teacher preparation program?

Research Question 3 For teacher candidates who score “3” or lower in the Group Assessment Procedure, is there a relationship between the overall rating score a teacher candidate receives from a cooperating teacher during student teaching and the candidate’s GPA at the time of admission?

Findings, Results, and Conclusions

Findings: Question 1 There is a significant, positive relationship between overall performance ratings in student teaching reported by cooperating teachers and overall performance ratings in the Group Assessment Procedure. Teacher candidates who earn high ratings in the Group Assessment Procedure tend to earn high ratings in student teaching. Statistically Speaking: A positive correlation between the two variables (r = .483, p < .01) measured at a level of statistical significance.

Findings: Question 2 There is no significant relationship between the overall rating score a teacher candidate receives during student teaching and a teacher candidateâ&#x20AC;&#x2122;s GPA at the time of admission to the teacher preparation program. Statistically Speaking: o

The R Square value of .033 indicates that only 3% of the variance in student teaching performance ratings (OR-CT) can be explained by GPA. The Pearson Correlation coefficient (R = .182) indicates a low, nonsignificant relationship between the variables.

Findings: Question 3 For teacher candidates who scored 3 or lower on the Group Assessment Procedure, no significant relationship was found between their student teaching performance ratings and their GPA at the time of admission. NOTE: Because only 10 participants scored 3 or lower on the Group Assessment Procedure, results should be noted with caution. Statistically Speaking: o

The Pearson Correlation coefficient (.052) indicates a low, nonsignificant correlation between student teaching performance scores and GPA. The R Square value of .003 indicates that only .3% of the variance in student teaching performance ratings (OR-CT) can be explained by GPA

CONCLUSIONS A teacher

candidateâ&#x20AC;&#x2122;s performance in the Group Assessment Procedure is more predictive of student teaching performance than GPA at the time of admission.

Teacher

candidatesâ&#x20AC;&#x2122; verbal communication skills, human interaction skills, and leadership skills have a greater influence on teaching performance skills than GPA at the time of admission to a teacher preparation program.

Recommendations for Future Research

Additional studies of the Group Assessment Procedure (GAP)

Using more diverse target populations (higher percentages of teacher candidates who are racial or ethnic minorities, males, and teacher candidates pursuing secondary teaching certification) At other institutions of higher education that are independent, nonprofit institutions Using the GAP as a hiring tool for teachers Using a qualitative design that generates theory and possible variables related to candidates who score low on the Group Assessment Procedure yet above average in student teaching

Additional studies that explore behavior-based admission criteria (in contrast to traditional, quantitative indices)

What are the practical implications for teacher preparation programs?

The School of Education at Spring Arbor University completely redesigned its induction process based on this research.

Why did SAU change its teacher candidate induction process? Like many other teacher preparation programs, we were not using admission criteria that would identify candidates who were most likely to succeed in student teaching. Therefore, we chose to add selection criteria that were more predictive of student teaching performance.

What admission criteria did we keep?

A minimum GPA requirement (2.70/4.0)

A minimum standardized test score (passing all three sections of the state mandated Michigan Basic Skills Test)

A minimum grade earned in select general education courses (Speech, College Writing, Psychology)

What did we add?

A teacher candidate induction course entitled, â&#x20AC;&#x153;Critical Skills for Professional Educatorsâ&#x20AC;?

A Professional Skills Lab (PSL) comprised of three assessments: Oral Reading Fluency Extemporaneous Writing 90 minute Group Interview (Group Assessment Procedure)

The Induction Course focuses on skills and dispositions

It provides explanations and models of the requisite skills (“hard” and “soft”) and dispositions of an effective teacher.

It provides weekly opportunities for targeted skill and disposition development—coupled with peer and instructor feedback (FORMATIVE ASSESSMENT)

It provides weekly advising—a semester-long orientation to the complexities of the teacher certification process at SAU—housed within a single course.

What is the Professional Skills Lab?

In essence, it is an admissions process that generates much more information about teacher candidates than traditional academic indicators can provide.

It provides potential candidates the opportunity to demonstrate their â&#x20AC;&#x153;recently polishedâ&#x20AC;? proficiencies in numerous areas: reading, writing, speaking, critical thinking, human interaction and leadership.

Conclusion Both “soft skills” and “hard skills” are vital assets of an effective teacher. SAU’s School of Education has developed a research-based induction process to evaluate and develop both skill sets to increase the likelihood that our candidates will become effective teachers.

Exploring Inquiry Principles of Art to Teach Mathematics Matthew Conley, Assoc. Professor, Ohio Dominican University Lisa Douglass, Asst. Professor, Ohio University Rachel Trinkley, Manager of Facilitated Experiences, Columbus Museum of Art

Schools are facing tough curricular decisions every day Many schools are removing the arts to focus on core curriculum in order to improve standardized test scores. Meanwhile, research shows that high quality arts education â&#x20AC;&#x153;fosters broad dispositions and skills, especially the capacity to think creativity and the capacity to make connections.â&#x20AC;? (The Qualities of Quality, Wallace Foundation, 2009)

We wondered how we might address this with our pre-service mathematics educators4 We found little integration at the university level content areas were segregated. We also knew that schools were under scrutiny for having poor math scores in the U.S. Could integrating mathematics and the arts help students better succeed?

4using a Constructivist approach Constructivism: Making or constructing knowledge and meaning through guided discovery, dialogue, and critical thinking. This type of thinking is what U.S. companies are looking for in their employees, but have a difficult time finding with what our schools are still teaching.

Columbus Museum of Art shared our philosophies Programs funded by the U.S. Dept. of Education provide evidence that looking and talking about art can improve critical thinking skills, creativity, collaboration, and communication! Columbus Museum of Artâ&#x20AC;&#x2122;s ODIP model (Observe, Describe, Intepret, Prove) was a perfect way to help move us towards an interdisciplinary approach to constructivist mathematics teaching. This is exactly what our future mathematics teachers needed to be implementing into their classrooms.

How our study worked We took our pre-service mathematics students to CMA to learn how to apply ODIP to art. We brought the ODIP model back into the university classroom, and learned how to use that in the mathematics classroom. We then took the mathematics students abroad to study The Arts in a different setting (art and architecture of Spain/Portugal) and applied to ODIP model again - this time integrating The Arts and Mathematics.

Learning ODIP at CMA

Applying ODIP in pre-service classrooms

Applying ODIP abroad

The Results Through journal entries, summative reflections, videotaping, and professorâ&#x20AC;&#x2122;s field notes, we found the following themes from these preservice mathematics teachers:

PROCESS APPLICATION Students were frustrated with the process of applying ODIP to a new situation. They found it ‘hard’, ‘challenging’, ‘confusing.’ They wanted to ‘just know the right answer.’ Many who succeed in math prefer linear, objective approaches to learning. As educators, this was a great experience for them to learn that they need to guide and assure students through new learning strategies.

PRACTICAL APPLICATION Some students grasped the ODIP concept and were very pragmatic about wanting to adopt the model into their future classroom. Carrie stated how she can now see how to integrate mathematics and science better. Meghan saw how ODIP could help pique student interest in mathematics. Neither future math teacher could see how THEY were the change agent, nor the model they were using. Beliefs really did not change.

THEORETICAL APPLICATION â&#x20AC;&#x153;This [ODIP strategy] will help me plan as a teacher and show my students the relationship of math to the real worldâ&#x20AC;? ~ Trent "I hope this will help my students not just be learners of math, but productive members of society who are users of math." ~ Trent Some students showed a deep understanding of constructivist teaching - going beyond practical into theoretical.

Educational significance Implementation of ODIP strategy provoked students to think about their own teaching philosophies and their own philosophies about learning. This study helped to show that the burden of thinking and learning falls upon the students. The integration of content and shared philosophies between CMA and university professors was surprisingly easy and encouraging.

CLOSING THE ACHIEVEMENT GAP: STRATEGIES TO IMPROVE MATHEMATICS ACHIEVEMENT OF ACADEMICALLY DEFICIENT STUDENTS AN URBAN SCHOOL. Taik Kim

THE PERFORMANCE OF U.S STUDENTS IN MATH The

performance of American students in mathematics has been a topic of national concern for more than three decades. Trends in International Mathematics and Science Study (TIMSS): 4th & 8th The Program for International Student Assessment (PISA): 15 year old students (Coordinated by the Organization for Economic Cooperation and Development (OECD)

TIMSS REPORT: AVERAGE MATHEMATICS SCORES OF 4TH & 8TH GRADERS 1995 2007 (4th) (4th)

2011 (4th)

1995 (8th)

2007 2011 (8th) (8th)

Singapo re

590

599

606

609

593

611

England

484

541

542

498

513

507

518

529

541

492

508

509

U.S

THE PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA) 2003

2006

2009

Singapore

N/A

562

U.K

N/A

495

492

U.S

483

474

487

OEDC AVE. 500

N/A

496

SAT TREND 1972

1983

1990

2000

2005

2010

509

503

500

514

520

516

SAT MEAN SCORE BY ETHNICITY

1987

2000

2010

Asian

541

565

591

White

514

530

536

Hispanic

450

459

460

Black

411

426

428

NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS(NAEP)

1990

2004

2008

Age 17

290

285

286

Age 13

257

259

260

Age 9

209

219

220

NAEP SCORE GAP BY ETHNICITY Age 9

Age 13

Age 17

White vs. Black (1990/2008)

27/26

27/28

21/26

White vs. Hispanic (1990/2008)

21/16

22/23

26/21

MAJOR ISSUES IN URBAN SCHOOLS 1. Low student achievement: Generally, students are two or more years below their grade level (Belfanz, Ruby, & MacIver)

2. Teacher quality: There are almost 50% more novice teachers in urban schools than in lower poverty schools. About 70% of math classes are taught by a teacher who doesnâ&#x20AC;&#x2122;t even have a college minor in math or a math-related field. (Peske & Haycock)

3. School environment: Disruptive & unruly behavior in classrooms affect studentsâ&#x20AC;&#x2122; academic achievement (Nelson et al., Sutherland, Alder, & Gunter)

PILOT STUDY Site: 8th grade classroom in an urban school Minority: 97% Economically Disadvantaged: 90.4 % State Indicators: 1 out of 14 School Rating: Academic Emergency Math Teacher: Male, Caucasian, 20 yrs of teaching experience

WHAT HAPPENED Mr. Fair spent most of his time correcting his studentsâ&#x20AC;&#x2122; behavior. Mr. Fair ordered the most unruly students to stay after school with him as punishment. Studentsâ&#x20AC;&#x2122; attitudes became more hostile. Mr. Fair walked away from the math class after 1 month.

MR. FAIRâ&#x20AC;&#x2122;S TEACHING METHODS Explaining the procedure by using an overhead projector Writing notes or formulas on the board Inviting students to answer questions Requesting the completion of worksheets

HOW TO IMPROVE URBAN EDUCATION: SUGGESTIONS 1. Using effective instructional methods: One to one adult-to-child tutoring (Slavin)

2. Improving teacher quality: A higher degree of correlation between various teacher qualities in schools (Lankford, Loeb, & Wyckoff) Improving effectiveness through professional development & improving higher education (Carey) 3. Increasing cultural sensitivity: Identified as a typical reason cited by teachers who leave the teaching profession (Ingersoil)

R3 MATH GROUP The R3 Math curriculum was specially designed to improve learning and teaching: 1. Re-teaching, 2. Re-enforcing, & 3. Re-building fundamental mathematical concepts. Using Intuitive Models Text-based teaching

WHY DO STUDENTS STRUGGLE? No basic foundation Lack of studying habits Poor understanding of basic mathematics skills

MAJOR FACTORS School & class environment Family & personal problems Peer pressure No motivation for the future

RESULTS Percentage of students above and at the proficiency level

Total number of students

Kimâ&#x20AC;&#x2122;s Kim s Class

75%

18 (4 IEP)

LongLong-term Substitute Class

28 ( 12 IEP)

School

16.1%

District

57.3%

N/A

State

71.5%

N/A

TEST SCORE COMPARISON BY GROUP Mean Test scores among Different Groups

group

50.00

R³ A B

Estimated Marginal Means

45.00

C DA

40.00

35.00

30.00

25.00

20.00 Pre

Middle

factor1

Post

R3 Math Group, A: school ‘A’ average math score, B: school ‘B’ average math score, C: school ‘C’ average math score, DA: District average math score.

SUMMARY Motivation is an important factor in studentsâ&#x20AC;&#x2122; learning. Tutoring alone is not enough to improve math skills (95% of all schools in the district provide tutoring programs). Using Intuitive Models can be a very effective teaching method making students take an active role in their own education.

CONCLUSION Improved

studentsâ&#x20AC;&#x2122; behavior along with better test scores Overall, Intuitive Models would be a very effective teaching method

R3 Math curriculum was successful to improve mathematics achievement of students in an urban school.

The

Center for Scholastic Inquiry International Research Conference April 18, 2013

Reasons that inspired a research study at a university early childhood center

Research focus

Elements of one-week research project

Activities, stakeholders, outcomes of study

Implications for educators

Questions

Schools across the nation facing a 21st century challenge: meet the changing academic needs of an increasingly, culturally diverse classroom

Our worldâ&#x20AC;&#x201D;is now an interdependent community - We face global social, economic, and political issues - Studentsâ&#x20AC;&#x201D;at all levels must learn and acquire core skills in literacy, math, science, and technology - To thrive together, children today should be encouraged to respect and embrace each otherâ&#x20AC;&#x2122;s cultural voices and knowledge

Construct a relevant experience that forged a meaningful, trusting partnership between parents, children, educators

Explore the effects on children’s learning as a result of enriching their school experience with multicultural events Explore the ways in which the parents’ voice and cultural knowledge could enrich their child’s school experience Explore opportunities for creating a trusting partnership between parents and educators

Aims: - Familiarize 20, three to five year old children about the rich diversity and family learning practices of several cultures: Chinese, Korean, African-American, Hispanic, and Native-American - Create a dialogue and lasting partnership between parents and educators - Weave household and cultural practices with everyday school learning

Setting: a university’s early childhood center, located in southeast United States

Length of study: one week, Mon. – Fri., 8:00 am – 3:00 pm, July 11 – 20, 2012

Participants: - twenty, 3 – 5 year olds of 5 cultural groups and parents - two undergraduate interns - director of the early childhood center - two university professors - myself, the principal investigator

Gaining Participant Consent - A packet containing schedule of events, permission forms, and a letter describing the study written in English, Chinese and Korean language was sent home to parents

Chosen Classroom Learning Strategy: - Focus on studentsâ&#x20AC;&#x2122; countries: Africa, China, Korea, Mexico, and United States - Lots of hands-on activities for children: drawing, coloring, writing, talking, singing, seeing, hearing, and tasting - Invited guest speakers: gave demonstrations and brought cultural artifacts for children to interact with

Children’s experiences - Counted from one to ten in Chinese, Spanish and English - Said hello in different languages: Hola (Spanish), Jambo (African) - Met different guests - Created salsa (brief explanation) - Learned to sing ‘Happy Birthday’ in Mandarin/Cantonese Chinese - Constructed arts and crafts such as “Korean drum” - Listened and interacted with story readers during read aloud

Learning and embracing Korean cultural practices - Special visitor: Michael Bateman, Chief Martial Arts Instructor - Children learned how to greet each other with a bow and say simple words in Korean - Children enjoyed a special martial arts presentation which involved chopping several pieces of wood with a hand. - During small group and center time, children made their own Korean musical drum (Field Notes, July 2012).

Inclusion of technology - Families with children at child development center narrated a special story on the iPad and posted pictures on a Shutterfly website - Families were encouraged to share--a family tradition, a song in native language, a memorable family story or other form of cultural knowledge - A mother and daughter shared a special song and artwork - A Korean father and daughter shared how they celebrated the daughterâ&#x20AC;&#x2122;s first birthday - Videos were placed on the Shutterfly account for families to view/enjoy www.shutterfly.com

Teachable moment about a Korean child’s first birthday, based on Chinese parents’ knowledge of Confucius tradition - Various objects (money, thread, rice and a pencil) are placed on the table in front of the baby - The baby is encouraged to pick one object...that selected object is viewed as an indicator of the baby’s future in life -- If the baby picks the money, then the child will be rich -- If the baby picks the thread, then the child will live a long life -- If the baby picks a pencil, then the child will be a scholar -- If the baby picks rice, then the child will have enough food throughout their life

The child’s first birthday tradition is changing - “Now some parents put a microphone on the table, which means the baby will become an entertainer, or a golf ball, wishing that the baby will be a famous golf player” (Field Notes, July 2012).

Data collection strategies: - Interviews with parents, children, 2 undergraduate interns - Observations of children’s interaction with cultural events - Used the early childhood center’s Shutterfly account to record parents’ narration of stories, songs, cultural traditions - Used camera to photograph children, presenters, events - Field notes recorded daily by me, the principal investigator - I-pad video recordings - ECC teachers met regularly with each other to reflect on children’s reactions to the study and as necessary adjusted instructional practices

Shutterfly Account: www.shutterfly.com

Interview feedback with two interns: â&#x20AC;&#x153;They agreed that it was important for the children to understand that they are not all the same and to notice their differences. Lunch is a prime example, because children from different cultures eat various foods. A Korean child today, had seaweed for lunch. Also, the students notice the types of Chinese and Korean foods and they are quick to tell you which food is Chinese or Korean. For future recommendations, the interns suggested developing more activities focused on the following: foods, dress, celebrations, and languagesâ&#x20AC;? (Field Notes, July 2012).

“Many of our children have traveled to China, Korea, one to the Middle East, and one to Thailand. For these children, English is a second language.” “While a study of places around the world may not be developmentally appropriate for a child three, four, or five years old, the exposure to such information is unquestionably important.” “Our children learned about different foods, celebrations such as Chinese New Year, birthdays, and were exposed to different dances, costumes, flags, currencies, languages, songs, art, and looking for the specific countries on a globe.”

Theme 1: Childrenâ&#x20AC;&#x2122;s questions and reactions caused us to continually reevaluate our instructional practices for promoting awareness of cultural diversity Theme 2: Children were most excited when able to look at and touch the clothes, taste food items, listen to different speakers, and create crafts of different cultures Theme 3: Parental involvement improved because their cultural knowledge was an integral element of their childâ&#x20AC;&#x2122;s school learning Theme 4: ECC teachers were able to see the childrenâ&#x20AC;&#x2122;s connection with the multicultural activities and many times, the children were the teachers Theme 5: Technology can become an important tool for children and families to document generational stories, traditions, and photos

Implication 1: Allow students to share their knowledge and life experiences; it is a foundation that guides their lens toward school Implication 2: Engagement of students was most obvious when multisensory learning activities were planned: talking, doing, playing, moving, singing, and even tasting Implication 3: Be open-minded toward allowing culturally diverse learning practices to frame what the children experience at school Implication 4: What our children learn is often what they see between parents and teachersâ&#x20AC;&#x201D;this project invited dialogue and forged new multicultural alliances

Bernstein, J., Zimmerman, T., Werner-Wilson, R., & Vosburg, J. (2000). Preschool children’s classification skills and a multicultural education intervention to promote acceptance of ethnic diversity. Journal of Research in Childhood Education, 14(2), 181-192. Bodur, Y. (2012). Impact of course and fieldwork on multicultural beliefs and attitudes. Educational Forum, 76(1), 41-56. Cavanaugh, L. M., & North Dakota Univ., (1995). Multicultural education: A checklist for selecting children’s literature in the classroom. Insights into Open Education. North Dakota University, Grand Forks Center for Teaching and Learning. Costley, K. C. (2012). Who are today’s students in a diverse society? Harke, C. L. (1997). Increasing the use of multicultural education in a preschool located in a homogeneous Midwest community. [master’s thesis] Retrieved from ERIC (ED415008) Lieberman, A. (2011). Can teachers really be leaders? Kappa Delta Pi Record, 48(1), 13-15. Mitchell, L. A. (2009). Becoming culturally responsive teachers in today’s diverse classroom. [Doctoral dissertation]. Retrieved from UMI (3343846). Smith, R. (2012). Educating children of poverty: School action alone is not enough. International Reading Association Reading Today, 29(4), 31-32. Sturgess, J., & Locke, T. (2009). Beyond “shrek”: Fairy tale magic in the multicultural classroom. Cambridge Journal of Education, 39(3), 379-402. Swick, K. J. (1995). Family involvement in early multicultural learning [Abstract]. ERIC Publications, ED 380240, 1-6.

The Confidence to Reach All Learners Using Dual Coding Theory to Maximize Learning

Who are our K-12 students?

(US Department of Education, 2010)

44.9% students of color 20% English language learners 13% of students served in programs for disabled 44.6% of students poverty level as determined by participation in free/reduced lunch programs

Who are our teachers?

(US Department of Education, 2010) 8

83% Caucasian 70% female

So what? Talking Point: Does it matter that teachers and students are demographically different? Discuss with someone sitting near you.

Pedagogy of Confidence (Yvette Jackson, 2011) School-dependent Learners = Fearful Teachers?

Recall Activity Thi

Think about your favorite vacation spot.

Share with a partner: What did you imagine during the recall activity? How did you imagine it?

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Dual Coding Theory

Marzano’s High-impact Practices 1.  In words 2.  In imagery The brain processes and stores information… Classroom Instruction That Works, 2001; 2012

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Brain Researach

Pat Wolfe (2006):

The brain is largely visual and pattern seeking.

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Bridge the GAP

Nonlinguistic Represntations

Equip preservice teachers with teaching strategies that incorporate nonlinguistic representations that make higher order thinking processes accessible to all students.

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Scholarship of Teaching and Learning Grant •  18 secondary (license K-12 or 5-12) participants •  Intentionally taught dual coding theory, modeled use of nonlinguistic representations (including eight Thinking Maps), and required students to use a variety of nonlinguistic representations and maps in lesson plans •  Department diversity awareness survey and list of strategies for working with diverse learners at beginning and end of semester •  Follow up with students in student teaching and beyond

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Thinking Maps David Hyerle’s eight cognitive maps, each of which uses a different higher order thinking process:

Circle Map—define in context

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Bubble Map—describe (adjectives)

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Double Bubble Mapâ&#x20AC;&#x201D;compare and contrast GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

GUSTAVUS ADOLPHUS COLLEGE

Tree Map—classify

gustavus.edu

Brace Map—part to whole

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Flow Map—sequence, process

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Multi-flow Map—cause and effect

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Bridge Map—create analogies

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Early Results •  90% of participants increased their confidence in “ability to engage all students in the learning by using a variety of methods that address diverse learning needs.” •  50% of participants identified use of nonlinguistic representations (nonlinguistic representations, visuals, or Thinking Maps) in a list of effective strategies for use with diverse learners.

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Obstacles •  Preconceived ideas about nonlinguistic Thinking Maps •  View use of nonlinguistic Thinking Maps as elementary or boring •  Inadequate experience in using the nonlinguistic representations •  Too much theory; not enough practice

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

Successes •  Willing to acknowledge the research •  Math and science people with processes

GUSTAVUS ADOLPHUS COLLEGE

gustavus.edu

References Hyerle, D. (2004). Student successes with Thinking Maps: School-based research, results and models for achievement using visual tools. L Alper, S. Curtis, & D. Hyerle, (Eds.). Thousand Oaks, CA: Corwin Press. Hyerle, D. (2009). Visual tools for transforming information into knowledge. Thousand Oaks, CA: Corwin Press. Jackson,Y. (2011). Pedagogy of confidence. New York: Teachers College Press. Marzano, R. J., Pickering, D., Pollock, J. E. (2001). Classroom instruction that works: research-based strategies for increasing student achievement. Alexandria,VA.: Association for Supervision and Curriculum Development. Wolfe, P. (2001). Brain matters: translating research into classroom practice. Alexandra, VA: Association for Supervision and Curriculum Development. Wolfe, P. (2006).Video introducing Thinking Maps. Raleigh, NC: Thinking Maps, Inc. Retrieved from http://www.thinkingmaps.com/videointro.php Â

Faculty and Students Speak Out: When Technology in the College Classroom is Productive or Distractive Dr. Myrna R. Olson and Austin Winger

Then

Olson & Winger, 2013

Now

Olson & Winger, 2013

Instructor’s View

Olson & Winger, 2013

Student’s View

Olson & Winger, 2013

Literature •  Majority of students believe technology enhances their learning and quality of interaction with faculty and peers –  Hands-on aspect was important

•  (Shuell & Farber, 2001)

•  After examining 14 technological learning tools in an undergraduate Marketing class

–  9 tools rated highly –  5 tools that rated as having no influence on learning: • FAQ pages

• Lab-‐only classes

• Online readings

• Electronic discussion group

• Chat room

•  (Clark, Flaherty, & Mo?ner, 2001) Olson & Winger, 2013

Literature •  Review of 66 “clicker” technology studies –  31 studies (with a total of 34 samples) compared measures between students using clickers in class and students not using them –  22 of the samples reported significant increases in student performance when clickers were used »  (Keogh, 2012)

Olson & Winger, 2013

Literature •  Educause Center for Applied Research (ECAR) has studied 100,000 undergraduate students from 184 U.S. institutions –  Students increasingly satisfied with instructors’ use of technology –  Hybrid or blended learning is preferred »  (Dahlstrom, 2012)

Olson & Winger, 2013

Literature •  Study of 129 students in two sections of General Psychology (taught by the same instructor) •  Examined students’ use of laptops in the class (laptops were designated as optional) –  As laptop use increased, class performance decreased »  (Fried, 2007)

Olson & Winger, 2013

Literature •  Web-based survey to 4,491 students from four universities located around the U.S. –  Instant messaging (IMing) during homework –  57% reported that IMing while doing schoolwork had a detrimental effect »  (Junco & Cotton, 2010)

•  Online survey from 269 students at small private university in Pennsylvania

–  How frequently students use cell phones before, during, and after class (text-messaging/texting) –  92% received or sent text messages during class –  30% reported texting in class every day »  (Tinder & Bohlander, 2011)

Olson & Winger, 2013

Literature •  Study of graduate students and faculty (N=197) at the College of Health Sciences in Arizona and Illinois

–  Examined cell phone use during class by graduate students and faculty –  53% of students reported using their cell phones during class –  Both students and faculty admitted to checking for messages during class (students at a much higher rate) »  (Burns & Lohenry, 2012)

Olson & Winger, 2013

Purpose •  Examine the pros and cons of technology use in the college classroom •  Gain insights on how to effectively integrate technology use •  Survey which technologies are most useful

Students & Faculty Olson & Winger, 2013

Does this look familiar? We want to ﬁnd this…

Olson & Winger, 2013

Research Questions 1. What technological devices are faculty using as teaching tools in their classrooms? 2. How do they believe use of these tools enhances student learning? 3. What technological devices are students using

in college classrooms? 4. How do they believe these devices enhance their learning? Olson & Winger, 2013

Research Questions

5. What are faculty members’ experiences with student use of technological devices in the classroom that have distracted students from learning? 6. What with are students’ e xperiences technological devices in the classroom that have distracted them from learning? No. 7 Olson & Winger, 2013

Research Questions 7. What effective ways have professors and/or students found to minimize distractions and increase benefits from the use of technological devices in the college classroom? Searching for the…

Olson & Winger, 2013

Methods Survey

• Qualtrics • Email blast • QuanXtaXve

Interview

• In person • Telephone • QualitaXve

Students, N= 135; Faculty, N= 153

Olson & Winger, 2013

Quantitative Results

Olson & Winger, 2013

Technologies Perceived as Productive for Learning by Students

Olson & Winger, 2013

Technologies Perceived as Productive for Learning by Faculty

Olson & Winger, 2013

Productive Technological Devices Used by Instructors

Olson & Winger, 2013

Factors Influencing Level of Distraction From Technology

Olson & Winger, 2013

Methods for Keeping Technology Use Appropriate in the College Classroom

Olson & Winger, 2013

Qualitative Results •  Use technology where it supports the lesson or assignment •  Understand that sometimes technology can interfere with establishing community •  Learn how to use technology by utilizing tutorials

Olson & Winger, 2013

Qualitative Results •  Give students options for using their technological devices in class to enhance learning •  Consider setting aside a few minutes for “tech breaks,” during which students can check their devices •  Understand that undergraduates may need more guidance on the appropriate use of their technological devices than graduate students. Olson & Winger, 2013

Tech Break

Olson & Winger, 2013

Conclusions Balance EﬀecXveness ProducXvity Support CreaXvity

Olson & Winger, 2013

Looking Ahead •  CollaboraXon among students, faculty, and other university professionals •  Examine new technology as it is created and evaluate its eﬀecXveness in the classroom •  Integrate technology use into university courses when appropriate and supports/ enhances learning

Olson & Winger, 2013

•  Our dependence on technology and our inability to be away from it for even a few minutes is just one clear indicator that we are not functioning at our best level. If your minds are always worrying about what we are missing, how can we focus on what we are getting? »  (Rosen, 2012)

Re-‐think

Re-‐connect Re-‐boot

Re-‐vitalize Olson & Winger, 2013

References •  Burns, S. M., & Lohenry, K. (2010). Cellular phone use in class: Implications for teaching and learning a pilot study. College Student Journal, 44(3), 805-810. •  Clark III, I., Flaherty, B., and Mottner, S. (2001). Student perceptions of educational technology tools. Journal Marketing Education, 23(3), 169-177. •  Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-59. •  Dahlstrom, E. (2012) ECAR Study of Undergraduate Students and Information Technology, 2012. Louisville, CO: Educause Center for Applied Research. •  Garrison, D. R., & Akyol, Z. (2009). Role of instructional technology in the transformation of higher education. Journal of Computing in Higher Education, 21(1), 19-30. DOI: 10.1007/ s12528-009-9014-7 Olson & Winger, 2013

References •  Junco, R., & Cotten, S. R. (2011). Perceived academic effects of instant messaging use. Computers & Education, 56(2), 370-378. •  Keogh, S. (2012). Clickers in the classroom: A review and a replication. Journal of Management Education, 36, 822-847. DOI: 10.1177/1052 5629124 54808 •  Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61(2), 179. •  Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52. •  Rosen, L. (2012). iDisorder: Understanding our obsession with technology and overcoming its hold on us. Macmillan, 2012. •  Shuell, T. J., & Farber, S. L. (2001). Students' perceptions of technology use in college courses. Journal of Educational Computing Research, 24(2), 119-38. •  Tindell, D. R., & Bohlander, R. W. (2012). The use and abuse of cell phones and text messaging in the classroom: A survey of college students. College Teaching, 60(1), 1-9. Olson & Winger, 2013

Discussion •  Questions? • Comments? • Sharing experiences

Olson & Winger, 2013

A NARRATIVE STUDY OF TEACHER AND STUDENT PERCEPTIONS OF CORRECTIVE FEEDBACK ON SPEECH PRODUCTION IN ENGLISH

By: Cynthia Alvarado, ABD

Abstract

The narrative research will focus on examining teacher and student perceptions of corrective feedback on speech production in English. Research on examining perceptions of corrective feedback has not been highlighted individually within teachers and students nor specifically given a voice in terms of a narrative study. This research proposal represents an investigation on perceptions of corrective feedback that occurs amongst teachers and students who are English language learners.

Chapter 1: Intro

According to Ellis (2009), in both structural and communicative approaches to language teaching, feedback is viewed as a means of fostering learner motivation and insuring linguistic accuracy.

Corrective feedback consists of 2 types: positive & negative.

By drawing attention to verbal behaviors deemed to be problematic and responding to them in particular ways, corrective feedback routines constitute a central focus into a linguistic community.

Statement of the Problem

Excessive corrective feedback on error can have a negative effect on motivation and teachers must be sensitive to studentsâ&#x20AC;&#x2122; reactions to correction (Lightbown & Spada, 2006).

- Research on examining perception and understanding of corrective feedback has not been highlighted individually within teachers and students - This area of research has not been given a voice in terms of a narrative study

Purpose of the Study

The purpose of this narrative study is to fill in the gap within the existing literature by providing a descriptive narration from both teachers and students. - By focusing specifically in one area that corrective feedback is directed, the researcher will be able to gather meaningful data that is described in-depth and is not distracted by other determining factors - The purpose will be achieved by documenting the participants statements through focus groups, questionnaires and member checks.

Teachers

Students

How do English language learning teachers perceive corrective feedback when teaching ELL students?

How do English language learners perceive corrective feedback in the ELL classroom?

What do the teachers understand about corrective feedback?

What do the students understand about corrective feedback?

Research Questions

Significance of the Study

The research will bring the statement of the problem to the consciousness of the participants and wider educator audience by creating an occasion for self-reflection and providing them with the opportunity to share their perceptions and understandings. Findings through this research can provide insight into both teacher and student realities among corrective feedback and thus create avenues for new teaching strategies / student consideration for the classroom

Chapter 2: Review of the Literature The only place where feedback on error is typically present with high frequency is the language classroom (Lightbown and Spada,1999).

Brown (2007) explains that teachers should respond to studentsâ&#x20AC;&#x2122; attempt to communicate with positive affirmation.

Constructivist Theory

Constructivism is a theory about knowledge and learning; it describes both what “knowing” is and how one “comes to know”(Fosnot, 1996).

Our prior experiences, knowledge, and learning affect how we interpret and experience new events; our interpretations, in turn affect construction of our knowledge structures and define our new learning (Marlowe & Page, 2005).

The individuality of the constructs is what constitutes that no two people will have the same experiences.

Teachers’ Perceptions on CF Although the need for CF may be present, the perception the teacher has toward CF may affect their decision.

Teachers can and do tailor their classroom interactions to address the needs of specific learners (Kennedy, 2010).

Vann, Meyer, and Lorenz (1984), for example, found that some teachers were inclined to view all errors as serious – “an error is an error”.

Studentsâ&#x20AC;&#x2122; Perceptions of CF

The overt use of corrective feedback in the classroom could lead the student to focus more on the behavior of the teacher rather than what they should be learning in the classroom.

Bang (1999) found that most students felt that oral correction was necessary for language learning, but they disagreed on when and how it should be done.

Chapter 3: Methodology Research Design

Validation & Reliability

Data Collection

Research Questions

Sampling Process

Research Design Qualitative narrative approach:

Narrative inquiry follows a reclusive, reflexive process of moving from field (with starting points in telling and living stories) to text (data) to the interim and final research texts(Clandinin & Huber, n.d.).

The study of experience serves as an inductive research strategy that involves a richly descriptive report that is needed for this study.

Setting

Intensive English Program (IEP)

The IEP program trains students in their oral and written skills which will help them enter their university degree program and complete it.

Students enrolled in the IEP program are foreign students who will be immersed into an English-speaking environment with the hopes of developing their second language skills.

Sampling Process Teachers • Currently employed with the Intensive English Program • Must have at least 1yr of experience teaching English language learners

Students • Currently enrolled in the Intensive English Program • English language learners

Data Collection Procedures Focus Groups • 45 min • Face-to-face as a group • Interview protocol • Audio recorded • Primary data

Questionnaire • 15-20 min • Open ended questions • One-on-one • Interview protocol • Audio recorded • Secondary Data

Member Checks • 20-30 min • Participants will review transcriptions w/ researcher • Researcher will discuss interpretations w/ participant for accuracy

Data Analysis

Data will be analyzed according to the Ginsburg Narrative Model - Present Narrative Segments - Statements on Theoretical Issues - Key Substantial Themes - Explain Interpretive Commentary & Generalizations

All focus groups transcriptions were studied inductively, and the responses will be themed according to the questions asked.

Validity & Reliability: Triangulation Focus Groups

Perception & Understanding

Questionnaire

Member Checking

Themes - Teachers 1. Corrective Feedback is necessary. 2. Teachers need to get to know students. 3. Teachers need to be positive. 4. Appropriate timing/Environment should be considered for CF. 5. Corrective Feedback is constructive criticism. 6. Emotions should be separated from CF.

Themes - Students 1. Corrective Feedback is necessary. 2. Teachers need to get to know students. 3. Teachers need to be positive. 4. Appropriate timing/Environment should be considered for CF. 5. Corrective Feedback is constructive criticism.

6. Emotions should be separated from CF.

Helpful

Reduces future Errors

Similarities

Situational

Needed

Differences Teachers

Students

• Constructive • Emotions should be separated

• Negative Connotation • Emotions are needed

Recommendations 1 2 3 4

• Target a new environment where more participants can be researched

• Utilize individual background questionnaires / compare & contrast amongst groups • Develop a survey for a mixed methods study – provide the ability to study the participants responses that are given independently • Consider working with English speaking students who are learning a new language

Reference

Bang,Y. (1999). Reactions of EFL students to oral error correction. Journal of Pan-Pacific Association of Applied Linguistics, 3, 39-51. Clandinin, D., & Huber, J. (2010). Narrative inquiry. In B. McGaw, E. Baker, & P. Peterson (Eds.), International encyclopedia of education (3rd ed.). New York,NY: Elsevier. Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1, 3-18. Fosnot, C. (1996). Constructivism:Theory, perspectives, and practice. New York, NY: Teachers College Press. Kennedy, S. (2010). Corrective feedback for learners of varied proficiency levels: A teacherâ&#x20AC;&#x2122;s choices. TESL Canada Journal, 27(2). Lightbown, P., & Spada, N. (2006). How languages are learned (3rd ed.). New York: Oxford University Press. Marlowe, B., & Page, M. (2005). Creating and sustaining the constructivist classroom. California: Corwin Press. Sage Publications Company. Vann, R. J., Meyer, D. E., & Lorenz, F. O. (1984) Error gravity: A study of faculty opinion on ESL errors. TESOL Quarterly, 18(3), 427-440.

The Professional Exploration Program: An Alternative Admissions Process Linda Kisabeth, Associate Professor Thomas M. Cooley Law School Auburn Hills, Michigan

2013 Center for Scholastic Inquiry International Research Conference

Early Origins of PEP 2001 – Use LSAT writing sample in admissions process? Good writing ability on the LSAT = law school success? Writing sample as key criteria for invitation to “Q school” 2003 Professional Exploration Program established 2004 Group of 35 admitted on writing sample alone

Who Gets Invited to PEP Most applicants who donâ&#x20AC;&#x2122;t meet admission index have LSAT writing sample scored according to a rubric Participation in program by invitation only based on writing sample score Exception: Group admitted without PEP (35) 3

How They Get Invited to PEP Rubric established by Chairperson of Research & Writing Department Writing samples assessed by PEP skills instructor Writing sample scores 2.5 or better (5.0 scale) invited to PEP 4

The PEP Program One week, off campus (hotel in Lansing, Mich) 9-10 hours Torts class (Intentional Torts) 12 hours legal skills instruction (Intro to Law) Self-evaluation of â&#x20AC;&#x153;fitnessâ&#x20AC;? to study law Academic & career testing (Nelson-Denny Reading Test, Myers-Briggs Type Indicator & Strong Interest Inventory) Tour of Lansing, Michigan campus Legal skills quiz Final exam on Torts (1 essay, 14 MC) 5

Who Gets Admitted to Cooley Selection committee = PEP faculty member & skills instructor, Director of ARC, Dean of Faculty, Dean of Students, Dean of Admissions and Dean of Enrollment & Student Services Decision to admit based on review of individual performance, character & professionalism assessment 6

After Theyâ&#x20AC;&#x2122;re Admitted Academic Assistance Plan = 2 meetings with PEP staff, meet with ARC Director if ND is low, attend 2 ARC seminars & pass Intro to Law first term 9 credits first 2 terms Lansing, Auburn Hills or Grand Rapids campus first 2 terms (Ann Arbor / Tampa, FL allowed later) Transfers pay $1,000 - estimated cost individual participation in PEP 7

PEP Data Studied 55 Programs = Jan 2003 through Dec 2012 9,899 writing samples assessed using rubric 4,116 scored 2.5 or higher invited to PEP 1,425 attended (35% of those invited) 829 of 1,425 participants offered admission 8

PEP Success Rates 136 of the 829 could not be classified so group studied became 693 Michaelmas Term (Sept-Dec) 2012 - 79% of 693 are successful

Non-PEP Group Success Rates 35 students admitted on writing sample alone without PEP 48% successful

PEP Participant Profile Averages of 829 participants offered admission Age = 27 UGPA = 2.71 LSAT = 139 Writing Sample = 2.7 11

PEP Demographics 494 Female (60%) 335 Male (40%) 317 African-American (38%) 66 Asian (8%) 15 Mexican American (2%) 8 Puerto Rican (1%) 56 Other Hispanic (7%) 40 Other, multiracial (5%) 3 Native American (less than 1%) 14 Did Not Indicate (1%) 519 Underrepresented Populations = 63% 12

Data Analysis: Why PEP Works 1. Writing sample predict success? 2. PEP skills quiz predict success? 3. PEP final exam predict success?

Writing Sample Score Results Writing Sample Score

Successful Unsuccessful Students Students

Success Rate

2.5 = 463 students

359

104

78%

3.0 = 198 students

165

83%

3.5 = 25 students

68%

4.0 = 7 students

57%

TOTAL: 693 students

545 (79%)

148 (21%)

Skills Quiz Results Skills Quiz Score

Successful Unsuccessful Success Students Rate Students

50% - 59% = 13 students

46%

60% - 69% = 42 students

64%

70% - 79% = 213 students

157

74%

80% - 89% = 335 students

280

84%

90% - 100% = 80 students

86%

No Score = 10 students

Total = 693 students

545 (79%)

148 (21%) 15

Final Exam Results Final Exam Score (68 Points Possible)

Successful Students

Unsuccessful Students

Success Rate

20 – 29 = 37 students

78%

30 – 39 = 293 students

236

81%

40 – 49 = 178 students

146

82%

50 – 59 = 26 students

92%

60 – 68 = 1 student

100%

N / A = 158 students

109

Total = 693 students

545 (79%)

148 (21%) 16

Reminder Success Rates December 2012 â&#x20AC;&#x201C; 55 programs total 1,425 participants, 829 offered admission Michaelmas 2012 - 79% successful (136 no classification, group studied = 693) Reminder â&#x20AC;&#x201C; 48% Non-PEP group successful

Conclusion Writing sample not as strong a predictor of success as PEP performance Writing sample still viable assessment tool in deciding whom to invite to PEP PEP performance clear predictor of academic success PEP is selecting participants that become successful law students more than 3 out of 4 times 18

Most Important Cooley’s mission includes “providing broad access to those who seek the opportunity to study law” Professional Exploration Program is a viable alternative admissions process – it works Embodies Cooley’s mission and allows hundreds of individuals, particularly those from underrepresented populations, to achieve a goal that may not otherwise have been attainable

Urban Charter Schools and Factors that Influence the Achievement of students from Culturally and Linguistically Diverse (CLD) Backgrounds York Williams, Ph.D. Assistant Professor of Special Education West Chester University

The Center for Scholastic Inquiry (CSI) Educational Conference Scottsdale, AZ Friday April 19, 2013

Action Research • Traditional research may sound straightforward, however, classrooms and school settings are difficult to control without disrupting the natural classroom environment. (Creswell, 2007; Mills, 1991) • Action researchers acknowledge and embrace the complicated nature of classroom environments rather than try to control them. • Action research is any systematic inquiry conducted by teacher researchers, principals, school counselors, or other stakeholders in the teaching/learning environment to gather information about the ways that their particular schools operate, how they teach, and how well their students learn.

Defining Action Research • Action research engages teachers in a fourstep process that includes: – Identifying an area of focus – Collecting data – Analyzing and interpreting data – Developing an action plan

Critical Action Research â&#x20AC;˘ Critical action research is also known as emancipatory action research because of its goal of liberation through knowledge gathering. â&#x20AC;˘ Critical action research also draws heavily from a body of theory called postmodernism.

Action Research Models

Meet Genner Genner Middle School is located in an urban school district in one of the most high-needs communities, on the west side of a major metropolitan city in the Mid Atlantic. There are a number of isolated communities that border the middle school that also have high crime and neighborhood violence. These communities possess students who are somewhat equally dispersed among the districts seven regional middle schools. The middle school possesses the lowest academic achievement out of all seven schools in this education cluster and the school has not made AYP its tenth year in a row, but did gain status of safe harbor for its ELL students under NCLB (2001). Additionally, this middle school possesses the largest number of diverse students who perform at the below proficient range. Genner has just been taken over by an Educational Management Organization (EMO) this school year because it has been declared a â&#x20AC;&#x153;Turn Aroundâ&#x20AC;? Renaissance charter School. The principal and school staff members are all new this year. However, there are problems with student behavior, achievement, and special education services. There is strong parent participation and teacher buy-in, and the school leader, Dr. Harris, remains open to ideas, but needs help.

Genner Cont. However, there are issues with teachers managing behavior, as well as an increase in behavior referrals which has raised alarm at the District Central Office. Further, the level of identification for students with learning disabilities in grades K-3 are at an all time high, with a back log of referrals. There are concerns here, and these may cause the school to lose it licensure after a few years, but more so, the students may continue to suffer. Lastly, there is no Child Study Team in place, and no Response to Intervention (RTI), so students do not have a school wide or tiered intervention process in place, and instead go straight to referral.

Questions We Raise: 4-Step Reflective Process • What is the problem here? • Identifying an area of focus • Collecting data

• Recommendations/Solutions? • Analyzing and interpreting data • Developing an action plan

• How will and How do you know?

(AYP)School Data from 2012

SD AYP 2012 Data Math

(AYP)School Data from the Two Districts 2012

SD AYP 2012 Data Reading

Discussion Data that should drive school reforms will allow us to be more focused. During the 2011-12 school year we will focus on the following areas for intervention: a. Reading and Math b. CRT with RTI c. SWPBS using RTI d. Understanding Urban Schools & SPED e. Culturally Responsive Assessment

Hypothesis As a result of targeting the areas of assessment, RTI, Behavior Support, and Culturally Responsive Teaching, we believe all the other areas will improve and learning can occur ( Montani & Frawley, 2011; Pavri, 2010).

Intervention 1: Reading and Math RTI The data shows that the schoolâ&#x20AC;&#x2122;s sub groups are far below proficient. This may be linked to teaching, poor skills, behavior or the curriculum. Fuchs et al. (2010) and Williams (2012) suggest that through implementing RTI, collaborative intervention can minimize the impact of misidentifying a learning disability.

Response to Intervention â&#x20AC;˘ RtI is the practice of (1) providing high-quality instruction/intervention matched to student needs and (2) using learning rate over time and level of performance to (3) make important educational decisions.(Batsche, et al., 2005) â&#x20AC;˘ Problem-solving is the process that is used to develop effective instruction/interventions.

RTI Background and Outcomes - In 2004, Congress reauthorized the Individuals With Disabilities Education Improvement Act (IDEIA 2004) - - Included landmark language in that law to encourage schools to break free of their reliance on the discredited IQAchievement Discrepancy method for identifying Learning Disabilities (Wright, 2007). - IDEIA 2004 gave schools the freedom to use the student’s “response to scientific, research-based intervention” diagnostically as a prime indicator of whether the child has a Learning Disability.

RTI Background and Outcomes • Maximize effect of core instruction for all students • Targeted instruction and interventions for at-risk learners • Significant improvements in pro-social behaviors • Reduction in over-representation of diverse student groups in low academic performance, special education, suspension/expulsion, and alternative education. • Overall improvement in achievement rates • Maximize efficiency and return on investment • AYP

Three-Tiered Model of School Supports & the Problem-solving Process ACADEMIC SYSTEMS

BEHAVIOR SYSTEMS Tier 3: Comprehensive & Intensive Students who need individualized interventions.

Tier 2: Strategic Interventions Students who need more support in addition to the core curriculum.

Tier 1: Core Curriculum All students, including students who require curricular enhancements for acceleration.

Tier 3: Intensive Interventions Students who need individualized intervention.

Tier 2: Targeted Group Interventions Students who need more support in addition to school-wide positive behavior program.

Tier 1: Universal Interventions All students in all settings.

RTI: Grows & Glows Affirmative

Negatives

(1) Valid student evaluation will become effective (2) There may be a decrease in students misidentification (3) All teachers will be trained (4) Fewer referrals to cut down on back log for the school psychologist. (5) Parent awareness as to how their child positively responds to instructional interventions

(1) (2) (3) (4)

RTI not done correctly and no impact More paper work and no assessment More referrals for RTI, than SPED Transferring needed resources from elsewhere in the school which could create an imbalance (5) Reproduction of the same problems

Intervention 2: CRT Gay discusses two focuses of culturally responsive teaching; how culture and social behavior influence teachersâ&#x20AC;&#x2122; attitudes and affects CRT, and how (CRT) can be used for special education. The best education for ethnically diverse students is just as much culturally responsive as it is developmentally appropriate. In other words, teachers should look at cultural orientations, background experiences, and ethnic identities as ways to facilitate their own teaching and learning within the classroom (2002).

Application of CRT History Surveys Music

Dance

Art

Poetry

Name Games

Style of Dress

ELL Night

Athletics

Courageous Conversations

Learning Styles

Expressions

Gifted Inclusion Literature Faire

Parental Participation

Language Expression Games

Interactions Style

Transition Activities (PCP)

Pop Culture

Social Work

Community Collaboration

Teacher Training

Grandparents Day

Desk Top Publishing

Counseling Activties

Assessment We will conduct a pre-assessment amongst the teachers and staff followed with observations, in a partnership with the local college SPED department to see what free training and internship opportunities exist. We will conduct a post assessment 3 months after the initial pre assessment and measure the effectiveness ongoing. Formatively, we will ask to observe teacher classes where they maintain CRT is taking place.

CRT: Grows & Glows Affirmative

Negatives

(1) CRT & student assessment (2) There may be a decrease in students behavior referrals (3) All teachers will be trained (4) Fewer referrals to cut down on back log for the school psychologist. (5) Parent awareness as to how their child shows academic and social progress

(1) (2) (3) (4)

CRT not done correctly and no impact More lip-serve and BHM no CRT More referrals for RTI, than SPED Only certain teachers will integrate CRT (5) Reproduction of the same behavior and learning problems

Intervention 3: PSWBS & RTI 1. Secure administrator agreement of active support and participation. 2. Self assessment of the current school-wide discipline system. 3. Data collection on a regular basis. 4. School-wide leadership team to guide and direct the process. 5. Commitment from at least 80% of the staff for active support and participation. 6. Implementation action plan based on data.

SWPBS has demonstrated • Improved overall performance of students (i.e., increases in positive interactions, decreases in ODRs • Improved performance of individual students with chronic problem behavior (i.e., being able to state school-wide expectations, • increases in self-management).

Secondary prevention (targeted intervention) supports students at risk.

Check in Check Out: A Targeted Intervention

SWPBS & RTI: Grows & Glows Affirmative

Negatives

(1) Positive behavior assessment (2) There may be a decrease in students behavior referrals (3) All teachers will be trained â&#x20AC;&#x201C; School Wide (4) Fewer EBD referrals to cut down on back log for the school psychologist. (5) Parent awareness as to how their child shows behavioral and social progress

(1) SWPBS not done correctly and no impact on school or students (2) More individualized Aversive Behavior Management (3) More behavior referrals for IAES (4) Only certain teachers will integrate participate (5) Reproduction of the same significant behaviors that impede learning

Summary & Conclusions With the amount of poor communication and the growing cultural gaps on the part of the school for so long, â&#x20AC;&#x2DC;identifiedâ&#x20AC;&#x2122; students would have continued to suffer from poor school teaching, behavior management, assessment and the achievement gap. Now, all students are assessed, and have a FAPE, and one that is measurable and effective, with a team approach. There can now be linkages to student strengths also transition goals, including positive behavior interventions. We feel now that by linking the home with the students needs and making the parents our allies, that we are off to a good start for Glennerâ&#x20AC;&#x2122;s population of students with special and other needs. Glenner students have a future. IDEA 2004 maintains that PSWBS, RTI, relevant assessment, culturally responsiveness with non-biased evaluations improves the school for all students.

Summary & Conclusions (Only 3 of Five Interventions Addressed) a. You want good graphics and images b. You want good flow of ideas c. You may want to have additional pages for charts and graphsâ&#x20AC;Ś

References • • • • • • • • • • • • • • • •

Kavale, K. A., & Spaulding, L. S. (2008). Is Response to Intervention good policy for specific learning disability? Leaning Disabilities Research & Practice, 23, 169-179.

Pavri, S. (2010). Response to intervention in the social-emotional-behavioral domain: Perspectives from urban schools. TEACHING Exceptional Children Plus, 6(3), 1-15. Montani, T.O. & Frawley, P. (April 2011). Culturally relevant informal assessment practices incorporated into educational evaluations. Annual Council for Exceptional Children, National Harbor, MD.

Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003).Responsiveness-to-intervention for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157–171. Williams, Y. (2012). An investigation of the factors that impact effective response to intervention (RTI) practices in urban charter schools. The National Center for the Study of Privatization in Education (NCSPE).

Supporting Value-Added Claims for the Accreditation of an Educational Leadership Program Using a Modified Solomon FourGroup Design

John Hanes, PhD, & Glenn Koonce, EdD Regent University Center for Scholastic Inquiry International Academic Research Conference April 19, 2013 Scottsdale, AZ

Presentation Goals 1. Discuss the movement to Value-Added evidence 2. Review the Educational Leadership Program Logic Model in terms of value-added evidence 3. Review the Systems Evaluation Partnership (SEP) Model in terms of value-added evidence 4. Describe initial steps to develop value-added evidence 5. Outline the Solomon Four-Group Design 6. List steps for gathering value-added evidence 7. Explore related issues

Higher Education: Movement to Value-Added Evidence 1 Over the past 20 years, two major shifts have occurred: 1. From emphasizing inputs to charting outcomes Inputs measure institutional capacities (e.g., libraries), student quality, faculty qualifications curriculum, and pedagogy Outcomes measure graduatesâ&#x20AC;&#x2122; characteristics Liu, O. L. (2011). Outcomes assessment in higher education: Challenges and future research in the context of Voluntary System of Accountability. Educational Measurement: Issues and Practice, 30 (3), 2-9.

Higher Education: Movement to Value-Added Evidence 2 Over the past 20 years, two major shifts have occurred: 2. From outcome status to value-added Outcomes convey information about graduatesâ&#x20AC;&#x2122; knowledge, attitudes, and behaviors Value-added addresses how the outcomes were generated (i.e., by the program itself, by the initial quality of the students admitted, etc.)

The Pressure for Value-Added 1 Some major sources listed hierarchically: 1. U.S. Department of Education(DOE) 2. DOEâ&#x20AC;&#x2122;s Commission on the Future of Higher Education 3. The Council for Higher Education Accreditation (CHEA)

The Pressure for Value-Added 2 4. The roughly 80 national and regional accrediting agencies including SACS and CAEP for Regent Univ. 5. Higher education organizations such as the American Association of State Colleges and Universities (AASCU) 6. Parents and students seeking comparisons for college selection in an age of economic challenge

An Undergraduate Value-Added Assessment Program AASCU is a co-sponsor of the Voluntary System of Accountability (VSA) which has 300+ institutions VSA uses assessments such as the Educational Testing Service Proficiency Profile to measure institutional performance based on student scores Assessments are made with a cross-sectional design measuring both freshmen and seniors in the same year in order to generate value-added data

Value-Added Graduate Program in Teacher Education Current Status 1 • Poised to expand into the nation’s teacher colleges • Data could help determine which teacher education pathways produce effective teachers • Two states with the most experience using such data, LA & TN have shown they can be a powerful catalyst for change

Value-Added Graduate Program in Teacher Education Current Status 2 • Both can point to programs that have seen improvements in value-added scores after altering aspects of their programming • For a concept that is only about a decade old, value-added for teacher education is already in at least 14 states • Except for Louisiana, the most mature system to date, teacher preparation programs have not been overly receptive

The Regent University School of Education (SOE) Educational Leadership Program Established in 1995, it has grown to be a national standardsbased (Interstate School Leaders Licensure Consortium, ISLLC) program receiving national accreditation from the Teacher Education Accreditation Council (TEAC) on January 9, 2009 One the first two educational leadership programs nationally to receive independent program accreditation from TEAC TEAC is now combining with NCATE to form the Council for the Accreditation of Educator Preparation (CAEP)

Five Models for Our Educational Leadership Program 1. Program Logic Model 2. Systems Evaluation Partnership (SEP) Model 3. Distinctions, Systems, Relationships, Perspectives (DSRP) Model 4. Understanding by Design Model 5. Christian Service Model

We will only include the first two models for their utility with the value-added approach

The Program Logic Model and Value-Added 1 TEAC initially noted three levels of claims that a program could make for itself: Status (key to TEAC accreditation). Reflects a claim about graduatesâ&#x20AC;&#x2122; knowledge, attitudes, or behaviors Value-Added (treated as subsidiary). Shows graduatesâ&#x20AC;&#x2122; improvement (hopefully) due to the program. Causal (treated as subsidiary). Involves control group to show that program caused improvement.

The Program Logic Model and Value-Added 2 We only made status claims concerning the outcomes for our graduates and program completers for the initial accreditation Our Program Logic Model, however, anticipated the value-added and causal claims We noted these potential subsidiary claims under the question: Compared to what? We located these claim possibilities between symbols representing our students and representing prior or current cohorts.

TEAC Logic Model Leadership Education FEEDBACK [Look longitudinally for patterns, trends, subgroup analyses, improvements, and sustainability]

SOURCES

INPUTS (Capacities) Radically Improved NEEDS: K-12 Education

ACTIVITIES Assure Quality

OUTPUTS

OUTCOMES

Prepare/equip Students to Lead

----------------Status Claims [Sec.2] -------------ISLLC Quality Cross-cutting Ultimate V

Appn. D Regent University

C H

Appn. C

Curriculum

Faculty

[Sec.1] School Systems

Internal Audit Arena

Facilities Supplies

Appn. A

R I

Faith

Fiscal Admin.

Student support

S TEAC

Admissions Advertising

T Partnerships

QP 2: Assessments (reasonable, credible, reliable, valid) QP 3.1: Planning based on evidence QP 3.2: Influential Quality Control system

Credit hours completed

Service Arena

Graduates

---Appn. B--(Parity), QP 4

M C

Course Quality

SOE Students

[Sec.3-5] Entry --Appn. F-- Assessments

Compared To What? Value-added & causal claims QP 3

Technology

K-12 Achievement

Strategic Decision-making

Multicultural

Seeking Serving Edifying

Caring Leadership Skills

Life-long Learning

Closing Gaps

E P

Student Satisfaction

References in RefWorks [Sec. 6] Student feedback

Professional Knowledge

Prior or Current Cohorts

Set internal targets & external benchmarks

Maintain consistency: Internal – approach & organization of Briefs External – with SCHEV, Title I & II, VITAL, ETS, State test scores

------------QP 1----------(Required in Crosswalk)

Unsupported Claims

-----Evidence for Status Claims [Sec. 4-5] ---Graduates Appn. E: 1, 2, 3, 5, 6, 7, 8, 9, 11

Post-Hoc Appn. E: 12, 13, 14, 15, 16, 17, 19

Payoff Appn. E: 4, 10, 18, 20, Futures Evaluation

The Research Question To what extent does the Regent University SOE Educational Leadership Program add value to studentsâ&#x20AC;&#x2122; competence in leading a K-12 school?

The Research Hypothesis The Regent University SOE Educational Leadership Program adds statistically significant value to achievement scores on the SLLA examination, a proxy for leadership competence in K-12 schools.

Participants SOE Masters Degree Students in Educational Leadership (Ed. Lead.) SOE Masters Degree students in other fields of Education (Other SOE) Masters Degree students in fields of study other than Education (Other Regent) All students are enrolled at Regent University.

Sampling Stratified Random Sampling across these three groups: 1. Educational Leadership Masters Degree 2. Other SOE Masters Degree 3. Other Regent Masters Degree We estimate that budget constraints will limit our sample size to 50 students maximum.

Instrumentation For a variety of reasons, we will utilize the School Leaders Licensure Assessment (SLLA) examination for the main instrument of our investigation. We believe that it represents a reasonable proxy or indicator for competent leadership practice, a proposed major step toward the autoptic proference of enhanced K-12 student learning that we seek.

The Systems Evaluation Partnership Model and Value-Added We needed an assessment vehicle to drive a first effort to measure a value-added element attributable to our program Our previously developed SEP model revealed the School Leaders Licensure Assessment (SLLA) as a good candidate for such a vehicle The general SEP model focuses on the theory/practice interface with attention to the connection of various components of a program and its outcomes

Regent SOE Educational Leadership Systems Evaluation Partnership Model ------------------- Research / Theory -------------------Integrate Faith and Learning

----- Practice ----Seeks, Serves, Edifies

Improved CL Results Course F&L Integration Increased Church, Civic Contributions

Develop Courses, Select Texts Improved SLLA Results Program Curriculum

Administer Ed. Lead. Program

Improved K-12 Achievement

Competent Practitioners Align ISSLC, TEAC, VADOE

Improved CPO’s, GPA’s Improved Internship Evaluations

Capable Students Credit Hours Completed

Recruit and Admit Students

Key:

Activity

Increased Professional Contributions

Output

Short-Term Outcome

Closed Gaps

Medium-Term Outcome

Long-Term Outcome

Regent SOE Educational Leadership Throughlines ------------------- Research / Theory -------------------Integrate Faith and Learning

----- Practice ----Seeks, Serves, Edifies

Improved CL Results Course F&L Integration Increased Church, Civic Contributions

Develop Courses, Select Texts Improved SLLA Results Program Curriculum

Administer Ed. Lead. Program

Improved K-12 Achievement

Competent Practitioners Align ISSLC, TEAC, VADOE

Improved CPO’s, GPA’s Improved Internship Evaluations

Capable Students Credit Hours Completed

Recruit and Admit Students

Key:

Activity

Increased Professional Contributions

Output

Short-Term Outcome

Closed Gaps

Medium-Term Outcome

Long-Term Outcome

Regent SOE Educational Leadership Low Hanging Fruit ------------------- Research / Theory -------------------Integrate Faith and Learning

----- Practice ----Seeks, Serves, Edifies

Improved CL Results Course F&L Integration Increased Church, Civic Contributions

Develop Courses, Select Texts Improved SLLA Results Program Curriculum

Administer Ed. Lead. Program

Improved K-12 Achievement

Competent Practitioners Align ISSLC, TEAC, VADOE

Improved CPO’s, GPA’s Improved Internship Evaluations

Capable Students Credit Hours Completed

Recruit and Admit Students

Key:

Activity

Increased Professional Contributions

Output

Short-Term Outcome

Closed Gaps

Medium-Term Outcome

Long-Term Outcome

Regent SOE Educational Leadership Grand Central Station and Do or Die ------------------- Research / Theory -------------------Integrate Faith and Learning

----- Practice ----Seeks, Serves, Edifies

Improved CL Results Course F&L Integration Increased Church, Civic Contributions

Develop Courses, Select Texts Improved SLLA Results Program Curriculum

Administer Ed. Lead. Program

Improved K-12 Achievement

Competent Practitioners Align ISSLC, TEAC, VADOE

Improved CPO’s, GPA’s Improved Internship Evaluations

Capable Students Credit Hours Completed

Recruit and Admit Students

Key:

Activity

Increased Professional Contributions

Output

Short-Term Outcome

Closed Gaps

Medium-Term Outcome

Long-Term Outcome

Regent SOE Educational Leadership Hubs ------------------- Research / Theory -------------------Integrate Faith and Learning

----- Practice ----Seeks, Serves, Edifies

Improved CL Results Course F&L Integration Increased Church, Civic Contributions

Develop Courses, Select Texts Improved SLLA Results Program Curriculum

Administer Ed. Lead. Program

Improved K-12 Achievement

Competent Practitioners Align ISSLC, TEAC, VADOE

Improved CPO’s, GPA’s Improved Internship Evaluations

Capable Students Credit Hours Completed

Recruit and Admit Students

Key:

Activity

Increased Professional Contributions

Output

Short-Term Outcome

Closed Gaps

Medium-Term Outcome

Long-Term Outcome

Initial Steps to “Value-Added” Utilizing the SLLA The SLLA (School Leaders Licensure Assessment) measures whether entry-level leaders have the standards-relevant knowledge believed necessary for competent professional practice. The SLLA is a fair and carefully validated assessment for use by states as part of the “LICENSURE” process for principals, superintendents, and school leaders. The School Leadership Series, Educational Testing Services (ETS), 2012 www.ets.org/sls

SLLA The SLLA reflects the most current research on professional judgment and experience of educators across the country. It is based on both national job analysis studies and a set of standards for school leaders called the Interstate School Leaders Licensure Consortium (ISLLC) standards. ETS (2012) www.ets.org/sls ISLLC standards were developed by the Council of Chief State School Officers (CCSSO) in collaboration with the National Policy Board on Educational Administration (NPBEA) to help strengthen preparation programs in school leadership. CCSSO (2008) www.ccsso.org/documents/2008/educational_leadership_policy_standards_2008.pdf

ISLLC Standards There are six ISLLC standards. Each standard is followed by the Knowledge required for the standard, the Dispositions or attitudes manifest by the accomplishment of the standard, and Performances that could be observed by an administrator who is accomplished in the standard. Council of Chief State School Officers. (1996). The interstate school leaders licensure consortium standards for school leaders. Washington, DC: Author.

â&#x20AC;&#x153;As with other professions, a test of knowledge and skills based on professional standards is a prerequisite to practice and separates the QUALIFIED from the UNQUALIFIEDâ&#x20AC;? (Adams & Copeland, 2005, pp. 20-21). Adams, J.E., & Copland, M. A. (2005). When learning counts: Rethinking licenses for school leaders. Seattle, WA: Wallace Foundation.

SLLA: Test-at-a Glance Test Name: School Leaders Licensure Assessment Test Code: 6011 Computer Delivered Online Test Length: 4 Hours Format: Section I – 100 Multiple Choice Questions ISLLC Standards I – V 2 Hours 20 Minutes Section 2 – 7 Constructed Response Questions ISLLC standards I, II & VI 1 Hour 40 Minutes Test Scored: Online and by Phone ETS The Praxis Series: www.ets.org?Media/Tests/SLS/Pdf/6011.pdf

Utilizing the SLLA 1 1. The School Leaders Licensure Assessment (SLLA) examination provides an objective, third party measure of an essential outcome indicator for our program. 2. It is low-hanging fruit because almost all of our graduates need a passing score for administrative position licensing. In Virginia the Qualifying score is 163, moderate for the nation. 3. The range of scores are between 120 and 200 points.

Utilizing the SLLA 2 4. It is offered multiple times throughout the calendar year (computer version). 5. The cost for taking the SLLA is $425.00. 6. The SLLA is easy for researchers to access the data (test scores). Examinees can designate 3 recipients per test without costs.

SLLA PASSING SCORES 156 U.S. Virgin Islands 160 Kentucky Tennessee 163 Arkansas District of Columbia Indiana Maine Maryland Michigan

Missouri New Jersey Pennsylvania Utah Virginia 165 Kansas 166 Louisiana 169 Mississippi

Education Testing Services (ETS) effective July 1, 2012: www.ets.org/Media/Tests/SLS/pdf/15884passingscores.pd

Value Added / Licensing-Plus The two are one and the same. In the Wallace Foundation study, When Learning Counts: Rethinking License for School Leaders (Adams & Copland, 2005, pp. 20-21), the term â&#x20AC;&#x153;Licensing-Plusâ&#x20AC;? includes the knowledge and skills that span organizational learning topics appropriate for an entry-level practitioner, the level a license represents. For school leaders the ISLLC standards and the SLLA represent a substantial and important step in licensure (p. 45). Adams, J.E., & Copland, M. A. (2005). When learning counts: Rethinking licenses for school leaders. Seattle, WA: Wallace Foundation.

For this same reason the SLLA is value added to our educational leadership preparation program outcomes and the Dependent Variable for our proposed study.

Design We will deploy a modified Solomon Four-Group Design for this investigation.

The Solomon Four-Group Design 1 1. This true experimental design addresses value-added and even approaches causal claims when properly deployed 2. It combines pretest-posttest and posttest-only control group designs allowing us to assess our program value added directly while comparing our students to students in other SOE programs (and to students from other Regent graduate programs with an additional group added to the original design) Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.

The Solomon Four-Group Design 2 1. The design protects against all threats to internal validity other than differential attrition (which we certainly might experience) 2. The design addresses the external validity threat of the interaction of testing and the treatment (our program) 3. Difference in randomization: we randomize selection and assignment for three streams of subjects rather than just one

The Solomon Four-Group Design 3 The Modified Solomon Four Group Design

R O1 R O3 R R R

X X

O2 O4 O5 O6 O7

R = Random selection/assignment O = Observations based on SLLA X = Treatment of Ed. Lead. program

Ed. Lead. Students Other SOE Ed. Lead. Students Other SOE Other Regent Some expected results: O2 > O1, O2 > O4, O5 > O6, and O5 > O7.

Analysis For our main analysis, we propose a 2 X 3 ANOVA to check for statistically significant differences and effect sizes among the five groupsâ&#x20AC;&#x2122; SLLA scores on the final (or only) administration of the examination for each group. Test condition has two possibilities: pretest-posttest or posttest only. Student group has three possibilities: Ed. Lead., Other SOE, and Other Regent students.

Basic Procedures Before students begin their programs, randomly select the Ed. Lead and Other SOE students for the initial round of SLLA examinations, invite, and sign-up for examination. Randomly select all other groups of students at this time. Invite and sign-up the other groups of students within 60 days of the time applicable SLLA administration. Over-sample all groups to allow for rejection and attrition. Pay ETS and students per SLLA administration schedule.

Steps for Gathering Value-Added Evidence SLLA data is reported to the university by test-takers coding the answer form ETS provides at the testing sites for the scores to come to the university. The university will receive a testing outcome report for each testing cycle and an annual report comparing university scores with state median and national median scores. The researchers are interested in funding from a faculty grant or other outside grant/funding possibilities (i.e. Wallace Foundation). The researchers are also interested in collaborating with ETS for funding or reduced or free testing for participants in the study. We believe ETS would be interested in the outcomes of this study and therefore agree to work with the researchers in this project.

Proposed Budget Test Fees:

70 X $425 = $29,750

Test Incentives: 70 X $100 = $7,000 Passing on first attempt (max.) 50 X $100 = $5,000 Increasing a passing score (max.) 20 X $100 = $2,000 Total $43,750

Related Issues 1 The Solomon Four Group design offers us the opportunity to exercise a preferable longitudinal approach as opposed to a cross-sectional design such as that employed by the VSA We can treat the data longitudinally because of the relatively short time frame (about one and a half years) for completion of the Masterâ&#x20AC;&#x2122;s degree in Educational Leadership. Likewise, our control/comparison groups will be drawn from masterâ&#x20AC;&#x2122;s degree programs with similarly brief scheduled time frames

Related Issues 2 Participant motivation is a special problem because more than half of the subjects in a balanced study would presumably not care about results on the SLLA examination. Our own students may not be particularly motivated when they take the pretest. Possible motivators include: 1.) SLLA success would open the door to educational administration for those who had not given it previous consideration; 2.) the examination is paid for; 3.) for the four hour exam, students would be paid $100; 4.) students who passed would be paid another $100; 5.) students who improve scores across two administrations would also receive an additional $100.

Related Issues 3 Employing motivation scales like the Student Opinion Survey (SOS) may be helpful in detecting low motivation students Students so identified can then have their actual performance taken into account as analysis proceeds Other possible motivators might include giving course credits within the SOE

Sundre, D. L, & Wise, S. L. (2003, April). Motivation filtering: An exploration of the impact of low examinee motivation on the psychometric quality of tests. Paper presented at the annual meeting of the National Council on Measurement and Education, Chicago, IL.

Related Issues 4 Low numbers of participants are expected due to the SLLA examination costs that we must incur in order to increase and maintain motivation. This involves a dance of tradeoffs among sufficient motivators, financing through grants, the extent of ETS collaboration, and n-size. We expect large effect size differences across the SLLA testing, but whether this can carry the day (for statistical significance and statistical conclusion validity) against a small n-size remains to be seen.

•