Institute Workshop Series

THE UNIVERSITY OF GEORGIA

U G A P R O F E S S I O N A L D E V E LO P M E N T

WORK SERIES SHOP

Presented by the Institute for Interdisciplinary Research in Education & Human Development

For more information about the Workshop Series, visit coe.uga.edu/research/the-institute/workshops/

Writing and Implementing Solid Evaluations Date: April 22, 2011 Registration Fee: $200 Presenter: Tom McKlin, Ph.D. President of The Findings Group, LLC Workshop Description: This workshop is designed to help those who pursue funding and those who evaluate those programs to build rigorous and winning evaluation plans. Creating a solid evaluation plan is the first step toward building evidence of effective practice in science, technology, engineering, and math (STEM) education.

Specifically, this program targets those who pursue STEM funding and will use example evaluation plans from National Science Foundation (NSF) programs. Further, investing in good evaluation planning enables a good overall proposal and helps guide and improve the day-to-day activities of the program. This workshop is suitable for those pursuing federal and state funding and for those who evaluate these programs.

Bring your proposal ideas and drafts and be prepared to: • Clearly and succinctly model your program • Generate evaluation questions • Define qualitative and quantitative methods for answering those questions • Anticipate analysis needs • Envision formative and summative reports Bio: Dr. McKlin serves as this year’s chair of the American Evaluation Association’s PreK12 Education Evaluation Topical Interest Group. He has extensive experience with federal and state grants, is a former Senior Research Scientist at Georgia Tech and former Director of Evaluation at Georgia’s Leadership Institute for School Improvement.

His company currently serves as the lead evaluation agency for projects at many of Georgia’s major colleges and universities, and Dr. McKlin consults with the CDC, the Computing Research Association, Georgia Bio, and the National Center for Women in Technology. He and his staff provide program modeling; needs analysis; development of program measures; extensive data collection services; an array of qualitative, quantitative, and mixed-methods analyses; and formative and summative reporting.

What in the World is Phenomenology?

Date: June 9-10, 2011 Registration Fee: $375 Presenter: Mark D. Vagle, Ph.D.

Workshop Description: In this exciting and engaging two-day workshop at the University of Georgia qualitative researchers, instructors of qualitative research methods, and graduate students will learn about the philosophical and methodological foundations of phenomenology and practice some of the core methodological techniques of phenomenological research.

In particular, participants will: • Explore the work of philosophers such Husserl, Heidegger, Merleau-Ponty, Sartre, and Gadamer • Map out the two main phenomenological research approaches in the human sciences—descriptive and interpretive • Discuss a new phenomenological research approach, post-intentional, developed by the presenter • Write, listen, see, and think phenomenologically by going on phenomenology walks, writing about lived experiences, and interviewing one another Bio: Mark D. Vagle is an award-winning instructor and a professor of education at The University of Georgia. He conducts, and teaches doctoral seminars focusing on, phenomenological research. Mark has presented and published numerous scholarly works based on his experiences as a phenomenological researcher at the annual meetings of organizations such as the American Educational Research Association and the Literacy Research Association and in journals such as the International Journal of Qualitative Studies in Education, Field Methods, Journal of Adolescent and Adult Literacy and Reflective Practice: International and Multidisciplinary Perspectives.

Unlocking the Black Box:

A practical guide to measuring the critical components of implementation fidelity

Date: June 17, 2011 Registration Fee: $275 Presenter: Jennifer Hamilton, Ph.D.

Workshop Description: It is one thing to employ a rigorous design in a research study, but it’s another thing to understand why impacts were or were not found. Because the intervention is the ‘cause’ in the ‘cause and effect’ relationship, measuring implementation fidelity is critical to unlocking the black box of inference. And although it is now seen as an important component of high quality evaluation studies, there exists little firm guidance on how implementation fidelity should be measured.

This workshop will review the concept of implementation fidelity, its increasing relevancy to the evaluation field, and will discuss the measurement of its 5 key components: 1. Adherence 2. Exposure 3. Quality of the delivery 4. Participant responsiveness 5. Program differentiation The workshop will explore how measurement can be tailored to the type of evaluation design and type of intervention being implemented.

Examples from multiple evaluation studies will be provided. The workshop will end with a review of some tools available for creating logic models, as well as for tracking and compiling multiple sources of data.

Bio: Jennifer Hamilton is a Westat Senior Study Director with 17 years of research experience in designing and conducting program evaluations in the field of K-12 education. Her clients have included urban school districts, the U.S. Department of Education, and city agencies, such as the New York City Mayor’s Office. Her recent work includes a focus on implementation fidelity, both at the national as well as the local grant level. This recent work includes measuring implementation fidelity in a national evaluation of a preschool literacy intervention, and in a district level grant that implemented a literacy intervention in the middle grades.

Introduction to Structural Equation Modeling Date: June 21-24, 2011 Registration Fee: $1200 Presenter: Deborah Bandalos, Ph.D.

Workshop Description: This workshop will provide a foundation in structural equation modeling by introducing participants to the basics of model estimation, evaluation of model fit, and model interpretation for basic structural equation models. The models to be covered included path models, confirmatory factor analysis models, and full structural models. Instruction will consist of lectures and hands-on activities. As part of the workshop, participants will be instructed in the use of the LISREL computer package. Afternoon activities will involve conducting analyses using the LISREL package. If time permits, participants will also be provided help in analyzing their own data.

Bio: Dr. Bandalos is a Professor and Director of the Assessment and Measurement Doctoral Program at James Madison University where she teaches courses in exploratory factor analysis, measurement theory, and missing data methodologies. Prior to accepting a position at James Madison University, Dr. Bandalos was a professor and director of the program in Research, Evaluation, Measurement, and Statistics at the University of Georgia and Assistant and Associate Professor in the Department of Educational Psychology at the University of Nebraska-Lincoln, serving as director of the program in Qualitative and Quantitative Methods in Education.

During her time at the University of Nebraska Dr. Bandalos was also affiliated with the Buros Center for Testing and the Survey Research and Methodology Program and was awarded the Gallup Research Professorship.

Validation: A General Framework & Prototypes Date: July 28-29, 2011 Registration Fee: $400 Presenter: Michael Kane, Ph.D. Workshop Description: In this seminar, Dr. Kane will introduce an argument-based framework for the validation of educational and psychological assessments, and will use the framework to analyze several types of interpretations/uses, including measures of observable attributes, traits, theoretical constructs, and decisions (e.g., placement decisions, certification decisions). The presentations will include discussions of the generalizability of scores and of a range of claims and decisions based on scores. The validation framework is quite general, but different interpretations and uses require different mixes of validity evidence, and a number of commonly occurring, or prototypical, cases will be considered.

Bio: Michael T. Kane, Ph.D., has held the Samuel J. Messick Chair in Validity at the Educational Testing Service in Princeton, New Jersey since September of 2009.

He served as Director of Research for the National Conference of Bar Examiners from September 2001 to August 2009, as a professor in the School of Education at the University of Wisconsin–Madison from 1991 to 2001, and as vice-president and senior research scientist at ACT from 1982 to 1991. He holds a B. A. in physics from Manhattan College, an M.A. in physics from SUNY, Stony Brook, and an M.S. in statistics and Ph.D. in education from Stanford University.

His research has focused on validity theory, the foundations of the theory of measurement, generalizability theory, licensure and certification testing, and standard setting.

WORKSHOP LOCATION & ACCOMMODATIONS

All workshops in the series are held at the Georgia Center for Continuing Education and Hotel which is located on the beautiful, historic campus of UGA in Athens, Georgia. The Georgia Center includes a 200-room hotel, four onsite dining options, banquet areas, conference rooms, auditoriums, a fitness center, and a computer lab — all under one roof.

RECENT WORKSHOP TOPICS DIAGNOSTIC CLASSIFICATION MODELS: THEORY, METHODS, AND APPLICATIONS Presented by Jonathan Templin, Ph.D. and Laine Bradshaw, Doctoral Candidate INTERVIEWING IN QUALITATIVE RESEARCH Presented by Kathryn Roulston, Ph.D. and Kathleen deMarrais, Ph.D. AN INTRODUCTION TO PROPENSITY SCORE MATCHING (PSM) Presented by Jay W. Rojewski, Ph.D. and postdoctoral fellows Sinan Gemici and In Heok Lee SINGLE SUBJECT RESEARCH METHODOLOGY IN BEHAVIORAL SCIENCES Presented by David L. Gast, Ph.D. and Kevin M. Ayres, Ph.D.

Presented by the Institute for Interdisciplinary Research in Education & Human Development For more Information about the Workshops Series, visit coe.uga.edu/research/the-institute/workshops/

The University of Georgia College of Education G-3 Aderhold Hall | Athens, GA 30602 706.542.6446 | www.coe.uga.edu