Educational Research Competencies for Analysis and Applications, 12th Edition GEOFFREY E MILLS Solut

Type:

Solution Manual

Educational Research Resource: Competencies for Analysis and Applications Edition:

12th Edition

Author(s):

Geoffrey E. Mills L. R. Gay

Instructor’s Resource Manual For

Educational Research: Competencies for Analysis and Applications Twelfth Edition Geoffrey Mills, Southern Oregon University Lorraine Gay, Late of Florida International University Prepared by: Adam Jordan, University of North Georgia

Table of Contents Preface

................................................................................................................................. iv

Chapter 1

Introduction to Educational Research ........................................................................1

Chapter 2

Ethics in Educational Research ..................................................................................6

Chapter 3

Selecting and Defining a Research Topic ..................................................................8

Chapter 4

Reviewing the Literature .........................................................................................13

Chapter 5

Preparing and Evaluating a Research Plan ..............................................................18

Chapter 6

Selecting a Sample...................................................................................................20

Chapter 7

Selecting Measurement Instruments ........................................................................24

Chapter 8

Survey Research ......................................................................................................27

Chapter 9

Correlational Research.............................................................................................29

Chapter 10 Causal-Comparative Research .................................................................................32 Chapter 11 Experimental Research ............................................................................................37 Chapter 12 Single-Subject Experimental Research ....................................................................44 Chapter 13 Narrative Research...................................................................................................48 Chapter 14 Ethnographic Research ............................................................................................51 Chapter 15 Case Study Research................................................................................................53 Chapter 16 Mixed Methods Research ........................................................................................55 Chapter 17 Action Research .......................................................................................................62 Chapter 18 Descriptive Statistics ...............................................................................................67 Chapter 19 Inferential Statistics .................................................................................................88 Chapter 20 Qualitative Data Collection…………………………………………………... …..97 Chapter 21 Qualitative Data Analysis and Interpretation.........................................................100 Chapter 22 Preparing a Research Report..................................................................................104 Chapter 23 Evaluating a Research Report ................................................................................106 Appendix

...............................................................................................................................108

References ...................................................................................................................................110 Handouts .....................................................................................................................................117

iii .

Preface Thank you for your supporting the 12th edition of Educational Research: Competencies for Analysis and Applications. Those instructors who have used previous editions of this text will find several helpful updates in this new edition. Perhaps the most notable addition is a new chapter, chapter 2, entitled: Ethics in Educational Research. This chapter was created in response to the helpful and constructive feedback of a number of colleagues and users of the text, and for that we say thank you. This chapter adds to the existing content on informed consent and protection from harm with a new and expanded section on action research and Institutional Review Boards (IRBs) that will be especially helpful for classroom-based, and school-based educational researchers. All chapters in the previous edition remain in the 12th edition and simply move up one chapter in the table of contents. Chapter 2 in the 11th edition is now chapter 3 in the 12th edition, and so on.

In addition to the new chapter on research ethics, other chapters also received significant revision. Chapter 4 has undergone significant revision because of the way technology has affected the literature review process. Chapter 7 has been significantly revised to provide an upto-date discussion of selecting measuring instruments especially as it relates to the use of personality and affective tests in schools. Chapter 12 on single-subject experimental research has been updated and expanded to include a classroom-based special education example that breathes life in to the research process for classroom teachers.

Another significant addition to this edition is the introduction of the open source statistics software package R in chapters 18 and 19. R represents an innovation in statistical software that has grown in popularity since the previous edition was published. Instructors who are unfamiliar with R will find that chapters 18 and 19 provide a basic and easy to follow guideline for obtaining and understanding R at an introductory level. Additionally, Appendix A contains R script that can be copied and pasted into the R browser so that users of the text do not need to write their own code if they do not feel comfortable doing so.

One of the most visible changes in the new edition, also one of the most significant, is the expansion of the digital learning and assessment resources embedded in the etext and the inclusion of MyLab in the text. MyLab for Education is an online homework, tutorial, and assessment program designed to work with the text to engage learners and to improve learning.

Within its structured environment, learners practice what they learn in the etext, test their understanding, and receive feedback to guide their learning and ensure their mastery of key learning outcomes. The MyLab portion of the new edition of Educational Research is designed to help learners (1) understand the basic vocabulary of educational research, (2) get hands-on experiences in reading and evaluating research articles, and (3) get guided practice in planning and developing a research proposal and in collecting and analyzing research data. The resources in MyLab for Education with Educational Research: Competencies for Analysis and Applications, 12th edition include: •

Self-Check assessments with feedback throughout the etext help readers assess how well they have mastered content.

•

Application Exercises—Thinking Like a Researcher allow the reader to practice specific tasks related to collecting and analyzing research data and also to practice planning and writing up a proposal. iv .

•

Application Exercises—Understanding Research provide readers with practice identifying key parts of research articles and using the concepts they learned in the chapter to read articles with understanding.

You will find that each chapter contains application exercises for each major learning objective, though not every learning objective, and self-checks for every learning objective.

We hope you enjoy this new edition!

v .

Chapter 1: Introduction to Educational Research Many of the learners in your class may be uncomfortable with the idea of interpreting or conducting research. In Chapter 1, the text addresses that some learners may be a little anxious about learning research methods. The text promotes a comfort level with the course content for students.

A variety of examples of research can also ease students’ initial anxiety. Generating examples in class and ‘thinking like a researcher’ will facilitate learners’ understanding. Therefore, the exercises below are designed to promote exposure to relevant research on topics that are familiar to the learners. The exercises also promote thinking like a researcher and the students will soon realize that they have the general knowledge to do research; they just need additional training and skills.

For those familiar with previous editions of the text, the ethics content formerly in Chapter 1 is now expanded in Chapter 2. Understanding ethics is critical for students’ understanding of research. Many students perceive they know ‘enough’ about research ethics but find, through experience find that there is more to understand. The ethics content always sparks a lot of interest and discussion and as such is a great way to start the course content, however, be aware that the material is now found in the second chapter of the text.

Activities embedded in the etext MyLab Education Application Exercise 1.1: Using the Scientific Method MyLab Education Application Exercise 1.2: Understanding Research: Recognizing the Characteristics of Quantitative and Qualitative Research MyLab Education Application Exercise 1.3: Reading Research: Identifying Steps in the Research Process Part 1 MyLab Education Application Exercise 1.4: Reading Research: Identifying Steps in the Research Process Part 2 MyLab Education Application Exercise 1.5: Explaining the Purpose of Different Research Methods MyLab Self-Checks for Each Learning Objective Additional Suggested Activities 1. Obtain evaluation reports from a local school or study of interest for your learners to read. If there are no local examples, consider the following examples, which are accessible through ERIC. The studies below are evaluation reports. These types of reports work well for introduction because they often include the steps the team or researcher used in the study in greater detail than is often found in published studies. This activity directly addresses the first outcome of Chapter 1 and illustrates the parts of the report to facilitate understanding of outcome.

If you do not have local examples, four examples of evaluation studies from diverse topic areas have broad appeal. The studies are:

1 .

Capizzano, J., Bischoff, K., Woodroffe, N., & Chaplin, D. (2007). Ingredients of a successful summer learning program: A case study of the building educated leaders for life (BELL) accelerated learning summer program. ED497332 Hartmann, T., Maluk, H. P., & Riffer, M. (2007). Teachers and students learning through service: A report on need in deed's developing work with teachers. ED499066 University of Southern California, Los Angeles. School of Education. (2007). Making the grade in college prep: A guide for improving college preparation programs. ED499275 Washington Univ., Seattle. Center on Reinventing Public Education. (2007). Inside charter schools: A systematic look at our nation's charter schools project update. ED495842 2. Also consider using brief research reports or shorter empirical articles in class. Divide the students into groups and give each group a research study. Three examples of varied types of studies that examine college students’ attendance follow. Have learners identify the component parts of the articles to support Tasks 1A, 1B, and 1C. Groups should identify each of the following. (a) Topic of the study and the problem investigated (b) Type of study (have them include type of approach and also whether qualitative or quantitative) (c) Procedures used to gather data (d) Method of data analysis (e) Major conclusion References of Examples to Use: Davidovitch, N., & Soen, D. (2006). Class attendance and students' evaluation of their college instructors. College Student Journal, 40(3), 691-703.

Debevec, K., Shih, M., & Kashyap, V. (2006). Learning strategies and performance in a technology integrated classroom. Journal of Research on Technology in Education, 38(3), 293-307.

Sommer, R., & Sommer, B. A. (2007). Credit for comments, comments for credit. Teaching of Psychology, 34(2), 104-106.

3. Task 1C asks learners to decide if topic statements represent different types of research. Provide examples in class to practice this skill. Some example items to include in your exercise are provided in Handout 2.1 at the end of this manual to be used as a handout or overhead.

For each topic statement, determine if the study is likely • • •

quantitative or qualitative basic or applied experimental, single-subject, correlational, causal-comparative, survey, narrative, ethnographic, case study, mixed methods, or action research 2 .

Here are the suggested responses to the topics presented in Handout 1.3: For Practice…

These answers are provided to guide discussion of these topics. As many topics can be approached from various angles, you or your students might ‘come up with’ additional ways to approach these broad topics. (a) A study that investigates what experts think about while solving problems.

This topic would likely be conducted as either as a qualitative study or as a quantitative survey study. This topic would also likely be an applied study because it would use existing theory to examine behaviors. (b) A comparison between achievement scores of learners with simulations embedded in their mathematics instructional materials and those without simulations in their materials.

This applied research topic would likely be conducted as either a causal-comparative or experimental quantitative study, depending upon whether it is ex post facto or if random selection and assignment are possible. (c) A study that examines one district’s response to the NCLB legislation.

This applied research topic could be addressed using a case study methodology. It is likely that qualitative data, such as observations, interviews, document analyses, would drive such a study. Quantitative test scores or change in scores might also be included in the case. (d) A study conducted by the high school art education teachers regarding the use of various instructional materials and media to facilitate student understanding of district-level objectives for art education.

This topic is likely an action research study, and would be an applied study that would include primarily qualitative methods although student understanding in this example might be assessed with quantitative measures. (e) A study that compares student grades in self-paced and instructor-taught college psychology courses.

This topic is a quantitative and applied study that represents causal-comparative research. (f) A study that determines the relationship between years of teaching and incidence of student disruptive behavior in the classroom.

This topic is most likely addressed as an applied, quantitative, correlational study. (g) A study that investigates what happens on a typical Saturday in our school’s detention program.

This topic is likely an applied, qualitative study. It might be approached using ethnographic methods but would most likely at least include observations and maybe interviews.

3 .

(h) A study that targets a reading-disabled student’s response to intervention.

This topic would most likely be conducted as an applied study. It might be approached either quantitatively or qualitatively, likely as a single-subject study or perhaps as a case study. Many of studies that address this and related topics are single-subject quantitative studies. (i) A study of 5 first-year students from rural communities on acclimation to a large university setting.

This topic would most likely be approached as an applied, primarily qualitative, likely narrative or ethnographic or study. Some mixed methods might be used. (j) A study that addresses how parents feel about the new district standards regarding calculators in math classes.

This topic is best addressed as an applied, quantitative, survey study. (k) A study that examines the relationship between in class attendance and course grades.

This topic is best addressed as an applied, quantitative, correlational study. (l) A study that addresses the experiences of ethnic minorities at primarily white universities during the 1950s and 1960s.

This topic is an applied, qualitative, narrative study. (m) A study to determine whether a summer reading program should be continued.

This topic could be approached from numerous ways. It is an applied study that might best be addressed from a mixed-methods approach. Qualitative data might include interviews and observations and quantitative data might include test scores or other measure of reading ability.

This study might be an evaluation study or an action study.

4. Consider the general topics such as school uniforms, year-round schooling, school size, school vouchers, transition to post-secondary education, laptop initiatives, cyberschooling, and extracurricular activities and academic achievement, and discuss how each could be a correlational study, a causal-comparative, or a survey study. Further discuss how each could be addressed by narrative, ethnography, case study, action research, or mixed methods research. Discuss the difficulties of true experimental studies in educational settings. Remember this exercise because you can use the same topics several more times in the class to have continuity.

Handout 1.4 provides a handout for this exercise using the topic of school uniforms. The following are some possible answers for this exercise given school uniforms as the research topic.

While some topics may be most often approached for study by a particular method or type of research some topics can easily be considered from many different types of research. One example of such a topic is school uniforms. Given the general topic of school uniforms, develop a research topic for each of the following approaches. a. Narrative: What is the experience of a class of students in a school that has just implemented a school uniform policy?

4 .

b. Ethnography: What are the perceived benefits and drawbacks of a school uniform policy according to parents and students? c.

Case Study: How do students and teachers react to the implementation of a school uniform policy?

d.

Single-Subject: Are there changes in the behavior problems of individual students with emotional disorders after the implementation of a school uniform policy?

e. Survey: How do teachers, parents, students, and community members feel about the implementation of a school uniform policy? f.

Correlational: Across schools with uniform policies in the state, what is the relationship between school violence and attendance? (Note that this topic would more likely be addressed in a causalcomparative study than correlational. Explain why to your students)

g. Action research: How should we decrease the wearing of gang colors and inappropriate clothing to school? h. Causal-comparative: Across the state, is there a difference in standardized test scores between low SES schools that have a school uniform policy and those who do not? i.

Mixed methods: In a class of students in a school that has just implemented a school uniform policy, how do the students feel about the change based on a survey and what do they say about the experience in their own words?

5 .

Chapter 2: Ethics in Educational Research As stated earlier, Chapter 2 is all new for the 12th edition and is dedicated solely to the discussion of ethics in educational research. This topic has always been of the utmost of importance, but as the field of educational research continues to expand and grow and as the world becomes more interconnected, discussing ethics with budding educational researchers is an extremely important endeavor.

The intent of this chapter is for learners to receive an overview of basic ethical principles in educational research and also to begin to think about the Institutional Review Board’s purpose and function. For this reason, the activities below are focused on helping students think through ethics in educational research.

Activities embedded in the etext MyLab Education Application Exercise 2.1: Thinking Like a Researcher: Understanding Research Ethics MyLab Education Application Exercise 2.2: Reflecting on Ethics in Educational Research MyLab Education Application Exercise 2.3: Obtaining IRB Approval for Action Researchers MyLab Education Application Exercise 2.4: Understanding Considerations in Accessing Research Sites MyLab Self-Checks for Each Learning Objective Additional Suggested Activities 1. Ivor A. Pritchard published an essay regarding research ethics and human subjects’ review. It is a nicely done piece that traces some history, introduces the IRB, and discusses the role of IRBs for practitioner researchers. It is a helpful article if there are many practitioner- researchers in the class that you teach. Have students access it from the library, read it, and bring it to class as part of an assignment or scavenger hunt. Discuss the main points in class. Handout 2.1 at the end of this manual can be used as guided notes for students to complete prior to class or as an overhead to guide discussion.

•

Pritchard, I. A. (2002). Travelers and trolls: Practitioner research and institutional review boards. Educational Researcher, 31(3), 3-13.

2. The websites by Ronald B. Standler documents select unauthorized medical experiments postNuremberg. This site documents several additional medical experiments other than the Tuskegee and Harold Blauer. In addition, showing the Milgram video in class is useful to demonstrate unethical treatment and generate a discussion of the importance of research ethics.

•

Standler, R. B. (1997). Nonconsensual medical experiments on human beings. Retrieved from rbs2.com/humres.htm

6 .

•

Milgram reference: Milgram, S. (1969). Obedience. (Videotape of the classic research on obedience to authority, based on footage shot at Yale University.) Available for purchase from Penn State Media Sales at MediaSales@psu.edu.

•

Milgram’s biographer, Thomas Blass, maintains a Milgram Website (stanleymilgram.com)

3. Of particular interest to some students are the differences among countries’ expectations for ethics and standards for studies involving human subjects. Academic institutions require students to obtain human subjects at their home (degree granting) institution as well as site level institution if they are conducting their studies in other countries. Discussing these apparent differences always provokes student participation and can provide an opportunity for students’ to reflect on our practices both at the institutional level and the national level. If you lack international diversity in your class, one means to introduce differences is to consider requirements for numerous institutions. The following website provides requirements and forms for the University of Glasgow: gla.ac.uk/services/dpfoioffice/ Students can access this site and make direct comparisons with your own institution. The Glasgow site provides very detailed information about their policies on student records that is interesting for conversation in the class.

4. Perhaps one of the most beneficial activities for learners would be to introduce them to the IRB process at your institution. Spend some time helping them understand the website and forms associated with your institution. Students are usually surprised to learn that IRB applications are often extensive and time consuming. If possible, it may be beneficial to invite the chair of the IRB at your institution to come and speak to your students. This may also be true for school level IRB administrators. This role will vary by district, but it is important for students to understand that in educational research, approval is often needed from multiple stakeholders and planning, therefore, must take place far in advance of the actual timeline for study implementation.

5. Once students have become familiar with the content of the chapter and perhaps explored their institution’s IRB website, have students produce draft IRB applications using the forms provided by your institution. Then, divide your class into groups of 4-5 and let each group serve as an impromptu IRB review board. Students can share their applications and receive peer feedback. Keep in mind that the instructor will need to facilitate this activity, as students will likely be unfamiliar with details of the IRB process, but they will gain insight into the complexity of an IRB. 6. It is a good idea to have your students go ahead and complete the CITI training required by most institutions. CITI stands for Collaborative Institutional Training Initiative. Students will need to go to the CITI website (https://about.citiprogram.org/en/homepage/) and obtain an account, but after logging in they can access the Social and Behavioral Research- Basic/Refresher course. Students will find that chapter 2 is quite helpful to successfully completing the CITI training.

7 .

Chapter 3: Selecting and Defining a Research Topic Activities embedded in the etext MyLab Application Exercise 3.1: Understanding Research: Identifying Problems in Research Reports Part 1 MyLab Application Exercise 3.2: Understanding Research: Identifying Problems in Research Reports Part 2 MyLab Application Exercise 3.3: Reading Research: Identifying Hypotheses MyLab Application Exercise 3.4: Thinking Like a Researcher: Developing a Hypothesis MyLab Self-Checks for Each Learning Objective Suggested Activities 1. Even after experience with Chapter 3, many students still struggle with topics and types of hypotheses. Providing general research examples and asking students to generate a research problem, as well as accompanying null, nondirectional, and directional hypothesis may facilitate their understanding. For this activity, use handout 3.1 as noted below and in the Appendix. You can use these as guided practice, or you can assign these for independent practice and follow up in class with the answers. Some students may have difficulty with generating hypothesis for non-experimental studies.

Handout 3.1 For each of the following general questions identify a research problem, and write a null, nondirectional, and directional hypothesis. a. How does increased standardized testing relate to student motivation?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: b. Are there gender differences in students’ perceptions of Inquiry-based instruction?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: 8 .

c. Does reading-disabled students’ comprehension benefit from guided notes?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: d. Do puzzle experiences in the primary grades matter in students’ later science interest?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: e. Are there differences in the number of discipline referrals per student in large versus small schools?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: Student answers will vary, however, example answers for Exercise 3.1 follow.

Handout 3.1 For each of the following general questions generate a research topic, and write a null, nondirectional, and directional hypothesis. a. How does increased standardized testing relate to student motivation?

Research Problem: This study examines the relationship between the number of standardized tests given and student motivation as defined by mastery orientation.

Null Hypothesis: There is no relationship between the number of standardized tests given to students and students’ mastery orientation.

Nondirectional hypothesis: There is a relationship between the number of standardized tests given to students and students’ mastery orientation.

Directional Hypothesis: As the number of standardized tests given to students increases, their mastery orientation decreases.

9 .

b. Are there gender differences in students’ perceptions of Inquiry-based instruction?

Research Problem: The purpose of this study is to investigate gender differences in students’ self-reported opinions of Inquiry Physics lessons.

Null Hypothesis: There are no gender differences in students’ self-reported opinions of Inquiry Physics lessons.

Nondirectional Hypothesis: There are gender differences in students’ self-reported opinions of Inquiry Physics lessons.

Directional Hypothesis: Girls’ self-reported opinions of Inquiry Physics lessons are higher than are boys’ opinions. c. Does reading-disabled students’ comprehension benefit from guided notes?

Research Problem: This study investigates differences in reading-disabled fifth grade students’ content area comprehension when they are provided guided notes compared to when they are not provided guided notes.

Null Hypothesis: There are no differences in fifth grade reading-disabled students’ comprehension when they are provided guided notes as compared to when they are not provided guided notes.

Nondirectional Hypothesis: There are differences in fifth grade reading-disabled students’ comprehension when they are and are not provided guided notes.

Directional Hypothesis: Fifth grade reading-disabled students comprehension is higher when they are provided guided notes versus when they are not provided guided notes. d. Do puzzle experiences in the primary grades matter in students’ later science interest?

Research Problem: The goal of this research is to examine structured puzzle experiences in first grade and students’ reported science interest in fourth grade.

Null Hypothesis: There is no relationship between students’ puzzle experiences in first grade and their reported science interest in fourth grade.

Nondirectional Hypothesis: There is a relationship between students’ puzzle experiences in first grade and their reported science interest in fourth grade.

Directional Hypothesis: Students’ with structured puzzle experiences in first grade report high science interest in fourth grade. e. Are there differences in the number of discipline referrals per student in large versus small schools?

Research Problem: This study examines differences in the number of per-student discipline referrals in large (greater than 500 students per grade) and small (less than 200 students per grade) schools.

10 .

Null Hypothesis: There are no differences in the number of per-student discipline referrals between large and small schools.

Nondirectional Hypothesis: There are differences in the number of per-student discipline referrals between large and small schools.

Directional Hypothesis: The number of per-student discipline referrals is greater in small schools.

2. Develop a discussion board for the course as a place for learners to post their questions for clarification or for feedback from their peers.

Most institutions now have course management systems that allow instructors and students to post to discussion boards for their classes. I have started to use this function more in my classes, and students enjoy being able to seek assistance from peers with even simple questions or points of discussion. Edutopia provides resources that can assist faculty in incorporating discussion boards into their instruction. Visit edutopia.org and search for discussion board. The site provides suggestions for best practices, strategies, and management tips.

3. Have students practice identifying the structure of experimental hypotheses. Use the following three examples as class practice. The exercise may occur in class or through an online discussion board.

For each of the following experimental study research statements identify P, X, and Y. Where P = the participants, X = the treatment or independent variable, and Y = the dependent variable. a. The purpose of this study is to investigate the effects of silent reading time on students’ independent reading comprehension as measured by standardized achievement tests. b. The purpose of this research study is to investigate if colored font facilitates intermediate grade students’ learning from educational games. c. The purpose of this investigation is to determine if there is an effect of students’ participation in an anti-drug training program on the number of students’ in school drug arrests

11 .

Answers: For each of the following experimental study research statements identify P, X, and Y. Where P=the participants, X= the treatment or independent variable, and Y=the dependent variable. a. The purpose of this study is to investigate the effects of silent reading time on students’ independent reading comprehension as measured by standardized achievement tests.

P=Students X=Silent reading time Y=Standardized achievement tests b. The purpose of this research study is to investigate if colored font facilitates intermediate grade students’ learning from educational games.

P=Students X=Colored font Y=Learning from educational games c. The purpose of this investigation is to determine if there is an effect of students’ participation in an anti-drug training program on the number of students’ in school drug arrests.

P=Students X=Anti-drug training program Y=School drug arrests

12 .

Chapter 4: Reviewing the Literature Activities embedded in the etext MyLab Education Application Exercise 4.1: Explaining the function of the Literature Review MyLab Education Application Exercise 4.2: Thinking Like a Researcher: Conducting Literature Reviews in Qualitative Research MyLab Education Application Exercise 4.3: Thinking Like a Researcher: Identifying Primary and Secondary Sources MyLab Education Application Exercise 4.4: Thinking Like a Researcher: Narrowing a Keyword Search MyLab Education Application Exercise 4.5: Writing a Literature Review MyLab Education Application Exercise 4.6: Conducting a Meta-Analysis MyLab Self-Checks for Each Learning Objective Suggested Activities 1. Several places on the web have posted lists of education and other journals. The Distance Education Clearinghouse hosted by the University of Wisconsin-Extension, for example, maintains a list of distance learning journals and magazines at depd.wisc.edu/html/mags3.htm.

Have students access this list and published lists from the library as an out of class assignment and bring them to class for discussion.

2. As the text indicates to the learners, they need to know their library. Three strategies may help students meet the outcomes for this chapter. First, many librarians are very willing to provide specialized services for classes. Consult with your library to check if you can arrange for a specialized class that meets the needs of your course objectives. Content of this specialized class might be a review of electronic databases and how to access them at the library, how to obtain interlibrary loan items, and recalling items. Second, several academic libraries now have virtual tours.

This can be assigned outside of class or students can catch up with a general tour. The only drawback is that such tours are generally not as tailored as a separate class tour. A third option is a ‘scavenger hunt’ assignment for the class. Consider having learners do this as a take-home assignment either alone or in pairs. Some examples of ‘hunt’ items might include: •

Where are the Buros Center for Testing Mental Measurement Yearbooks housed or accessed?

Bring all the reviews for one of the tests reviewed in the nineteenth edition to class. As our library has both electronic and hard copy volumes you can bring either format.

•

Conduct an electronic search of the author Robert Sternberg. Conduct this search in an education database, a psychology database, in a multisearch electronic database, and your library’s catalog.

Describe any differences in citations provided between the education and psychology databases.

What is the title of one of his books? Where is it located in the library? Is it currently available?

13 .

•

Find the most recent copy of the Journal of Experimental Education and review the ‘notes to authors’ information. Summarize the process for submitting a manuscript to this journal. Note one thing about the submission process to this journal that you did not know before you accessed the notes to authors’ information.

•

Conduct a research database search on test anxiety through your library. Conduct a Google search on test anxiety, and conduct a Google Scholar search on test anxiety. Compare the results.

Bring one empirical article to class.

3. As an opportunity to discuss sources, provide a topic to the class and assign students to conduct various searches on the topic. For example, have students search in Google, Google Scholar, ERIC, and the library databases. (If you used the scavenger hunt activity above you can use the results of the last item as a starting point for this activity.) Ask students to consider the results of their searches.

•

Which of these searches yielded peer-reviewed articles?

•

What information did you find that was not peer-reviewed articles that was helpful to pursue for your topic of interest?

•

What were the benefits and drawbacks of using each of the searches for the purposes of your topic of interest?

•

What information was NOT helpful to you as you consider your topic of interest? From which search was that information accessed?

Examples might include, study strategies, test anxiety, attention deficit disorder, academic motivation, effects of diet on training, principals’ leadership styles. These work well as examples because each will yield a variety of source materials for discussion.

•

Which of these searches yielded peer-reviewed articles?

All will yield at least some peer-reviewed articles and some that are not peer-reviewed. Take time to discuss sources and the difference between articles published online in scholarly and other outlets. This is also a time to discuss ED designated articles.

•

What information did you find that was not peer-reviewed articles that was helpful to pursue for your topic of interest?

Of course this will vary but the main objective is that they also find instrumentation, and other practice based sources that might be helpful (e.g., IES publications).

•

What were the benefits and drawbacks of using each of the searches for the purposes of your topic of interest?

It is important to share materials that are clearly not objective to demonstrate the need to take care in source identification.

•

What information was NOT helpful to you as you consider your topic of interest? From which search was that information accessed?

14 .

Advertisements are one great example as are commercially available programs.

4. Use the following references on the topic of homework to discuss and review the types of published articles. Place students in small groups and let them work through the questions and then discuss the articles as a group.

Chang, M., Singh, K., & Mo, Y. (2007). Science engagement and science achievement: Longitudinal models using NELS data. Educational Research and Evaluation, 13(4), 349-371.

Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework improve academic achievement?

A synthesis of research, 1987-2003. Review of Educational Research, 76(1), 1-62.

Heitzmann, R. (2007). Target homework to maximize learning. Education Digest: Essential Readings Condensed for Quick Review, 72(7), 40-43.

Jackson, B. (2007). Homework inoculation and the limits of research. Phi Delta Kappan, 89(1), 5559.

Skaggs, A. N. (2007). Homework: A nightly ritual beginning in the elementary grades. Online Submission; 52p. ED497419 Trautwein, U. (2007). The homework-achievement relation reconsidered: Differentiating homework time, homework frequency, and homework effort. Learning and Instruction, 17(3), 372-388.

Handout 4.4 1. Provide two keywords that would be representative for this article.

2. Is this a primary source or a secondary source? How do you know?

3. Which database might be appropriate to use to find this article?

4. If your article represents a study, please answer the following. a. What was the problem statement of the study?

b. Who was studied?

c. How was the study conducted?

15 .

d. Is this a quantitative or qualitative study?

5. If your article does NOT present a study, please answer the following. a. Is this a literature review? How do you know?

b. Is this an essay or a position piece? If so, is this research piece objective? How does the author present the critical issue presented.

6. Suggest one purpose for which you might use this article.

Handout 4.4 (Selected answers for discussion) 1. Provide two keywords that would be representative for this article.

For most of these articles Homework would be a representative keyword. For the Chang and others’ piece science engagement or science learning or science achievement would also be a keyword. For the rest of the articles academic achievement might be a second keyword.

2. Is this a primary source or a secondary source? How do you know?

Chang and others is primary. Cooper and others is secondary. Heitzman and Jackson are interesting sources and discussion regarding whether they are primary or secondary will ensue.

The Skaggs piece is a secondary source that is a synthesis of existing research. Students will include the justification of data collected as required for a primary source. Trautwein and others is primary.

3. Which database might be appropriate to use to find this article?

These articles are all indexed in ERIC. Some would also have been found in Psychology or Sociology indices. A multi-search would have yielded these. Google Scholar also yielded the articles with varying degrees of priority.

4. If your article represents a study, please answer the following. a. What was the problem statement of the study? b. Who was studied? c. How was the study conducted? d. Is this a quantitative or qualitative study?

One might argue problem statements for the review pieces as well. Of interest in the Chang, Singh, and Mo (2007) is the use of an existing data set. The research studies are quantitative.

16 .

5. If your article does NOT present a study, please answer the following. a. Is this a literature review? How do you know?

Of the articles, the literature reviews (e.g., Cooper and others and Skaggs) are clear as they don’t provide primary source data. The journal for the Cooper piece also is a clear hint. b. Is this an essay or a position piece? If so, is this research piece objective? How does the author present the critical issue presented.

All of the articles articulate a clear easily identifiable objective. Whether the Heitzmann or Jackson pieces are objective can be discussed.

6. Suggest one purpose for which you might use this article.

All of the articles could be used in a paper about the effects of homework. Some might be shared with other teachers or with parents. One might use these to inform decision- making and policy related to homework practices.

17 .

Chapter 5: Preparing and Evaluating a Research Plan Activities embedded in the etext MyLab Education Application Exercise 5.1: Thinking Like a Researcher: Preparing a Quantitative Research Plan MyLab Education Application Exercise 5.2: Thinking Like a Researcher: Preparing a Qualitative Research Plan MyLab Education Application Exercise 5.3: Thinking Like a Researcher: Evaluating a Research Plan MyLab Education Application Exercise 5.4: Reviewing and Revising a Research Proposal MyLab Self-Checks for Each Learning Objective Suggested Activities

1. During discussion of this content it is a good idea to share several examples of research proposals or plans--some for research, some for funding agencies, others for theses or dissertations, and some for requirements for your course. Explain that expectations vary across institutions and audience.

Providing a range of examples gives the students a much better idea about how to prepare a research plan commensurate with one that they might be expected to complete for a course, a degree, or for funding. These examples should include both quantitative and qualitative research plans and proposals. Search the following sites for proposal guidelines. a. AERA’s dissertation grant program call for proposals: aera.net/ b. Your institution’s regulations for thesis and dissertation proposals. c. The guidelines for federal funding from the Institute of Educational Sciences, homepage: ed.gov/about/offices/list/ies/index.html d. HUD’s doctoral dissertation research grant program call for proposals: portal.hud.gov e. Several special interest groups (SIGs) and divisions of AERA and APA have student mentoring programs that have annual calls for proposals. Check those most directly related to your students’ interests.

2. Many of the proposal examples available are quantitative in nature. To supplement the text content you might consider other proposal guidelines for qualitative research. One easily accessible examples appears below. Consider assigning each as outside reading and asking students to identify the similarities and differences among these and the text. Students can also discuss whether qualitative scholarship has changed since the publication of these recommendations when compared to the text presentation.

Heath, A.W. (1997). The proposal in qualitative research. The Qualitative Report, 3(1). Retrieved from nova.edu/ssss/QR/QR3-1/heath.html 3. As part of preparing a research plan and in preparation for Task 3, share examples of students’ previous research proposals. First, obtain permission from my students to save electronic versions of their proposals to share with future students. While a few students would rather not share their ‘less 18 .

than perfect’ proposals; many students are willing to share their proposals as benchmarks in their development as researchers. Students find these proposals to be good models for their own course work.

19 .

Chapter 6: Selecting a Sample Activities embedded in the etext MyLab Application Exercise 6.1: Reading Research: Sampling in a Quantitative Study MyLab Application Exercise 6.2: Reading Research: Sampling in a Qualitative Study MyLab Application Exercise 6.3: Reading Research: Sampling in a Qualitative Study MyLab Application Exercise 6.4: Reading Research: Selecting a Qualitative Sampling Technique MyLab Self-Checks for Each Learning Objective Suggested Activities 1. Although sampling is appropriate at many points of the course, now is a reasonable time to introduce a priori power analysis. Students were introduced to effect sizes through discussion of meta-analysis in Chapter 3, which can serve as a foundation for discussion of how we might determine how large of a sample we will need. G*Power is a free and useful power analysis tool available at gpower.hhu.de/en.html This software is available for either Mac OS or Microsoft Windows. This tool can be used as an additional component to activities in this chapter. Begin by suggesting various effect sizes and posing ‘differences’ research questions. Pick one example of a large effect size (e.g., .8) and one much more common in educational research, so students realize the role effect size plays in power analysis and sampling determinations.

2. To facilitate understanding of quantitative sampling, provide examples that are very general and that can be sampled in most ways. The following example can be used to illustrate quantitative sampling.

Some suggestions are included.

You are interested in studying school safety issues. Do parents, teachers, administrators, and students feel their schools are safe?

Discuss a hypothetical list of about 100 teachers in a school as the population. Include in the list some additional variables such as gender, years teaching, or grade level if you use teachers.

Use the example to modeled all quantitative sampling techniques: simple random sampling, stratified sampling, cluster sampling, and systematic sampling. Then, have the students work in small groups to walk through sampling as if they were going to sample one of the other population groups (i.e., parents, administrators, or students).

3. As a small group activity, have students consider conducting a study that includes various sampling decisions in qualitative or quantitative research. The following example can be used for this exercise.

You are interested in studying school absenteeism. How do students, teachers, administrators, and social service personnel define absenteeism and what are the perceived results of absenteeism from each perspective?

20 .

This example will require a number of operational definitions (e.g., what defines absenteeism).

Students will need to determine how they will approach the study, quantitatively or qualitatively, which will change the nature of the sampling for the study. For a quantitative study, students will need to determine how to address sampling decisions regarding representativeness from constituents such as teachers (e.g., grade level, content area representation), administrators (e.g., school or district level), social service personnel (e.g., from the district or the courts). In a qualitative study, it is important to consider whether the sample will permit sufficient depth to understand absenteeism.

When starting this exercise, post the question and allow students to work in pairs or small groups and encourage them to use their text. Then ask them to work together to figure out how they might conduct a study to address this general research problem. Finally, move around to the groups and start to provide prompts such as: • What is your research question(s)?

• What type of methodology do you intend to use?

• Are you going to sample all representative groups or select one or two?

• How many participants will you need to sample?

• How will you define your population(s)?

• What sampling strategy will you employ? Why? How? Of course other questions will also guide students’ thinking about sampling. One benefit of this exercise is that each group will define the populations and the construct of interest differently. Further, wide variance in how the problem is tackled generally occurs such that students can see the importance of planning and the role sampling plays in conducting research. This activity can take considerable class time. If your class is accustomed to working in groups and have already read Chapter 6, and you restrict the activity [e.g., What type of methodology will you use? How will you define your population(s)? What sampling strategy will you use? Walk through the steps to sampling using your selected strategy]. A truncated version of this activity can be completed in 20 minutes. Otherwise, allow approximately 45 minutes for the activity and follow-up discussion.

4. A discussion about generalizability of sample is good to include with Chapter 6 content. One sample article to consider for use is Harris & Brooks (1998). Although the study is a dated it is useful, because many students taking research methods can easily relate to the study and the study is not heavily laden with any domain-specific content knowledge requirements. The article works well to discuss generalizability of sample. It is also a good idea to use the published instrument in this study as an example with Chapter 7 content so it is a nice transitional activity. It will help the class discussion on sampling if students have read the article prior to class but it is not necessary to have them do so.

•

Harris, M. B., & Brooks, L. J. (1998). Challenges for older students in higher education. Journal of Research and Development in Education, 31, 226-235.

In the Harris and Brooks (1998) study, the researchers sampled ‘older students’. Sometimes we might refer to such learners as nontraditional learners. In their study, these researchers collected data from students in a couple different classes. As an in-class exercise, consider the general research question: What are the challenges faced by nontraditional undergraduate students? How might one conduct such a study on your campus? Walk the students through the following questions. This works well in small groups or pairs and then to discuss and share as a large group.

21 .

1.

2.

3.

4.

5.

What is the target population of our study?

What is our accessible or available population?

What challenges might we face if we tried to employ a random sampling technique?

How would we conduct this study using systematic sampling?

If we are considering nontraditional students at our institution to be our population, and there are 28,000 undergraduate students at our school, of which 20% are nontraditional, what is the sample size we need to conduct the study?

6. If you were interested in conducting this study as a qualitative study, how might you use snowball sampling to obtain your sample?

Sample Answers for Activity 4 1. This answer will vary. All nontraditional undergraduate students in the country is a common response. However, your students might want to consider only those on the East Coast, or only those at 4-year institutions, or whatever.

2. The accessible population would vary as well. One typical population would be the nontraditional students at your institution. Some might want to do State, or Region, or System, etc.

3. Random sampling could be used in this study. Among others, challenges in using random sampling would include that the sample is quite large, it might be challenging to obtain all the possible members of the population, and part time students might be difficult to identify and contact.

4. Systematic sampling might work well for this study. If we could access a list of all nontraditional students (perhaps from the registrar) we could contact every nth name from the list.

5. 5600 is about 20% of 28,000. We will need about 360 participants to have a representative sample. Of course, given we might mail or email the survey we are using, we would need to account for those who would not reply so we will need to contact many more than 360 students.

6. To use snowball sampling, one might hang out at the student union, identify and approach a potential participant and have participants identify others who might fit the study criteria. This might work well as nontraditional students often have established both formal and informal networks.

5. To illustrate qualitative sampling, use the scenario from activity 1: You are interested in studying school safety issues. What are parents, teachers, administrators, and students experiences with school safety?

Discuss the hypothetical list of about 100 teachers in a school as the population. Include in the list some additional variables such as gender, years teaching, or grade level if you use teachers.

Use the example to modeled all qualitative sampling approaches: intensity sampling, homogenous sampling, criterion sampling, snowball sampling, and random purposive sampling. Then, have the students work in small groups to walk through sampling as if they were going to sample one of the other population groups (i.e., parents, administrators, or students).

22 .

Chapter 7: Selecting Measuring Instruments Activities embedded in the etext MyLab Education Application Exercise 7.1: Distinguishing Between Constructs and Variables MyLab Education Application Exercise 7.2: Thinking Like a Researcher: Identifying Measurement Scales MyLab Education Application Exercise 7.3: Understanding Research: Evaluating the Measuring Instruments Used in a Quantitative Study MyLab Education Application Exercise 7.4: Thinking Like a Researcher: Searching for Instruments MyLab Education Application Exercise 7.5: Exploring Validity and Reliability MyLab Education Self-Checks for Each Learning Objective Suggested Activities Chapter 7 has a large amount of very diverse content and will likely take more class time relative to other chapters. In addition to quantity of content, there are several topics, such as variables, included in Chapter 7 that students in introductory research courses find difficult.

1. The text discusses different types of measurement instruments. Share examples of sample items from standardized achievement tests, aptitude tests, personality measures, and attitude scales to support text content. Consider the following sources for each. Comparing the purposes of the instruments helps students to see the differences between aptitude and achievement and between personality and attitude, both of which are sometimes initially confusing to students in my class. The links below provide publishers sites.

Test/Type Stanford-Binet/Aptitude

Citation from publisher riversidepublishing.com

Otis-Lennon/Aptitude

pearsonassessments.com

Stanford 9/Achievement

pearsonassessments.com

Iowa Test of Basic Skills/Achievement

riversidepublishing.com

Myers Briggs/Personality

myersbriggs.org

The Big Five/Personality

outofservice.com/bigfive/

2. Handout 7.2 (located in the Appendix) contains four examples to use as a handout. It lists some potential research topics to use for guided practice with the students in identifying independent and dependent variables and to practice operationalizing variables. Have students do the exercise in pairs, and then discuss it as a group. The class discussion will also provide an opportunity to discuss mediating and moderating variables. An alternative is to have each pair pick two and then discuss all of the examples as a group assuming not all pairs will select the same two examples.

23 .

Answers to Handout 7.2: For each research topic identify the dependent and independent variable. Develop the topic by operationalizing variables.

Are there differences between rural and urban children’s attitudes regarding diversity?

The dependent variable is attitudes regarding diversity.

The independent variable is type of children and it has two levels, rural and urban.

In this example several variables need to be operationalized. Let’s start with the dependent variable.

Attitudes regarding diversity could be racial or ethnic diversity, it could be diversity related to being from elsewhere, or living differently. To develop this topic diversity would have to be operationally defined.

In this example we also need to operationalize rural and urban. This might be done with population statistics.

Children would also need to be operationalized. A small child’s attitude regarding diversity versus a middle-school child’s, or a teenager’s might be very different constructs.

Is there a relationship between years of post-secondary schooling and social competence?

This is an example of a correlational study and when we discuss variables in correlational studies we usually refer to them as criterion and predictor variables. In this case the criterion variable (more akin to dependent—what is measured) is social competence. The predictor variable is post-secondary schooling.

To conduct this study we would need to operationalize post-secondary schooling. Does business school count? Beauty school? Culinary? Other trade schools? Do the years have to be full time schooling?

We also would have to operationalize social competence. This might be facilitated by the instrument used. Or perhaps scenarios could be provided and the participant could state appropriate social behaviors given the scenario.

How will learners enrolled in an intensive mathematics program over the summer achieve in math compared to those who are not enrolled in the program?

This is a causal-comparative study and the dependent variable is math achievement. The independent variable is enrollment in the intensive summer program (enrolled, not enrolled).

Math achievement might be operationalized as scores in class, actual skill test outcomes, or standardized test scores, for example.

To conduct this study it is likely that the independent variable has already been operationalized because it is the program. However, don’t forget to consider that attendance might be down over the summer and perhaps students are enrolled and don’t really come. One could operationalize enrollment as enrolled and attended and active 80% of the time, for example to take a precaution against that.

24 .

What are the academic variables that account for a successful collegiate experiences?

This might be a descriptive study and we would not identify dependent and independent variables, as such but as variables of interest instead. This study might be conducted as a correlational study with a predictive or path model. Where successful collegiate experience would be the criterion and academic variables would be the predictors.

To operationalize academic variables one might consider GPA, HS rank, HS GPA, SAT scores, honors achieved, number of AP course credits, number of courses registered and completed.

To operationalize successful collegiate experience, one might use social predictors, or self-report measures of adjustment. One might also consider using persistence.

3. Recently the Standards for Educational and Psychological Testing also known by some as the joint standards because the publication in a collaborative effort of APA, AERA, and NCME has been under revision. The last version of the standards was published in 2014. Students should read the standards whether they intend to be practitioners or researchers. The content in the standards is helpful to support Chapter 6 content.

4. To support Task 5 to select three instruments appropriate for collecting data pertinent to the student’s study, have students complete reviews of instruments and work through them in class. Bring in copies of an instrument that has also been reviewed in the Mental Measurements Yearbook (MMY), along with the MMY reviews. This exercise works best in small groups. Compare your class reviews with the reviewers’ entries in MMY. As students may lack enough background knowledge to conduct an adequate review, the Rudner (1994) article on evaluating tests is a helpful reference.

Rudner, L. M. (1994). Questions to ask when evaluating tests. Practical Assessment, Research, & Evaluation, 4(2). Retrieved from PAREonline.net/getvn.asp?v=4&n=2 5. APA has a special website dedicated to testing information. This site is an EXCELLENT resource.

A good class activity is to have students review sections of the site in small groups and share what they have found. Sections of the site and related links focus on test use and standardized testing.

Both of these topics are always interesting topics for class discussion. The site is available at apa.org/science/programs/testing/index.aspx 6. One additional resource that contains peer-reviewed articles on item construction and related topics is the electronic journal Practical Assessment, Research, and Evaluation (pareonline.net). This journal includes articles that address important assessment and evaluation topics. The primary audience for the journal is practitioners.

7. Lead a class discussion on operationalizing variables. Murphy and Alexander (2000) reviewed literature related to the construct of motivation overall and several sub-constructs. Many prominent researchers’ work is presented in the article and displayed as systematically not clearly operationalized. While motivation is clearly a construct that needs to be operationalized, using an example in class like student achievement or learning can also illustrate differences in the way that we will define constructs and subsequent interpretation.

Murphy, P. K., & Alexander, P. A. (2000). A motivated exploration of motivation terminology.

Contemporary Educational Psychology, 25, 3-53.

25 .

Chapter 8: Survey Research Activities embedded in the etext MyLab Education Application Exercise 8.1: Understanding Survey Research MyLab Education Application Exercise 8.2: Reading Research: Identifying Steps in the Research Process for a Survey Study MyLab Education Application Exercise 8.3: Understanding Research: Evaluating a Survey Study MyLab Education Self-Checks for Each Learning Objective

Suggested Activities and Resources 1. As the text notes, many resources are available to assist in constructing surveys to be delivered electronically. Select one of the web-based survey tools listed in the text (SurveyMonkey.com, Zoomerang, LimeSurvey, eSurveypro, or Qualtrics). Use it as the basis for one of the following applied exercises for Chapter 8 content.

• One strategy would be to create an instrument as a class activity. Select a topic (e.g., school uniforms, school vouchers, school safety) and ask students to work in pairs generate a set number of items to address that topic.

• An alternative strategy is to upload an existing scale. In the Chapter 7 exercises, a website by James McCroskey (jamescmccroskey.com/measures/) provides numerous measures that can be used for research purposes with appropriate citation. Have students enter one of these measures into a web-based survey tool. This activity is best done in pairs. Although it takes time, students will learn that although the technology is helpful it is not as easy as sending a file through email.

For any of these options, require that demographic items and also an open-item be included to assure that they practice creating a survey that includes varied item formats.

2. Many researchers are conducting surveys that are delivered electronically. Some references of articles also follow to support this content. It is relevant content to include in class to supplement Chapter 8. The Archer (2003) article, for example, provides some of the benefits of conducting webbased surveys. The article is brief and can serve as an excellent foundation for class discussion.

• • • •

Archer, T. M. (2003). Web-based surveys. Journal of Extension, 41(4). Retrieved from joe.org/joe/2003august/tt6.shtml Bertot, J. C. & McClure, C. R. (1996). Electronic Surveys: Methodological implications for using the World Wide Web to collect survey data. Proceedings of the ASIS Annual Meeting, 33, 173-185.

Goree, C. T., & Marszalek, J. F. III (1995). Electronic Surveys: Ethical issues for researchers.

College Student Affairs Journal, 15, 75-79.

Watson, S. C. (1998). A primer in survey research. Journal of Continuing Higher Education, 46, 31-40.

3. Bring bad examples of surveys to class and have the class restructure them as a class exercise. Bad examples are easy to find, both examples you have received or from the research literature.

Sometimes it is fun to bring in magazine types of surveys or those found on line. To reinforce 26 .

concepts, have students identify the level of data generated from each of the items on your sample survey and we work together to modify the survey items.

27 .

Chapter 9: Correlational Research Activities embedded in the etext MyLab Education Application Exercise 9.1: Examining the Purpose of Correlational Research MyLab Education Application Exercise 9.2: Reading Research: Identifying Steps in the Research Process for a Correlational Study MyLab Education Application Exercise 9.3: Thinking Like a Researcher: Interpreting Correlational Data MyLab Education Application Exercise 9.4: Understanding Different Statistical Approaches to Correlation MyLab Education Application Exercise 9.5: Understanding Research: Evaluating a Correlational Study MyLab Education Application Exercise 9.6: Understanding Correlational Research MyLab Education Application Exercise 9.7: Interpreting Correlations MyLab Education Self-Checks for Each Learning Objective Suggested Activities and Resources 1. Many students do not understand what a correlation is or means. Prior to addressing Chapter 9 content in class, have students log onto a free calculation site for a Pearson r. One example is go onto wessa.net and select the correlation module. This site allows visitors to calculate a Pearson r correlations. Have students collect data on two variables from 10 people. They should then record the correlation. This is a more meaningful activity if students also create a scatter plot of the data they obtain. Without the graph, correlation remains a fuzzy concept for some.

2. To understand simple correlation conceptually, the Glover, Ronning, and Bruning (1990) text (p. 109) presented correlations from the literature on correlations on intelligence tests related to genetics and environmental influences. Among other numbers, they provide the following correlations: identical twins reared together (.90), identical twins reared separately (.74), heterozygotic twins of same sex reared together (.60), heterozygotic twins of different sex reared together (.55), Non-twin siblings reared together (.50), non-twin siblings reared separately (.35), unrelated children reared in the same home (.20). After introducing the concept of correlation have students estimate the correlations and then share the numbers. Draw approximate scatter plots to illustrate the correlations graphically.

3. A recent topic of interest to many students is the relationship between teacher behaviors and increased standardized testing, perhaps related to NCLB. For this activity, have students work with a peer to generate several correlational research questions related to the topic. Request that students generate examples of two variable possibilities, as well as prediction possibilities or path models with multiple variables. The examples below can be used to start the activity. Students generally get engaged with this topic and generate several examples.

Is there a relationship between assigned homework and standardized mathematics scores?

28 .

Is there a correlation between amount of writing practice done with sample items and writing standardized test scores?

Does professional development, years of teaching the same grade, or SES of the school’s students best predict third grade reading test scores?

4. In order to understand correlational studies, one successful example of a criterion variable is retention or college student persistence. Start with a large Venn diagram on the board and have students generate with you the potential variables that might predict college students’ inclination (or ability) to remain enrolled in post-secondary education. Make a list of all of the variables and then have students predict how much variance each of the variables might account for. As this exercise progresses take ‘variance’ out of the diagram. Soon students will see that they have too many variables and will draw the conclusion that these variables do not count for independent or unique variance and that the predictor variables are also related. It is an example everyone can understand and it always works well. Some review of theoretical and empirical research can help you guide the activity; some examples follow: Peltier, G. L., Laden, R., & Matranga, M. (1999). Student persistence in college: A review of research. Journal of College Student Retention, 1, 357-375.

Tinto, V. (1998). Colleges as communities: Taking research on student persistence seriously.

Review of Higher Education, 21, 167-177.

Terenzini, P. T., Pascarella, E. T., & Blimling, G. S. (1999). Students’ out-of-class experiences and their influence on learning and cognitive development: A literature review. Journal of College Student Development, 40, 610-623.

5. Explain the existing secondary datasets. As mentioned generally in other activities, extant databases may provide an opportunity for interesting research questions. Many of the database studies employ correlational and modeling examples. The IES website on the ECLS-K 98-99 nces.ed.gov/ecls/kindergarten.asp provides links to some of the publications that have used these data.

Many demonstrate modeling and predictions and can support that prediction studies extend correlational research.

6. Another area of research that addresses correlational relationships, but uses various modeling techniques, is the area of academic self-concept as related to achievement. This topic is a good one to use when you have a large range of prior knowledge and interests in a class, because, for example, the more knowledgeable and able students will be very interested in the analysis strategies and interpreting the models; while the students who are just being introduced to research will understand the research question being asked. The topic lends itself well to discussion about reciprocal relationships in educational research and also to causal ordering and structural modeling.

The fundamental question is the relationship between academic self-concept and achievement and the ordering of the relationship. Of course, those students who are high achievers also often have high academic self-concept. Therefore, higher academic self-concept is related to higher achievement. Is there a causal relationship between these variables? Does academic achievement lead to higher selfconcept? Does higher academic self-concept lead to higher achievement? Through discussion with the class it becomes apparent that it could be either way---or both!

29 .

To facilitate discussion and introduce students to both the topic and causal ordering through models, Herbert Marsh’s work stands out as a great foundation. I have provided some of the best references from the last 15 years below. The arguments in his work are laid out for novices and the analyses provided will make even the most advanced student interested. For this exercise I have used a JIGSAW cooperative learning technique that works well. I have stratified my students in the groups so that each group has a student who can assist with interpreting the analysis. Assign students, or allow them to select with assistance so you get a reasonable distribution of students to articles, to read one of the articles listed below. Upon arrival at class, divide students into groups based upon which article they read. In these groups the students discuss the article and identify the main points. Task 2 criteria works well as a framework to identify the main points. These people are now ‘experts’ on the articles that they have read. After discussion in these groups each expert goes to a second group where experts from each article are present and all of the articles are discussed in this group.

This activity does take a little class time but students enjoy it and the cooperative nature of the activity allows for more hands-on time with correlational research. One of the strengths of this activity is that peers act as scaffolds and provide support for each other in reading and interpreting research.

Guay, F., Marsh, H., Boivin, M. (2003). Academic self-concept and academic achievement: Developmental perspectives on their causal ordering. Journal of Educational Psychology, 95(1), 124136.

Marsh, H.W. (1990). Causal ordering of academic self-concept and academic achievement: A multiwave, longitudinal analysis. Journal of Educational Psychology, 82(4), 646-656 Marsh, H.W., Bryne, B. M., Yeung, A. S. (1999). Causal ordering of academic self-concept and achievement: Reanalysis of a pioneering study and revised recommendations. Educational Psychologist, 34(3), 155-167.

Marsh, H. W., Trautwein, U., Ludtke, O., Koller, O., Baumert, J. (2005). Academic self-concept, interest, grades, and standardized test scores: Reciprocal effects models of causal ordering. Child Development, 76(2), 397-416.

Other related articles include Helmke, A., van Aken, M. (1995). The causal ordering of academic achievement and self-concept of ability during elementary school: A longitudinal study. Journal of Educational Psychology, 87(4), 624-637.

Marsh, H. W., Hau, K. T., Kong, C. K. (2002). Multilevel causal ordering of academic self-concept and achievement: Influence of language of instruction (English compared to Chinese) for Hong Kong students. American Educational Research Journal, 39(3), 727-763.

Marsh, H.W., & Perry, C. (2005). Self-concept contributes to winning gold medals: Causal ordering of self-concept and elite swimming performance. Journal of Sport and Exercise Psychology, 27(1), 71-91.

30 .

Chapter 10: Causal-Comparative Research Activities embedded in the etext MyLab Education Application Exercise 10.1: Understanding Research: Evaluating a CausalComparative Study MyLab Education Application Exercise 10.2: Reading Research: Identifying Steps in the Research Process for a Causal-Comparative Study MyLab Education Application Exercise 10.3: Thinking Like a Researcher: Planning a CausalComparative Study MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. Chapter 10 content can be supplemented by having students read examples of causal-comparative research. In addition, a class example, such as one that addresses gender differences, or aptitude differences (high IQ, low IQ) groups are clear examples to illustrate the content of Chapter 10.

2. To support the learning outcomes of Chapter 10, one commonly discussed issue in education is students’ attitudes and achievement in mathematics and science. Often, there are gender differences in both. The TIMSS publication referenced below, Gender Differences in Achievement, is a great resource to use to discuss not only Chapter 10 content, but also international testing and extant database research. This also is of interest to and supports engagement of my international students as the context is not only United States schooling.

Mullis, I.V.S., Martin, M.O., Fierros, E.G., Goldberg, A. L., & Stemler, S.E. (2000). Gender differences in achievement. IEA’s Third Internatonal Mathematics and Science Study (TIMSS).

International Association for the Evaluation of Educational Achievement: Boston, MA. a. Share the general purpose of the research and the report and provide a link to the report: timss.bc.edu/timss1995i/TIMSSPDF/t95_gender_all.pdf b. Have students generate research questions that would illustrate causal-comparative research.

These would include gender differences but also require at least one additional causalcomparative question per student (e.g., year of testing, national status, ability status). c. Ask students to represent the design of their research question. d. Does this research question represent a prospective or retrospective causal comparative study? e. Examine the use of descriptive as well as inferential statistics in the report to introduce content later found in Chapters 18 and 19.

3. Chapter 23 includes specific criteria for evaluating causal-comparative studies, and suggests that when reading causal-comparative studies one should specifically consider the following concerns.

Previewing these review criteria with Chapter 10 content and class examples of articles or topics will provide an additional means to promote transfer. The students can identify which type of causalcomparative research, retrospective or prospective, as noted in the text and below. o

Are the characteristics or experiences that differentiate the groups (i.e., the grouping variable) clearly defined or described?

31 .

o

Are critical extraneous variables identified?

o

Were any control procedures applied to equate the groups on extraneous variables?

o

Are causal relations discussed with due caution?

o

Are plausible alternative hypothesis discussed?

The text discusses two types of causal-comparative research: retrospective and prospective. This may be confusing for the students. One strategy is to illustrate the differences between the two with examples. Make note, however, that the independent variable and the dependent variable might be the same. In retrospective causal-comparative studies, we know there are differences in a dependent measure and we are interested in whether potential differences in independent variables are present as well. Dependent measures that are good examples are test scores, grades, motivation, interest in a particular subject area, or creativity. In prospective causal-comparative studies, we know there are different groups. Gender, SES, ability, learning disabilities, and prior knowledge are good examples.

We test to see if these differences might correspond to differences in dependent measures. Work through examples in class that illustrate the differences between the two types.

Retrospective Causal-Comparative Example It is known that participants differ on some dependent measure, for example, standardized test scores, and the researcher is interested in testing hypothesis related to independent variables, for example, ethnicity.

Prospective Causal-Comparative Example It is known that participants differ on some point (e.g., gender), and the researcher addresses whether participants differ on dependent variables (e.g., interest in engineering courses).

4. Reading and discussing examples of causal-comparative research, of course, also assists the students.

One area where numerous causal-comparative studies are conducted is in racial and ethnicity differences. Below is a reference for a paper that presents a causal-comparative study that has broad appeal. For this activity it is not necessary to have students get the article, the topic alone can serve as a basis for discussion. Some general guided questions follow that can facilitate thinking about causal-comparative research. Introduce the topic and have the students work in pairs to answer the following questions (10-15 minutes) and then discuss some of the answers as a full group. Not all answers should have to focus on racial differences.

Carter, R. T., Scales, J. E., Juby, H.L., Collins, N.M., Wan, C.M. (2003). Seeking career services on campus: Racial differences in referral, process, and outcome. Journal of Career Assessment, 11(4), 383-404.

Given the topic of seeking career services at a college or university, generate three causalcomparative research questions.

For each question provide the dependent variable and suggest one way you would measure that dependent variable.

For each question provide the independent variable and levels of that variable.

32 .

For each question determine if the study is a prospective causal-comparative study or a retrospective casual comparative study.

The following are example responses that you can use to begin the discussion.

Given the topic of seeking career services at a college or university, generate three causalcomparative research questions.

Are there differences between the proportions of men versus women who seek career counseling?

Are there differences between African American and Caucasian students in frequency of counseling sought?

Are there differences in satisfaction with career counseling among ethnic and racial groups of students? * Other avenues for additional causal comparative research question examples include traditional/nontraditional; differences by major; year of study; geographical location; type of institution (private, public) For each question provide the dependent variable and suggest one way you would measure that dependent variable.

The dependent variable for this question is a proportion so it is measured as who seeks counseling as a ratio of total number of students enrolled.

The dependent variable for this question is frequency of counseling sought and is measured by a count.

The dependent variable for this question is satisfaction and likely would be measured via a survey or interview.

For each question provide the independent variable and levels of that variable.

Gender (two levels; male and female) Race (two levels; African American and Caucasian) Ethnic group (undetermined number of levels from the question but would likely include; Hispanic, Asian American, African American, Native American, etc)

33 .

Handout 10.5 Generate a causal-comparative research question that one could examine to address high school class attendance.

What is the independent variable? Use Figure 9.1 in your text to label the independent variable.

Identify the levels of the independent variable.

What is the dependent variable? How will you measure the construct?

34 .

Handout 10.5 (Sample answers) Generate a causal-comparative research question that one could examine to address high school class attendance.

Answers will vary. Suggested responses include: What differences exist between students of different SES groups in their high school class attendance?

What is the independent variable? Use Figure 10.1 in your text to label the independent variable.

Answers will vary. Suggested responses include: As examples have students grouped by whether they are athletes or involved in activities with attendance policies, academic level of student, homeroom teacher, academic level, class standing, gender, or SES. Remind students to categorize as family-related, personality, etc. as found in Figure 10.1.

Identify the levels of the independent variable.

Answers will vary: Gender (male, female) SES (group by family class, parent education or something like that) ‘Walkers’ versus ‘Busers’ First-year versus Seniors Athlete or non-athlete Participant or non-participant in school activities What is the dependent variable? How will you measure the construct?

Generally students report attendance is the dependent variable (sometimes they note improved attendance). Usually we measure it as missed days in a quarter or term. Their response to this answer does not generally change across the two questions that they generate. This allows for you to help them to notice that numerous ‘causes’ can affect the same outcome.

35 .

Chapter 11: Experimental Research Activities embedded in the etext MyLab Education Application Exercise 11.1: Evaluating Research Articles: Identifying Steps in the Research Process for an Experimental Study MyLab Education Application Exercise 11.2: Evaluating Research Articles: Evaluating an Experimental Study MyLab Education Application Exercise 11.3: Designing an Experimental Study MyLab Education Self-Checks for Each Learning Objective Suggested Activities The content in Chapter 11 tends to be a bit overwhelming for novice students. It introduces new terminology to remember. Several examples of most text content are really necessary for student understanding. The four-part approach below can assist with teaching the content.

1. Introduce threats to validity with a variety of examples. Next, present each of the main research designs briefly with an example. Pull together how each design addresses the threats to validity. This is a bit overwhelming for the class so also use another example and step through it again.

2. One example that works exceptionally well for this exercise is SAT preparation training programs.

As a class consider the ways one could conduct SAT training programs. The example works very well to discuss all of the research designs. Start with pre-experimental and go through all of the between subjects designs. Further, as you illustrate the designs, you can also easily discuss the threats to validity. Also include, with interspersed examples, how compensatory rivalry, experimenter effects, the placebo effect, and treatment diffusion could affect the validity.

3. As you work through both Activities 1 and 2, diagram the designs with the class. At first they think it is really weird but as they catch on it is really helpful to their understanding to use the notations.

4. Next, allow students to work in groups to work through another example. Have students generate how a study could be conducted for all types of designs. Some topics to consider include a. Type of language learning programs is another example that works well. Any form of text supplements, such as animation, pictures, or added questions. b. Various tutoring programs (e.g., peer tutoring, reciprocal tutoring, teacher tutoring, paid tutors) is another example that works well. c. Psychomotor domain examples also work well. Groups of students learning a skill, such as tennis or golf. The threats to external validity are particularly clear to students given a performance or skill domain.

5. An IES publication produced by the coalition of evidence based policy and the what works clearinghouse entitled Key Items to Get Right When Conducting a Randomized Control Trial in Education written by J. Baron can serve as outside text support regarding how to conduct experimental work in education. The short 2005 publication is essentially a checklist with discussion that breaks the research process down into 4 sections and provides key items to address in each 36 .

section (planning, random assignment process, measuring outcomes, and analysis). Assign this publication for out of class reading. Handout 10.5 as found below can provide guiding questions to answer as they read to bring back and use as foundation for discussion in class. There are other additional points that are also important in the article but these guided questions can serve as the basis for a discussion of many of the primary points made in the publication that overlap with Chapter 10 content.

Handout 11.5 Use the Baron (2005) publication, Key Items to Get Right When Conducting a Randomized Control Trial in Education. Answer the following questions.

The publication is available at: coalition4evidence.org/wp-content/uploads/2012/05/Guide-Key-items-to-Get-Right-RCT.pdf

1. The first section of the publication notes key items to get right in planning the study. a. What does the author mean by surrogate outcomes? In a study of mathematical problem solving what might be an example of a surrogate outcome? Why are surrogate outcomes a concern? b. The author presents two reasons when it may be more advisable to randomly assign groups instead of individuals. In the first reason, ‘spillover’ effects are presented. How do such spillover effects relate to threats to validity as discussed in Chapter 10 of your text? Select two examples and explain. c. As discussed elsewhere, the author promotes the use of a power analysis. What key features of the design are necessary for including in an a priori power analysis? Pick one and provide an example.

2. The second section of the publication notes key items to consider during the random assignment process. a. Why would it be advisable to have someone not invested in the study conduct the random assignment? b. Random assignment is sometimes challenging. What does the author mean when he says make sure the random assignment is truly random? What is one example of a non truly random assignment someone might conduct? Why does example present a concern? c. Why would a participant need a unique personal identifier? d. The author suggests that researchers monitor what services the control group might be receiving. Suggest one reason why this can be a concern. Provide an example.

3. The third section of the publication notes key items to get right in measuring outcomes for the study sample. a. What are some of the key characteristics you would seek in your instrumentation? Explain in your own words why each is important.

37 .

b. What is one way you might be able to limit subjectivity in administration? c. Why might you want to keep track of those who fail to complete the study? How does this relate to the threats discussed in your text? d. What are some of the benefits and drawbacks of collecting long term outcomes?

4. The fourth section notes key items to get right in the analysis of study results a. How should you analyze those individuals who may have benefited from cross-over effects?

Why? b. In addition to statistical significance of the intervention’s effect what else should be reported?

38 .

Handout 11.5 (Points to lead discussion) Use the Baron (2005) publication, Key Items to Get Right When Conducting a Randomized Control Trial in Education. Answer the following questions.

The publication is available at: coalition4evidence.org/wp-content/uploads/2012/05/Guide-Key-items-to-Get-Right-RCT.pdf

1. The first section of the publication notes key items to get right in planning the study. a. What does the author mean by surrogate outcomes? In a study of mathematical problem solving what might be an example of a surrogate outcome? Why are surrogate outcomes a concern?

The author means outcomes that are not exactly the purpose of the intervention. Often researchers don’t measure outcomes such as behavior but instead reach to measure attitude. In a study of mathematical problem solving instead of measuring problem solving one might measure other mathematics skills necessary for problem solving, or beliefs about ability to solve problems as surrogate outcomes. Surrogate outcomes are a concern because they may not actually be analogous to outcomes in the true outcomes of a targeted intervention. b. The author presents two reasons when it may be more advisable to randomly assign groups instead of individuals. In the first reason, ‘spillover’ effects are presented. How do such spillover effects relate to threats to validity as discussed in Chapter 10 of your text? Select two examples and explain.

Spillover effects are related to treatment diffusion as discussed in our text. Those not intended to receive the intervention might indeed receive some. In school based studies, teachers might talk about the intervention and others might try some of the components, for example, or students studying together might share a new comprehension strategy. Spillover effects can also lead to other reactive effects, such as the John Henry effect. If teachers or students in the control group think that they are not getting the treatment, they may compensate and try harder or use a different strategy, in affect changing the nature of the control group. c. As discussed elsewhere, the author promotes the use of a power analysis. What key features of the design are necessary for including in an a priori power analysis? Pick one and provide an example.

We shared in class how to do a priori power analysis to determine sample size (G power). Power analysis can help determine the costs of a study (how many participants are needed).

• • •

A power analysis needs to consider whether individuals, groups, or both will be randomly assigned. (This will be important to calculate number needed based upon individual or group effect sizes.)

Whether subgroups will be considered prior to randomization (This determines decisions in stratification.)

Whether fixed or random effects models are considered (This will be important when we consider the size of the target population.)

39 .

•

Whether statistical techniques will also be used to detect effects. (This may assist in number needed.)

2. The second section of the publication notes key items to consider during the random assignment process. a. Why would it be advisable to have someone not invested in the study conduct the random assignment?

This can help to combat experimenter effects. It is best, when possible, those assigning and scoring are’ blind’ to condition. b. Random assignment is sometimes challenging. What does the author mean when he says make sure the random assignment is truly random? What is one example of a non truly random assignment someone might conduct? Why does example present a concern?

The students will generate varying examples, but usually they suggest things like either/or or assign random groups by time of day, or something of the like. Remind students about how selection threats can cause problems both independently and in combination with other threats. c. Why would a participant need a unique personal identifier?

This provides an opportunity to remind students of human subjects procedures. Individual identifiers also make true random assignment more easily executed. Further, identifiers assist in the tracking of individuals in the study. d. The author suggests that researchers monitor what services the control group might be receiving. Suggest one reason why this can be a concern. Provide an example.

One reason to monitor what is happening in the control group is to control for validity threats—both internal and external. If a new reading intervention is being tested and elements of that intervention are also in the control group instruction, this will result in an inability of determining what elements of the treatment worked and it also could be related to selection or history or other internal validity threats.

3. The third section of the publication notes key items to get right in measuring outcomes for the study sample. a. What are some of the key characteristics you would seek in your instrumentation? Explain in your own words why each is important.

This is the time to remind students of reliability and validity concerns. In short, if instrumentation is not reliable or valid we will be unable to adequately determine the effect of our intervention. Poor measures are also is related to instrumentation threats and also regression threats. b. What is one way you might be able to limit subjectivity in administration?

Well trained administrators that administer across settings and conditions can decrease subjectivity and resultant experimenter effects.

40 .

c. Why might you want to keep track of those who fail to complete the study? How does this relate to the threats discussed in your text?

Attrition might be one reason to keep track of those who fail to complete the study. It may also be important for reactive effects. d. What are some of the benefits and drawbacks of collecting long term outcomes?

One drawback is attrition or mortality. Another drawback is cost. Benefits include longer term analysis of effects and external validity knowledge, especially if participants can be tracked and a battery of measures adequately address critical constructs used over time.

4. The fourth section notes key items to get right in the analysis of study results a. How should you analyze those individuals who may have benefited from cross-over effects?

Why? Although noted, these individuals should be kept in their original group so as not to inflate the treatment effect. This is a great time to revisit Type 1 and Type 2 errors. b. In addition to statistical significance of the intervention’s effect what else should be reported?

Experimenters should also present effect size data to help to judge the practical as well as statistical significance of the intervention effects.

41 .

6. The text notes some of the difficulties educational researchers face when employing experimental designs. These difficulties center around several key issues. Some of these issues are listed below.

Use these as a starting place to discuss experimental research in educational settings as well as threats to internal and external validity. Some discussion of these issues now will lead to a deeper understanding not only of Chapter 11 content, but also of action research as presented in Chapter 17.

Who wants to be a control group member? Does there have to be a control group? How do we conduct an experiment and still teach all learners if the outcome is achievement?

Sometimes it is unethical to NOT provide a treatment (especially one with some efficacy evidence) to some students and not to others. How can we resolve this ethical concern?

How can we effectively randomly assign students to treatments? In school settings this often leads to threats of internal validity, such as treatment diffusion and compensatory rivalry.

How do we balance the need for sound research and the amount of instructional time it takes to conduct research?

In classroom-based research there are numerous threats to external validity. Does the teacher matter?

Does the group dynamic of the class influence our findings? How do we conduct research that can generalize?

42 .

Chapter 12: Single-subject Experimental Research Activities embedded in the etext MyLab Education Application Exercise 12.1: Evaluating Research Articles: Identifying Steps in the Research Process for a Single-Subject Study MyLab Education Application Exercise 12.2: Evaluating Research Articles: Evaluating a Single-Subject Design Study MyLab Education Application Exercise 12.3: Designing Single-Subject Studies MyLab Education Application Exercise 12.4: The Importance of Replication MyLab Education Self-Checks for Each Learning Objective Suggested Activities The Chapter 12 content will appeal directly to the special education and counseling and clinical psychology students. Speech and language specialists also tend to have experience with these designs.

The activities and references below, however, can help all students see broader applications for singlesubject designs.

1. One challenge students face when we get to Chapter 12 is that we focused most of our research questions and hypotheses on the notion we were looking at groups in research. It is helpful to discuss examples of research questions that are suited for single subject designs and those that are not. Use the following examples as an activity to start discussion of Single-Subject Research. These questions are found in a handout form in the Appendix. Students will need about 8-10 minutes and it works well to have them work in pairs.

Handout 12.1 Determine if each of these general questions could be addressed in a single-subject design. If so, how might be the study conducted? Please draw a representation of one proposed design.

1. Do students who sit at tables in science class achieve more than those who sit in desks?

2. Do students taught physics through direct instruction learn more than those taught through experiential learning?

3. Does providing students recess decrease out of seat behaviors?

4. Does providing students web-based practice increase math fluency?

5. Does structured peer-tutoring benefit second language vocabulary?

43 .

Handout 12.1 (Some thoughts to guide the discussion) Determine if each of these general questions could be addressed in a single-subject design. If so, how might be the study conducted?

1. Do students who sit at tables in science class achieve more than those who sit in desks?

This question would be challenging to address with a single-subject design. The purpose of the study is to compare groups.

2. Do students taught physics through direct instruction learn more than those taught through experiential learning?

On the surface, this question also appears to be about groups. However, in single-subject designs you can manipulate multiple independent variables. This could be done in this study with a crossover design and a number of individuals. Generally though this would likely be conducted as a group design study.

3. Does providing students recess decrease out of seat behaviors?

This study could easily be conducted with a single-subject design. Individual students out of seat behaviors could be tracked for baseline, data could be collected in a treatment phase where students do receive recess.

4. Does providing students web-based practice increase math fluency?

As with the above example, this study could easily be conducted with a single-subject design. We would not expect a complete return to baseline in a study such as this. We could implement a changing criterion design in this study.

5. Does structured peer-tutoring benefit second language vocabulary?

This example is much like the one above for math fluency. This type of study is a bit more complex given the peer component but nonetheless this question can be addressed in a single subject design.

2. As with the between-group designs, begin with a single topic area and work through all of the designs with one example. This allows students to see the connections among the designs. Again, as with the group designs, such an exercise also allows for clear discussion of validity threats. Unless your class includes ONLY special educators or clinical or school psychologists, a topic that is more general will likely work better than a specific one. Behavioral interventions that have broad appeal are acting out in class or homework completion. These two topic areas can be catered to various student populations in age of student, target behavior, and intervention strategy.

3. Have students generate how a study could be conducted for all types of designs. Some topics to consider follow. a. Participation in class, increased silent study time, and tardiness are all topics that work well for this activity. b. A performance skill or domain, such as those found in exercise and training c. or workforce settings, also work well to demonstrate behavioral interventions.

44 .

4. The text mentions that single-subject designs have been given more attention recently as viable alternatives to group experimental designs. The following references review the use of single-subject designs in several fields. In most of these articles the benefits of single subject designs and some limitations of group designs in each discipline are discussed. All four of these articles present a position that most students, even those with limited discipline-based prior knowledge, will easily understand.

Callahan, C., & Barisa, M. T. (2005). Statistical process control and rehabilitation outcome: The single-subject design reconsidered. Rehabilitation Psychology, 50(1), 24-33.

Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., Wolery, M. (2005). The use of single subject research to identify evidence based practice in special education. Exceptional Children, 71(2), 165-179.

Martin, G. L., Thompson, K., & Regehr, K. (2004). Studies using single-subject designs in sport psychology: 30 years of research. Behavior Analyst, 27(2), 263-280.

Skinner, C. H. (2004). Single-subject designs: Procedures that allow school psychologists to contribute the intervention evaluation and validation process. Journal of Applied School Psychology, 20(2), 1-10.

5. Some references to support single-subject designs and class activities are presented below.

Campbell (2003) presents a synthesis of research on autism since the late 60’s that has used singlesubject designs. Special educators and school psychologists, as well as others in your class, will recognize the individuality of symptoms that autistic children often present, so the topic area lends itself well to discussing single-subject designs. The article summarizes that literature well. Campbell presents a listing of all the journals he hand-searched as part of the methodology for the study. This list of journals is a valuable resource of models for those interested in conducting a single-subjects study. The article also addresses some of the validity threats inherent in the studies in the synthesis and can provide a clear connection between design and validity for the students.

Bailey and Angell (2005), Fallon and others (2004), Jones & Schwartz (2004), Lammi & Law (2003), Schilling and Schwartz (2004), and Trolinder, Choi, and Proctor (2004) all present recent studies that have employed single-subject designs from a variety of research disciplines. The Boulineuau et al.

(2004) article illustrates the use of a more descriptive single-subject design in a learning domain. The Shadish and Rindskopf (2007) article is a recent publication from a funded IES project that presents meta-analytic strategies for single-subject designs.

Bailey, R.L., & Angell, M. E. (2005). Improving feeding skills and mealtime behaviors in children and youth with disabilities. Education and Training in Developmental Disabilities, 40(1), 80-96.

Boulineau, T., Fore, C., Hagan-Burke, S, & Burke, M.D. (2004). Use of story-mapping to increase the story grammar text comprehension of elementary students with learning disabilities.

Learning Disability Quarterly, 27(2), 105-121.

Campbell, J. M (2003). Efficacy of behavioral interventions for reducing problem behavior in persons with autism: A quantitative synthesis of single-subject research. Research in Developmental Disabilities, 24(2), 120-138.

45 .

Fallon, K. A., Light, J., McNaughton, D., Drager, K., Hammer, C. (2004). The effects of direct instruction on the single word reading skills of children who require augmentative and alternative communication. Journal of Speech, Language, and Hearing Research, 47(6), 1124-1439.

Jones, C. D., & Schwartz, I. S. (2004). Siblings, peers, and adults: Differential effects of models for children with autism. Topics in Early Childhood Special Education, 24(4), 187-198.

Lammi, B. M., Law, M. (2003). The effects of Family Centered Functional Therapy on the occupational performance of children with cerebral palsy. Canadian Journal of Occupational Therapy, 70(5), 285-297.

Schilling, D. L., & Schwartz, I. S. (2004). Alternative seating for young children with autism spectrum disorder: Effects on classroom behavior. Journal of Autism and Developmental Disorders, 34(4), 423-432.

Shadish, W. R., & Rindskopf, D. M. (2007). Methods for evidence-based practice: Quantitative synthesis of single-subject designs. New Directions for Evaluation, 113, 95-109.

46 .

Chapter 13: Narrative Research Activities embedded in the etext MyLab Education Application Exercise 13.1: Understanding the Purpose of Narrative Research MyLab Education Application Exercise 13.2: Evaluating Research Articles: Identifying Steps in the Research Process for a Narrative Study MyLab Education Application Exercise 13.3: The Key Characteristics of Narrative Research MyLab Education Application Exercise 13.4: Evaluating Research Articles: Evaluating a Narrative Study MyLab Education Application Exercise 13.5: Collecting Narrative Data MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. In November 2009, the Educational Researcher published an issue Discourse on Narrative. This is a current resource for your students that discusses narrative research more deeply than the text. Have students read the opening article by Coulter and Smith (2009) The Construction Zone. The questions below support reading and discussing the article.

Handout 13.1 Coulter and Smith introduce this themed issue of Educational Researcher with a piece titled The Construction Zone: Literary Elements in Narrative Research.

1. What is the primary purpose of this article as suggested by the authors?

2. Compare the description found in the article to that presented in text for narrative analysis and analysis of narrative.

3. What is meant by literal truth and storied truth?

4. The authors present several literary elements. Select two and dissect at least one example of each from those shared from the texts they selected to illustrate in their article.

Handout 13.1 Reference points from the article as foundation and instructor questions for discussion 1. What is the primary purpose of this article as suggested by the authors?

47 .

The authors focus on the construction of narratives as representations of research studies—they describe literary elements that will aid researchers Where else have you thought about these elements? How do these elements relate to your conceptions of ‘research’?

This article makes narrative research very real when compared to reading about it.

What surprised you in this article?

2. Compare the description found in the article to that presented in text for narrative analysis and analysis of narrative.

The article and the text definitions are very similar. Narrative analysis is also termed narrative construction by Barone and others as mentioned in the article.

What do you think about the term narrative construction? Is this an accurate account of what narrative analysis is? Is this research?

3. What is meant by literal truth and storied truth?

Literal truth is what is sought in objective traditional research. Storied truth is a representation constructed through the participants and the researcher.

How does storied truth represent research? (Students will really vary in their positions on this point of discussion. The article actually works to increase the varying stances represented. Most of the classes that I teach struggle with this as research. As such, I have to ‘side’ a little with the author to keep the discussion moving. Some of the context in my case is the likely reason and your experience may be different. The variance in experience with this stance is likely due to selection into a traditional research methods class and the nature of scholars at the department and or institution.)

4. The authors present several literary elements. Select two and dissect at least one example of each from those shared from the texts they selected to illustrate in their article.

The examples that the students will select will vary. For your quick reference, the elements presented include: Point of view (lens through which narrative is told) Person (who tells the story) Omniscience (amount of knowledge the narrator assumes in the narrative) Narrator reliability (degree to which reader can trust the narrator) Narrative voice (researcher can vary point of view and may share many voices) Authorial distance (immediacy between events, time, and place of writing) Tone (intensity and volume) Metaphor, figurative language, and theme (Unifying metaphor, figurative language, and them 48 .

increases likelihood of reader emotional response) Placing action within scenes (sharing setting, actors, and action: requires rich detail) Time (time need not be chronological) 2. The best way for the students to really understand narrative is to have them read narrative research reports. An easily accessible brief ERIC document that can assist in the introduction to narrative research is referenced below. Among other characteristics of narrative research illustrated in this summary, the authors demonstrate portions of narrative analysis, how interviews can relate to stories, and how restorying shapes a final report. Cultural anthropology journals are a good place for students to find additional narratives to assist their understanding.

Zimmerman, L., W., McQueen, L., & Guy, G. (2003). Connecting women’s voices: a symposium on the intersection of gender, race, and class. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. [ERIC Number ED479147].

3. Ask the students to listen to a This American Life (TAL) segment on National Public Radio as a way to introduce narratives. Several TAL segments are much like narrative research and can illustrate how the restorying process can work. Free streaming audio and podcasts are available. The website (thisamericanlife.org) also includes overviews and links to critical reviews of their work.

4. One means to assist students in a beginning understanding of narrative is to simply have them interview each other regarding a personal challenge. In essence, this is one means of starting to conduct an oral history. As students share a personal challenge they generally embellish a lot. The interviewer should take notes to practice these skills and pose questions to himself or herself regarding parts of the ‘story’ that would be of further interest to research. You might ask students to request a timeline from their participants and use this strategy to assist the oral history. Some example challenges include • Going back to school • Teaching a new topic/content/curriculum • Balancing life with a family and other obligations • Learning research methods • The personal history of a prized hobby • Their grandparents’ ‘story’ 5. Several websites provide additional resources for students to learn about narrative research and read examples. The AERA Narrative Research Special Interest Group (SIG) houses a website (sites.google.com/site/aeranarrativeresearchsig/) that announces important conferences and opportunities for narrative researchers. In addition, the International Congress of Qualitative Inquiry is an annual conference focused on qualitative research. It includes papers by narrative researchers along with other types of qualitative research. The full program and abstracts are available online (icqi.org).

49 .

Chapter 14: Ethnographic Research Activities embedded in the etext MyLab Education Application Exercise 14.1: Evaluating Research Articles: Identifying the Steps of Ethnographic Research MyLab Education Application Exercise 14.2: Identifying the Characteristics of Ethnographic Research MyLab Education Application Exercise 14.3: Designing an Ethnographic Study MyLab Education Application Exercise 14.4: Conducting Ethnographic Research MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. One activity for the class is to have students generate ideas for future ethnographic. This prompts students to think about the types of topics that an ethnographer might study and those types of research topics that are not well suited to ethnographic research.

2. Students are generally interested in ethnographies. They are generally accessible and the topics ethnographies in education address often have broad appeal. Share a couple recent ethnographies with your students. The journal Ethnography and Education published by Routledge is a good source. This journal includes interesting topics found in the existing issues. Browsing a volume or having students each read a manuscript and share with the class would provide grounded examples of ethnographies and qualitative research overall. The call for authors and the aims and scope of the journal mirror what Chapter 14 presents regarding purpose and nature of ethnographies. See tandfonline.com/loi/reae20 and search for the Aims & Scope and Authors and Submissions sections.

To support this activity, assign students to browse the abstracts of issues of the journal as a homework assignment. The abstracts for this journal are available to browse online at tandfonline.com/loi/reae20. One way to frame the assignment is to have them each review 3 abstracts of their choice and answer the following questions. In addition to introducing students to ethnography through a more hands on approach, these questions can serve as foundation for discussion of the major points of ethnography.

•

Does the article tackle or discuss principles of ethnography as a method or does it present a study?

•

If the article is a methods contribution, what is the main argument the authors stress (e.g., multiple sources, time in field, comparison of ethnography to other approaches?)

•

If the article presents a study:

•

Is the study about context, experience, process or something else?

•

What are the data sources mentioned in the abstract?

50 .

•

How long was the researcher in the field (if not stated make a guess)?

•

Does the abstract draw conclusions or make claims? If so summarize.

3. Several articles presented in Ethnography and Education also directly address the tensions ethnographers feel when conducting research, provide methodological direction to ethnographers, or address the methodologies of ethnography directly. Articles from this journal can be excellent resources for fledgling researchers and will be of interest to many. The methodological articles, in particular, can assist you with guiding students through Chapter 14 content.

51 .

Chapter 15: Case Study Research Activities embedded in the etext MyLab Education Application Exercise 15.1: Evaluating Research Articles: Evaluating a Case Study MyLab Education Application Exercise 15.2: Evaluating Research Articles: Identifying the Steps in Case Study Research MyLab Education Application Exercise 15.3: Conducting Case Study Research MyLab Education Application Exercise 15.4: Designing a Case Study MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. The Explorable website (explorable.com) provides an overview of many of the topics covered in the text. The section on case study research can be found in the research design section. When students struggle with course content this resource can provide additional examples and serve as a review to supplement the text. Share the site as necessary.

2. For Chapter 15 content, one of the best strategies to facilitate student understanding is to expose the class to case studies. Some recent examples that my students have enjoyed are referenced below.

The Woodruff and Schallert (2008) represents the cases of student athletes and the motivational conflicts that they experience. Self-determination theory served as a theoretical grounding point for the study. The Saracho (2008) article addresses findings from examining a literacy program for fathers. To work through the steps in designing a case study, one strategy would be to select one of the articles and to step through the research questions and the methods used in the study. The methodologies used in these cases provide elements to support Chapter 15. Students can also work in pairs or small groups by topic area with an alternative case study. The questions below are general enough for alternative articles.

Saracho, O. N. (2008). A literacy program for fathers: A case study. Early Childhood Education Journal, 35(4), 351-356.

Woodruff, A. L., & Schallert, D. L. (2008). Studying to play, playing to study: Nine college studentathletes’ motivational sense of self. Contemporary Educational Psychology, 33, 34-57.

1. Define or describe the bounded system that is the focus of this case study.

2. What are the research questions of the study? What is the unit of analysis?

3. Is this a single or multiple case study? How do you know?

4. What were the data sources?

5. How were the data analyzed?

6. Discuss one theme from the case.

52 .

3. Colorado State University maintains a site on writing (writing.colostate.edu). This site is a valuable resource for your students generally, but the site also has specific sections that address types of research and writing various types of research. For example, it includes a review of types of case studies, analyzing data, reporting, identifying a theoretical frame, among others related topics. The website also includes a very helpful annotated bibliography students will find useful to support their independent work if they decide upon case study methodology. The bibliography is also a nice resource for prepping class content if you intend to cover case study in a more depth than the text. Go online to writing.colostate.edu and search for case study.

53 .

Chapter 16: Mixed Methods Research Activities embedded in the etext MyLab Education Application Exercise 16.1: Identifying the Purpose of Mixed Methods MyLab Education Application Exercise 16.2: Evaluating Research Articles: Evaluating a Mixed Methods Study Part 1 MyLab Education Application Exercise 16.3: Evaluating Research Articles: Identifying Steps in the Research Process for a Mixed Methods Study MyLab Education Application Exercise 16.4: Evaluating Research Articles: Evaluating a Mixed Methods Study Part 2 MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. To familiarize students with mixed-methods research consider the abstracts from examples of recent mixed methods studies. Handout 16.1 provides example abstracts with references to use. Answers for those provided are found below. As an in-class exercise, group students in pairs or small groups and allow them to select one of the abstracts and answer the following: a. Identify the purpose of the study. b. Based upon the abstract, what is the basic design (explanatory sequential, exploratory sequential, or convergent parallel) used for the study? Provide a rationale for your answer. c. List one data source that is quantitative and one data source that is qualitative.

Student answers will likely be more elaborate than the brief answers below. This exercise usually takes 15-20 minutes if your students work in small groups and then you briefly review each abstract as an entire group.

Plowright (2014) a. Identify the purpose of the study.

This study examined postgraduate students understanding of the use of mixed methods research. b. Based upon the abstract, what is the basic design (explanatory sequential, exploratory sequential, or convergent parallel) used for the study? Provide a rationale for your answer.

Based on the abstract, this study appears to employ an explanatory sequential design. The researcher began with a quantitative survey and followed-up with small group discussions. c. List one data source that is quantitative and one data source that is qualitative.

Based upon the abstract, the online questionnaire survey seems to be quantitative. The smallgroup, face-to-face discussions appear to be qualitative.

54 .

Crede & Borrego (2007) a. Identify the purpose of the study.

The authors developed a survey to examine graduate engineering student retention b. Based upon the abstract, what is the basic design (explanatory sequential, exploratory sequential, or convergent parallel) used for the study? Provide a rationale for your answer.

The study appeared to employ and exploratory sequential design. The authors began with qualitative ethnographic inquiry, which they used to develop a quantitative survey. They then analyzed the validity and reliability of the survey, which is involve quantitative methods. c. List one data source that is quantitative and one data source that is qualitative.

The abstract identifies interviews and observations as qualitative data collected and the survey as the quantitative source.

Coryn, Schröter, & McCowen (2014) a. Identify the purpose of the study.

This study examined the factors influencing the academic achievement of native Hawaiian students in public schools. b. Based upon the abstract, what is the basic design (explanatory sequential, exploratory sequential, or convergent parallel) used for the study? Provide a rationale for your answer.

The abstract noted equal priority of qualitative and quantitative methods, which indicates a convergent parallel basic design. c. List one data source that is quantitative and one data source that is qualitative.

The abstract does not discuss the data sources. The full article describes observations, surveys, and semi-structured interviews.

Sorensen, P., Twidle, J., Childs, A., & Godwin, J. (2007) a. Identify the purpose of the study.

Very generally, the purpose of the study is to examine Internet use by teachers in schools in England. b. Based upon the abstract, what is the basic design (explanatory sequential, exploratory sequential, or convergent parallel) used for the study? Provide a rationale for your answer.

Based upon the abstract it appears that study used an explanatory sequential design. The abstract described questionnaires as the basis for analyzing trends. Then, follow-up focus groups and case studies yielded more insight into the issues identified.

55 .

d. List one data source that is quantitative and one data source that is qualitative.

It appears that the questionnaires included quantitative data, and the qualitative focus groups and case studies were used to explain the quantitative data.

2. To support the Chapter 16 content, have the class read and discuss in class the following article: Achinstein, Ogawa, & Speilgman (2004). The content of the article is of interest to many because it includes policy, teacher beliefs, and teacher development. Several methods of data collection and data reporting are included in the study. For example, among other data collected, the researchers employed observations, case studies, and questionnaires. Findings are reported both descriptively and through inferential analyses. Have students read the article and document the data sources, the provided rationale if available or an implicit reason why the data source was included, how the data were collected, and how the data were reported. Some of the data reported relies on quantitative descriptive information that is implicit to the study design. Encourage learners to use a matrix, such as the one shown below, to organize their reading and their answer.

Achinstein, B., Ogawa, R. T., & Speiglman, A. (2004). Are we creating separate and unequal tracks of teachers? The effects of state policy, local conditions, and teacher characteristics on new teacher socialization. American Educational Research Journal, 41, 557-604.

Data source

Rationale

How collected

56 .

How reported

3. As noted in the text, action research often includes either a qualitative or mixed methods approach.

For Chapter 16, have students work in pairs to address the following questions. Students will need about 20 minutes. After students have answered these questions it is beneficial to use them as class discussion not only to support Chapter 16 content but also to provide a foundation for the use of this case for Chapter 17 activities. Possible directions and answers for each question are provided. Your students’ responses will vary.

Ellen teaches the pull out gifted science program for intermediate grades in a moderately sized school district. Her program goal is to provide quality instructional experiences for exceptional science students. Targeted student outcomes for her program include not only achievement as measured by both classroom based and standardized tests, but also student interest in and motivation for science.

Two contributing factors have made Ellen question one of her instructional units, the fetal pig dissection project. First, while the district supports her program, her budget has come under scrutiny.

The fetal pig dissection is one of her more expensive units. Second, while some of the students cite this unit as their favorite, other students and their families have questioned the necessity of the dissection on moral grounds. She questions whether alternative instruction for the unit might be possible. Ellen decides to conduct a study to investigate this issue.

Please answer the following questions based upon Ellen’s case.

1. Why might Ellen want to address her issue by using a mixed methods approach? Support your response.

2. Generate a general research topic for Ellen’s study that uses a mixed methods approach.

If you were Ellen, how might you approach this study?

3. Identify the quantitative data sources you would use to address this study. What research questions would you answer and how would you analyze these data?

4. Identify the qualitative data sources you would use to address this study. What guiding hypothesis would you use and who would you select as participants?

5. Classify your study design. Support your response.

Possible responses to the above questions: Please answer the following questions based upon Ellen’s case.

1. Why might Ellen want to address her issue by using a mixed methods approach? Support your response.

• Her issue includes components that are best addressed by both quantitative and qualitative methods. Ellen is interested in both achievement and motivation and interest. She also will have to tackle beliefs and opinions. One method will not address all aspects of this study.

2. Generate a general research topic for Ellen’s study that uses a mixed methods approach.

• Are there reasons to consider instructional alternatives to the fetal pig dissection? If so, are there alternatives that maintain quality of instruction, student achievement, and student interest and motivation?

57 .

If you were Ellen, how might you approach this study?

3. Identify the quantitative data sources you would use to address this study. What research questions would you answer and how would you analyze these data?

• Standardized test scores, classroom based tests, interest and motivation questionnaires, and perhaps survey data collected from students, families, and administrators.

• Descriptive questions that could be answered from a survey address people’s opinion about the benefits of using the fetal pig dissection. Descriptive questions and analyses might also address motivation and interest. If students use this approach they should report means and standard deviations and other descriptive statistics as their method of analyses.

• Difference questions could address students’ achievement by standardized and classroom based test scores depending upon use of the fetal pig or an alternative approach. If students use this approach they should report the use of chi-square, t-test, or ANOVA types of analyses depending on their data source and number of levels in their planned study.

4. Identify the qualitative data sources you would use to address this study. What guiding hypothesis would you use and whom would you select as participants? Why would you select these participants?

• The responses should support their inclusion of particular students, parents, administrators, or potential others (e.g., former students, local animal rights group members, the high school advanced placement teachers; university science professors). It may be that one might want to select those who are opposed to the unit and those who feel it is critical. An alternative would be to question former students and key stakeholders that have these students in the future to determine how important the unit is perceived to be for their later success and interest and motivation for science.

5. Classify your study design. Support your response.

• Students’ responses will vary for this study depending upon their questions and their general approach. It is a good time to discuss designing mixed methods studies. Some will focus on achievement data and will likely use an explanatory sequential design. Some will focus on the beliefs of the parents and other and use an exploratory sequential design. Others (usually a smaller number of students) will have a two- part focus that is well supported by the convergent parallel design.

4. Sage publishes the Journal of Mixed Methods Research that can serve as an excellent resource for your course as you cover Chapter 16 content. The website can provide you with links to a free issue if your library does not subscribe. If your students to have access to the journal, the editorials and articles can extend Chapter 16 content, as they often discuss methodological issues of mixed methods. The journal is broad and includes studies from many areas of research that your students will find interesting. One article published by Hart and colleagues (2009) and referenced below provides a review and retrospective of ten years of mixed methods in mathematics education. It is a nice article to use because it is a mixed method study of mixed methods. Guided questions that can stimulate discussion are provided.

Hart, L.C., Smith, S. Z., Swars, S.L., & Smith, M.E. (2009). An examination of research methods in mathematics education. Journal of Mixed Methods Research, 3, 26-41.

58 .

1. How does this article compare to other articles that you have read in research journals? Note two observations.

2. The authors provide several definitions of mixed methods research. Which are similar to the stance taken by your text? Are there any that seem inconsistent with the perspective shared by your text?

3. Focus specifically on the second research purpose shared by the authors.

In what ways are quantitative and qualitative methods balanced and/or integrated in mixed methods articles in a case study of one of the examined journals?

As your text notes the integration and weight of mixed methods data is critical in mixed methods designs. What do the authors conclude regarding this research focus? Do their findings surprise you? Why or why not?

4. Having read this article, what is one ‘take home’ message you received from the authors regarding the use of mixed methods in education?

Possible responses to the above questions: 1. How does this article compare to other articles that you have read in research journals? Note two observations.

Answers will vary but students will likely note that the article is writing in the first person with a narrative approach. Students will also likely be surprised by the historical content included in the article regarding the field and the researchers as they tackled this project.

2. The authors provide several definitions of mixed methods research. Which are similar to the stance taken by your text? Are there any that seem inconsistent with the perspective shared by your text?

The nuances of definition are shared in this article, are the methods marginally mixed? Are they synergetic, are they discrete? Does the nature of mixed methods vary over time? The classic definitions as also supported in the text are present but deeper reflection on practice is provided.

3. Focus specifically on the second research purpose shared by the authors.

In what ways are quantitative and qualitative methods balanced and/or integrated in mixed methods articles in a case study of one of the examined journals?

As your text notes the integration and weight of mixed methods data is critical in mixed methods designs. What do the authors conclude regarding this research focus? Do their findings surprise you? Why or why not?

Conclusions are shared by both types of data. The tables illustrate article breakdown by method and illustrate the application of each of the mixed methods (and more) as discussed in the text.

The authors are very clear about trustworthiness which is important to discuss in support of Chapter 16 and also Chapter 21 content. The authors note of the challenges they faced in coding.

They address the issue of quantifying qualitative data which is a really important discussion point in qualitative and mixed methods research that may supplement the text content.

59 .

4. Having read this article, what is one ‘take home’ message you received from the authors regarding the use of mixed methods in education?

The state of the research literature did not change that much over the 10 year time frame. Mixed methods research is challenging to conduct. The role of each form of data and the synergy among data sources is important.

60 .

Chapter 17: Action Research Activities embedded in the etext MyLab Education Application Exercise 17.1: Evaluating Research Articles: Evaluating an Action Research Project Part 1 MyLab Education Application Exercise 17.2: Evaluating Research Articles: Evaluating an Action Research Project Part 2 MyLab Education Application Exercise 17.3: Explaining the Types of Action Research MyLab Education Application Exercise 17.4: Evaluating Research Articles: Identifying Steps in the Action Research Project MyLab Education Application Exercise 17.5: Planning an Action Research Study MyLab Education Self-Checks for Each Learning Objective Suggested Activities Chapter 17 presents Action Research. Task 8 requires students to develop a design for an action research study. Students are generally very interested in action research because the application of the research process is clear.

1. The journal Action Research is an international peer-reviewed quarterly journal that publishes articles about action research as well as action research studies. The journal was established in the early 2000s and for the last few years has been publishing reviews of action research that cover two-year time spans. These reviews may be particularly relevant for students who are considering action research studies. The scope of the represented work is quite broad. The citation for one of these reviews is found below. In this article, Dick overviews some of the most prominent publications in the form of edited volumes and other contributions. Have students read this article and then access one of the sources mentioned. Each student can return with a brief summary of the source they accessed. The summary should include: a. Full Citation b. Type of source (e.g., handbook, textbook, journal publication) c. Focus of the source (e.g., theoretical, philosophical, epistemological, methodological, research report) d. A minimum of two connections between the source and the outcomes or content of text Chapter 16.

If submitted electronically, taken together these summaries can provide a valuable resource for your students. You may elect to ask students in advance to identify their source to assure variance in the selected works. This activity would work well as a collaborative activity, in a traditional, blended, or online course.

Dick, B. (2010). Action research literature 2008-2010: Themes and trends. Action Research, 9, 122143.

61 .

2. Mark Smith has a nice entry for action research in the encyclopedia of informal education at infed.org (search for action research). Students can read the article prior to class coverage of Chapter 17 or alternatively, have students work in pairs to compare the content of the article to that presented in Chapter 17. Handout 17.1 can accompany this activity.

Potential answers to Handout 17.1 activity are found below (1-4).

------------------------------------------------------------------------------------------------------------In his article in the encyclopedia of information education (infed.org) Mark Smith presents an overview of action research.

1. In the article Smith presents two definitions of action research. How do each of these definitions compare to the position presented in your text?

Student answers will vary but at minimum should include that Smith presents two definitions: the first is action research as self-reflective inquiry employed by practitioners—reflective practice.

The second is broader and refers to collection of information that is designed to bring about change. The text definition includes systematic inquiry of practitioners collecting information about the way schools function. Some will respond regarding practical and critical action.

2. On page 2 of Smith’s article, he presents a spiral of steps in action research based on Lewin’s work. How does this sequence of steps correspond to the dialectic action research spiral presented in your text? Later in his article Smith includes basic phases of action research process (p.4). How do these three phases relate to the dialectic action research spiral?

Again, student answers will vary but should include that Lewin’s work (identifying an initial idea, fact finding, planning, taking action, evaluating, changing plans, and taking additional action), although portrayed as a cycle, presents somewhat of a sequential and step-by-step procedure. The dialectic action research spiral includes identifying an area of focus, collecting data, analyzing and interpreting data, and developing an action plan. The three phases presented in the Smith plan include looking, thinking, and acting. There are clear connections among these that learners will likely elaborate upon.

3. Both the chapter and the article present characteristics of action research based on Stringer’s work. List these characteristics and suggest how action research in a school might demonstrate these characteristics.

The characteristics, presented both in the article and the chapter include: Democratic, equitable, liberating, life enhancing. Students’ examples will vary widely as to how research in a school might demonstrate these characteristics. Students can be referred to Table 17.1 to assist them with examples.

4. Provide a critique that Smith notes regarding Lewin’s work. Do you feel that the approach discussed in your text addresses this critique? Why or Why not?

Student answers will likely be more lengthy and elaborate. In short, however, Smith critiques Lewin as being perceived to be too sequential and prescriptive. He then proceeds to illustrate a historical perspective on Lewin’s contributions and hedges a bit on the critique. The text emphasizes action research as more iterative than sequential and as such addresses this critique of action research.

62 .

Suggested Activities (continued) 3. This activity is based upon the case of Ellen found in Chapter 16. The description of the case is also found below. For Chapter 17 use the case to practice Task 9 criteria in pairs or small groups in class.

Ellen teaches the pull out gifted science program for intermediate grades in a moderately sized school district. Her program goal is to provide quality instructional experiences for exceptional science students. Targeted student outcomes for her program include not only achievement as measured by both classroom based and standardized tests, but also student interest in and motivation for science.

Two contributing factors have made Ellen question one of her instructional units, the fetal pig dissection project. First, while the district supports her program, her budget has come under scrutiny.

The fetal pig dissection is one of her more expensive units. Second, while some of the students cite this unit as their favorite, other students and their families have questioned the necessity of the dissection on moral grounds. She questions whether alternative instruction for the unit might be possible. Ellen decides to conduct a study to investigate this issue.

1. Write an area of focus statement.

2. Define the variables.

3. Develop the research questions.

4. Describe the intervention or innovation.

5. Describe the membership of the action research group.

6. Describe negotiations that need to be undertaken.

7. Develop a timeline.

8. Develop a statement of resources.

9. Develop data collection ideas.

Possible responses to the above questions: 1. Write an area of focus statement.

• Potential answers will greatly vary. A possible answer follows.

The purpose of the study is to investigate and describe students’ and families’ feelings and beliefs about the fetal pig dissection unit and to examine alternative instructional methodologies for teaching content found in the fetal pig dissection unit.

63 .

2. Define the variables.

• Depending upon the focus statement and whether students are using a strictly qualitative or quantitative approach or a mixed-methods approach the variables will vary. Some possible answers follow. student achievement, student interest, student motivation, parental beliefs, student beliefs, later science class enrollment, cost of alternative instructional methods 3. Develop the research questions.

• Again, student answers will vary regarding their approach. Representative research questions follow.

Are there differences in students’ interest for science and motivation for science between those enrolled in a fetal pig dissection and those not enrolled?

Are there differences in student achievement in science between those provided instruction with the fetal pig dissection unit and those provided an alternative approach (e.g., computer simulation)?

Are student and family beliefs regarding the use of the fetal pig dissection curriculum a factor in student interest and motivation for science?

What alternative instructional approaches might be viable alternatives to the fetal pig dissection unit?

4. Describe the intervention or innovation.

• Answers to this question will again vary but might include alternative approaches to instruction. Answers might also discuss how the researcher will approach the investigation.

5. Describe the membership of the action research group.

• In the scenario it seems as if Ellen is the action researcher. She might elect to include others such as other science teachers, science teachers her students will take in the future, or even a parent in this particular study.

6. Describe negotiations that need to be undertaken.

• Depending upon district policy, one may need several permissions or may not need any. It would be recommended that the action researcher conducting this study at least notify an administrator (e.g., science coordinator, or principal), if only to give him or her a heads-up in case parents or others question any surveys or interview procedures. If one approaches the study by comparing alternative instructional approaches, the researcher should be sure that students are given comparable instruction.

7. Develop a timeline.

• This study could take a few weeks or the academic year, or two, depending upon design and research approach. Students will likely underestimate the time it will take to complete the study.

8. Develop a statement of resources.

• Depending upon who is included as researchers and if there is the possible of administration or PTO assistance, answers to this question will vary. Some resources will include other curricular materials as well. It is beneficial to have students list what they think they would need to complete their study and then determine where they might get those materials or resources.

9. Develop data collection ideas.

64 .

•

Usually, when students get to this question they have a pretty solid idea of the types of data they would like to collect. It is beneficial to have them list procedurally how they would collect each type of data and how long it would take.

4. As a follow-up to Activity 3, group students by major or current position and have them develop a research topic and hypothetical action research plan on a topic of their choosing. Have them work through the hypothetical plan using the Task 9 criteria as found in the text and above. In addition, have each group identify 2 recent empirical studies that can inform their action research proposal.

This activity is best done over two class periods or one longer session. If completed over two sessions, have students get into groups, and identify a research topic (approximately 15 minutes). As an out of class assignment, have students work through the task 9 criteria, drafting ideas, and have them identify and bring to class example references. At the second session students will need at least 30 minutes to finish the assignment as a group –discussing and compromising on their group plan. If the activity is done in one session it will likely take approximately 70 minutes and students will need to have access to electronic databases and a means to save the articles they find to supplement their plan.

5. The American Educational Research Association (AERA) has a special interest group (SIG) dedicated to action research. Contact information and resources are found on the web at the AERA website (aera.org). The website includes references to recent action research publications, the SIG newsletter, and links to other action research websites and e-journals. Your students who are interested in action research will find it a great resource and may even want to join the SIG!

65 .

Chapter 18: Descriptive Statistics Activities embedded in the etext MyLab Education Application Exercise 18.1: Preparing Quantitative Data for Analysis MyLab Education Application Exercise 18.2: Evaluating Research Articles: Understanding Descriptive Statistics MyLab Education Application Exercise 18.3: Conducting Data Analysis MyLab Education Application Exercise 18.4: Graphing with R MyLab Education Self-Checks for Each Learning Objective Activities and Resources To gain competency in the content presented in Chapters 18 and 19, students need to have practice analyzing data. One way to accomplish this is to use existing datasets available online as referenced in the Chapter 18 and Chapter 19 activities.

Before beginning the activity below or another analysis, it is a good idea to describe how to open SPSS, and how to enter data. Alternatively, you could assign reading some introductory information. Many will not know how to use a database or how to navigate SPSS or an alternative statistics package. Pairing less experienced with more experienced peers works but be careful. It is easy for an able student to move through these exercises quickly and leave the less able peer in the dust, knowing less than when he started. If your class is very able you could have them complete the assignment out of class. For those students who might have their own research data, consider allowing them to use their own data and work though parallel questions.

The best experience comes from students collecting the data and then entering it themselves. Any potential survey would work but included in Activity 2 below is a draft of a few open-ended questions and companion activities.

The datasets used for the activities are of relatively small size. Therefore, consistent with your text’s suggestions, analyses can be conducted in Excel, SPSS student or nonstudent versions, or any number of other statistical packages.

Suggested Activity 1. The dummy dataset below includes the following variables for 40 participants: ID number, age, gender, hours spent playing video games in a given week (Hours), hours spent on exercise in a given week (Exercise), and the number of nights in a given week that the participants’ families ate dinner together. As a review, ask students to generate two research questions and corresponding hypotheses for these variables and then share the following activity.

18.1 Doug and David are researchers who examine youth health. Numerous recent studies have suggested that youth in the 10-15 year old range are increasingly engaged in many hours of video game playing, they are not active enough (as defined by at least one hour of exercise per day), and they are disconnected to their family units. However, in some of their own previous 66 .

research Doug and David found that although students play a lot of video games they remain active and engage in sufficient exercise. In their current study they asked children to keep a log and then to report for a given week the number of hours of video game playing and exercise that they engage in and also to report the number of dinners that their families eat together as a measure of family connection. They also collected gender and age as demographic data.

ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

AGE 12 13 12 11 11 12 13 12 12 12 12 14 13 13 12 12 11 14 14 12 12 12 13 11 12 12 14 12 12 13 13 14 11 12 12 13 12 12 13 11

Gender 1 1 2 1 1 2 1 2 1 2 1 1 2 1 2 1 1 1 2 2 2 2 2 1 1 2 1 2 1 2 1 2 1 1 1 1 2 1 2 2

Hours 3 14 12 6 2.5 15 14 16 6 8 8 5 13 12 8.5 7.75 8.25 9 12 16.5 5.75 9.75 12.5 11.25 8.25 24 1.5 4 2.75 14 2.5 9.25 8.75 11.5 12.5 8.25 7.5 5.75 6.25 6.5

Exercise 14 10 2 7 8.5 6 5 6 8 9 12 14 12 3 7 6 5.5 8 7 11.25 10 4 3 4 4 0 14.5 12 12 2.5 13 10 8 7.75 6.5 7 12.25 14 11.5 10.5

67 .

Dinner 5 3 2 3 3 2 1 4 3 3 3 3 3 2 2 4 6 2 1 3 4 5 1 3 1 2 3 1 3 4 7 2 3 3 3 2 2 4 1 2

1. Generate one descriptive research question that can be addressed by Doug and David’s data.

2. Generate one relationship research question that can be addressed by these data.

3. State one null hypothesis that can be tested with these data.

4. Identify the level of measurement for each of the following variables: Age (reported in years), Gender (reported as Female=1; Male=2), Hours of video game playing in the week (reported to the quarter hour), Exercise (reported to the quarter hour), and Dinners taken together as a family (0-7 dinners in the week).

5. Please enter the data into a file after double-checking for accuracy.

6. Please provide descriptive information for all of the variables. Which is the best measure of central tendency for each variable? Why? 7. Are there more girls or boys in the sample? On average, do youth in the sample play more time playing video games or exercising?

8. Graph the dinner (pie chart) and gender (histogram) data. Graph age, hours of game playing, and hours of exercise data with frequency distributions.

Handout 18.1 (answers) 1. Generate one descriptive research question that can be addressed by Doug and David’s data.

Answers to this question will vary but examples include: How many boys are there in the sample? What is the average number of hours spent playing video games? What is the average age of participants? What is the mode number dinners participants in the sample had as a family?

2. Generate one relationship research question that can be addressed by these data.

Answers to this question will vary but examples include: Is there a relationship between hours playing video games and hours exercising? Is there a relationship between age and number of dinners had together as a family?

3. State one null hypothesis that can be tested with these data.

Answers to this question will vary but examples include: There is no difference in the number of hours spent exercising between boys and girls.

4. Identify the level of measurement for each of the following variables: Age (reported in years), Gender (reported as Female=1; Male=2), Hours of video game playing in the week (reported to the quarter hour), Exercise (reported to the quarter hour), Dinners taken together as a family (0-7 dinners in the week). Age: Arguably Interval, Gender: Nominal, Hours of game playing and hours of exercise in the week: Ratio; Dinners taken together is pretty tricky. One might make a case for interval.

5. Please enter the data into a file after double-checking for accuracy.

6. Please provide descriptive information for all of the variables. Which is the best measure of central tendency for each variable? Why? As categorical data, Gender should be reported by 68 .

total number in the sample of each and mode as central tendency. The median is an appropriate measure of central tendency for several of the other variables. Generally, I would suggest that all three measures of central tendency be reported for the other variables in this small sample. Since the sample is small and not normally distributed on several of the variables, descriptive data will be very important. I would graph the data to note the underlying distributions. At least some of the descriptives are provided below for each variable.

f=1; m=2 Frequency Valid

Percent

Valid Percent

Cumulative Percent

1

22

55.0

55.0

55.0

2

18

45.0

45.0

100.0

Total

40

100.0

100.0

Exercise Frequency Valid

Percent

Valid Percent

Cumulative Percent

0

1

2.5

2.5

2.5

2

1

2.5

2.5

5.0

2.5

1

2.5

2.5

7.5

3

2

5.0

5.0

12.5

4

3

7.5

7.5

20.0

5

1

2.5

2.5

22.5

5.5

1

2.5

2.5

25.0

6

3

7.5

7.5

32.5

6.5

1

2.5

2.5

35.0

7

4

10.0

10.0

45.0

7.75

1

2.5

2.5

47.5

8

3

7.5

7.5

55.0

8.5

1

2.5

2.5

57.5

9

1

2.5

2.5

60.0

3

7.5

7.5

67.5

10

69 .

10.5

1

2.5

2.5

70.0

11.25

1

2.5

2.5

72.5

11.5

1

2.5

2.5

75.0

12

4

10.0

10.0

85.0

12.25

1

2.5

2.5

87.5

13

1

2.5

2.5

90.0

14

3

7.5

7.5

97.5

14.5

1

2.5

2.5

100.0

100.0

100.0

100.0

Std. Deviation

Std. Error Mean

Total

Group Statistics f=1; m=2 Exercise

Hours

N

Mean

1

22

8.7159

3.65661

.77959

2

18

7.5556

3.95429

.93204

1

22

7.6591

3.82780

.81609

2

18

11.1389

4.90290

1.15562

Dinner Frequency Valid

Percent

Valid Percent

Cumulative Percent

1

6

15.0

15.0

15.0

2

10

25.0

25.0

40.0

3

15

37.5

37.5

77.5

4

5

12.5

12.5

90.0

5

2

5.0

5.0

95.0

6

1

2.5

2.5

97.5

7

1

2.5

2.5

100.0

Total

40

100.0

100.0

70 .

7. Are there more girls or boys in the sample? On average, do youth in the sample play more time playing video games or exercising? There are more females. The youth spend slightly more time playing video games overall. However, by age 14 they spend more hours exercising on average.

8. Graph the dinner (pie chart) and gender (histogram) data. Graph age, hours of game playing, and hours of exercise data with frequency distributions. (Examples of some follow)

71 .

72 .

2. Have students work in pairs. As a caution, the activity takes quite a bit of time. Schedule a computer lab if you do not have access to computers in the classroom setting. A dataset (complete with problems and questions) for the students to enter is presented below. Students will make mistakes, such as entering height as 64 for 6 foot 4 inches. They will enter nominal data as ratio and will elect to or not elect to throw ‘errors’ in the data out. These choices are important for them to make and provide the instructor opportunities to illustrate data management and data cleaning strategies.

The questions included with the activity require learners to run all descriptive statistics and graph the data. Take time as the instructor to illustrate the importance of visual analysis and descriptive statistics to support content from Chapter 18. (Activity answers appear following the dataset.)

Have the students complete the following tasks: 1. Please enter your data into a data file and turn in a data description and code sheet.

2. Please graph the descriptive information for each of the variables. You must use at least 2 different types of graphs or visual displays.

3. Turn in a copy of the graphs and the data file.

73 .

4. What is the average age of the class?

5. What is the average number of hours of exercise per week?

6. Was HS GPA or College GPA higher?

7. How many children do the class members have total?

8. How many members of our class are single children?

9. Provide printouts of graphed data for gender and martial status.

Tool to collect data. Have each student collect three cases.

Gender________________

College GPA______________

Age___________________

Number of siblings____________

Height_________________

Marital status_______________

Weight________________

Number of children __________

High school GPA ________________

Exercise per week (hr) ____________

Example dataset Gen

age

ht

wt

male female female f F m m F M F M F F F F Male female male female female

21 6’1’’ 190 21 5’5” 135 18 5.51/2 135 24 5’4’ 210 48 5’7’ 117 52 5’8’ 145 48 6FT 190 48 5’2’ 200 40 6’4’ 185 22 5’8 125 26 6’0’ 175 24 5’4 125 5’61/2 150 3.9 31 5’71/2 135 23 5’1 130 48 6’1’ 210 38 5’3’ 127 23 5’11 180 39 5’6 145 29 5’6 145

HSgpa Cgpa 2.4 3.2 3.5 3.96 2.9 3.6 3.7 3.2 3.1 3.0 3.5 3.4 3.0 3.5 2.0 3.5 3.75 3.554 3.0

2.0 3.0 2.5 2.73/3.11 2.9 3.3/4.0 3.8 3.7 3.5 3.2 2.8 3.7 3.7 3.3 2.5 3.65 3.78 n/a 3.0 74

.

siblings

marital

4 3 2 5 3 1 2 4 0 2 1 0 3 1 1 1 1 6 1 2

single single single s M W D D M S S S Married Single single married divorced single married married

children exercise none 10 hour none 3 none 5 0 0 2 3 2 15 2 6 1 0 3 2 0 1-2hr 0 6 0 1 4 4hours 0 3 hours 0 3 hours 2 9 hours 3 0 hours 0 0 1 3 hours 5 hours

female male female female female female f M F M Male Female male F F male male female female female female F F M Male Female male male male female F M F Female Female female male male female F M M female male female Female Female

32 25 48 31 25 56 26 25 24 29 48 30 23 22 2.5 34 37 36 23 31 38 37 21 20 26 24 26 20 22 20 30 49 44 36 30 31 22 26 32 36 40 65 31 35 28 31 27

5’9 160 3.0 5’9 210 2.8 5’6 142 3.5 5’6 135# 3.95 5’3 140 3.76 5’2 136 ?/3.7 5’3 133 3.5 5’11 190 3.8 5’2 140 3.7 5ft11in 225lbs 3.46 5ft10in 215lbs 2.13 5ft9in 250lbs 3.25 6’ 200lbs 3.94 5’3 140 3.35 5’2 160 3.2 6’2 150 3.2 5’11 175 3.4 5’8 165 2.6 5’5 126 2.3 5’3 134 2.7 5’1 202 5.2 5’4 135 3.76 5’5 125 3.4 5’11 160 2.9 5’11 175 2.5 5’9’ 166 3.5 6’1 235 4.0 6’00 165 2.7 5-7 135 2.6 5’0 110 3.0 5’3 115 3.85 6’0’ 250 2.5 5’8 150 3.5 5’5 120lbs 3.6 5’4.5” 185lbs 3.2 5’7” 145lbs 4.0 f’11 170 3.3 5’10 150 3.8 5’5 100 2.8 5’6 265 2.2 5’10 260 2.9 5’9 175 3.5 5’6 128 6’5 225 3.5 5’4 120 3.8 5’1 90lbs 2.8 55 110 3.0

3.0 2.3 4.0 3.82 3.82 3.5 3.2 3.6 3.45 3.80 2.25 3.46 4.0 3.25 2.9 2.97 3.0 3.9 3.1 3.4 3.89 4.0 3.3 3.1 2.2 3.3 3.7 2.5 3.0 3.68 3.875 3.0 3.65 3.6 3.0 3.75 2.8 3.2 3.8 NA 3.9 3.4 3.9 2.92 3.75 4.0 3.0

75 .

3 married 0 single 8 married 2 married 4 married 4 M 2 M 3 S 2 S 4 single 1 divorced 2 divorced 1 S 1 S 1 S 2 S 2 M 2 M 0 single 1 single 8 single 1 S 1 S 2 S 6 married 1 NA 3 NA 2 single 0 single 1 single 0 single 1 married 3 married 1 M 1 M 1 M 3 single 4 single 0 single 3 married 4 married 3 married 5 0 single 2 marr 2 married 2 single 1and1/2 divorced

1 0 2 1 1 2 1 0 0 0 0 0 0 0 0 1 2 3 1 0 1 0 0 0 2 NA NA 0 0 0 0 2 2 1 1 0 none 0 0 4 6 5 0 2 1 0 2

0 10hrs 3 hrs 5 3 3 3 7 10.5 2 1 3 3 1 2 1 1/2hr 1/2hr 0 3 0 4 6 5 21 NA 14 1 10 6 0 6 0 1 3 3 10hrs 10 hrs 0 2 1 31/2hrs 15hrs 3-4hrs 0 0

Chapter 18 Activity 2 Suggested Answers The included dataset allows for opportunities to discuss aspects of instrument design, data entry, and analysis concerns. The instructor can either discuss up front things to think about as a class or one can let them figure it out and discuss it along the way.

Please note that the answers found for the Chapter 18 questions will differ depending upon data entry decisions made by the student. If you are having students enter the data have them follow the following steps: •

First, give each case a sequential number and write it on the hardcopy. This will allow you to discuss them by case number if needed.

•

Second, open SPSS, select “Type in new data”.

•

Third, create variable names.

•

Fourth, change gender and marital status to nominal level data and identify which number represents each level.

•

Fifth, enter the data. I would encourage them to enter the data by case not vertically as that is more consistent with general practice. Indicate to the students that they should do. For example one can read the other enter. Encourage them to THINK before they start entering.

•

Six, double check and then examine through descriptives

Decisions and notes about the data from which the answers are provided Female is 1, Male is 2 Marital status is: single is 1, married is 2, divorced is 3, widowed is 4, N/A is dropped Cases with more than one GPA—the first was used f’11 was entered as 5’11 1-2 hours was represented by 1.5 1/2’s were systematically entered and included as 1/2’s The half-sibling was ignored.

N/A was entered as missing GPA was entered to 2 decimals 1. Please enter your data into a data file and turn in a data description and code sheet.

For this problem, students should be aware about coding data from the chapter. Stepping through coding and requiring students to have notes for the codes that they used is good practice. The codes should be clear for them to go back to. They should also include any decisions that they had to make regarding entry, noted by case number.

2. Please graph the descriptive information for each of the variables. You must use at least 2 different types of graphs.

76 .

This might not be turned in but it is good practice to run through them. Suggest histograms and pie charts for the nominal data. Line graphs and area for the other data. Spend some time discussing the role of visual analysis.

Some of these analyses are included in the answers and posted below.

The descriptives should include measures of central tendency (appropriate for the data level), measures of dispersion, and measures of relative position to support chapter 13. As a minimum, request that students have modal information and range and sum for the nominal. Median, mean, variance, standard deviation, and percentile information for the other variables. Encourage students to request contingency tables just so they see what they look like. Tables with all answers are presented. One frequency distribution, that of height, is provided.

Make note of the missing data cases with the students.

3. Turn in a copy of the graphs and the data file.

This is included as an opportunity for student s to save their work. They are usually so into it that they haven’t saved anything.

4. What is the average age of the class?

Mean = 31.78 years (from the descriptive table) 5. What is the average number of hours of exercise per week?

4.04 hours (from the descriptive table) 6. Was HS GPA or College GPA higher?

College, just a little, 3.28 versus 3.31 (from the descriptive table) 7. How many children do the class members have total?

65 (use the sum function) 8. How many members of our class are single children?

8 (found through a frequency distribution) 9. Provide printouts of graphed data for gender and martial status.

Example graphs are provided in the answers.

77 .

N

Valid Missing

Mode Range Sum

GENDER 68 0 1.00 1.00 94.00

MARITAL 66 2 1.00 3.00 107.00

Statistics

GENDER

Valid

1.00 2.00 Total

Frequency

Percent

42 26 68

61.8 38.2 100.0

Frequency

Percent

33 26 6 1 66 2 68

48.5 38.2 8.8 1.5 97.1 2.9 100.0

Valid Cumulative Percent Percent 61.8 61.8 38.2 100.0 100.0

MARITAL

Valid

Missing Total

1.00 2.00 3.00 4.00 Total System

Descriptive Statistics N Minimum Maximum AGE HEIGHT WEIGHT HS_GPA C_GPA SIBLINGS CHILDREN

67 68 68 66 66 68 66

18.00 60.00 90.00 2.00 2.00 .00 .00

65.00 77.00 265.00 5.20 4.00 8.00 6.00

EXERCISE Valid N (listwise)

67 61

.00

21.00

Valid Percent 50.0 39.4 9.1 1.5 100.0

Mean

Std. Variance Deviation 31.7761 10.1680 103.389 67.3235 3.9998 15.998 162.5000 40.6494 1652.373 3.2850 .5549 .308 3.3053 .5034 .253 2.1618 1.7416 3.033 .9848 1.3416 1.800 4.0448

78 .

Cumulative Percent 50.0 89.4 98.5 100.0

4.3421

18.854

Statistics

N

Valid Missing

AGE 67 1 31.7761 30.0000 31.00 10.1680

HEIGHT WEIGHT 68 68 0 0 67.3235 162.5000 67.0000 150.0000 71.00 135.00 3.9998 40.6494

Mean Median Mode Std. Deviation Variance 103.3885 15.9982 1652.3731 Range 47.00 17.00 175.00 Sum 2129.00 4578.00 11050.00 a Multiple modes exist. The smallest value is shown

HS_GPA 66 2 3.2850 3.4000 3.50 .5549 .3079 3.20 216.81

C_GPA SIBLINGS CHILDREN EXERCISE 66 68 66 67 2 0 2 1 3.3053 2.1618 .9848 4.0448 3.3500 2.0000 .0000 3.0000 3.00 1.00 .00 3.00 .5034 1.7416 1.3416 4.3421 .2534 2.00 218.15

HEIGHT

Valid

60.00 61.00 62.00 63.00 64.00 64.50 65.00 65.50 66.00 66.50 67.00 67.50 68.00 69.00 70.00 71.00 72.00 73.00 74.00 76.00 77.00 Total

Frequency

Percent

1 3 4 6 4 1 6 1 6 1 3 1 4 5 3 7 6 3 1 1 1 68

1.5 4.4 5.9 8.8 5.9 1.5 8.8 1.5 8.8 1.5 4.4 1.5 5.9 7.4 4.4 10.3 8.8 4.4 1.5 1.5 1.5 100.0

Valid Percent 1.5 4.4 5.9 8.8 5.9 1.5 8.8 1.5 8.8 1.5 4.4 1.5 5.9 7.4 4.4 10.3 8.8 4.4 1.5 1.5 1.5 100.0

79 .

Cumulative Percent 1.5 5.9 11.8 20.6 26.5 27.9 36.8 38.2 47.1 48.5 52.9 54.4 60.3 67.6 72.1 82.4 91.2 95.6 97.1 98.5 100.0

3.0331 8.00 147.00

1.7998 6.00 65.00

18.8540 21.00 271.00

80 .

81 .

82 .

83 .

84 .

a.

3. New to this edition is the introduction of the statistical software R. R may be intimidating to some students at first, but by showing students the basics they may come to enjoy working with R. Examples are written in text for R program so that students do not have to flip back and forth between the chapter and the exercises. Still, consider helping your students download R onto their computers. Simply go to https://cran.r-project.org and have students select the format appropriate for their device.

4. Once students have downloaded R, consider walking them through a very basic example with the intent of learning the program, not statistical techniques. In order to do that, you may use the following code. This code will create a dataset of 10 numbers and allow you to find some basic descriptive statistics for the dataset. The dataset is small enough that students could calculate these statistics by hand to confirm their R output.

85 .

example <- c(100, 100, 98, 34, 96, 92, 80, 76, 65, 100) example mean(example) median(example) mode(example) range(example) sd(example) Your results should look like this:

86 .

Chapter 19: Inferential Statistics Activities embedded in the etext MyLab Education Application Exercise 19.1: Testing a Hypothesis MyLab Education Application Exercise 19.2: Identifying the Appropriate Test of Significance MyLab Education Application Exercise 19.3: Evaluating Research Articles: Understanding Inferential Statistics

Suggested Activities 1. In addition to capabilities in Excel and SPSS, there are online free data analysis programs available in addition to R, such as Simple Interactive Statistical Analysis (SISA; quantitativeskills.com/sisa/).

Through SISA students can conduct basic analyses and power estimates, Bonferoni adjustments, and many other important tools as well as houses a random number generator. It is a fantastic resource for students to know about.

2. Use the dataset from Chapter 18, Activity 1 to have your students answer the following questions.

1. What is the correlation between hours of game playing and exercise?

2. Is there a significant difference between boys and girls in amount of video time playing?

3. Is there a significant difference between boys and girls in amount of exercise time?

4. At what age in the sample do boys report the most video use?

3. Use the dataset from the Chapter 18, Activity 2 to have your students answer the following questions.

Again, depending on how they entered the data there might be slight differences in their responses.

1. What is the correlation between high school GPA and college GPA? 2. What is the correlation between age and weight? Graph this correlation with a scatterplot.

3. What is the relationship between height and weight?

4. Do males or females exercise more? Significantly more?

87 .

Chapter 19 Suggested Activity 2 Answers 1. Use the dataset from Chapter 18, Activity 1 to have your students answer the following questions.

1. What is the correlation between hours of game playing and exercise?

There is a -.646 correlation between hours of game playing and hours of exercise.

Correlations Hours Hours

Pearson Correlation

Exercise -.646**

1.000

Sig. (2-tailed)

Exercise

.000

N

40.000

40

Pearson Correlation

-.646**

1.000

Sig. (2-tailed)

.000

N

40

40.000

**. Correlation is significant at the 0.01 level (2-tailed).

2. Is there a significant difference between boys and girls in amount of video time playing?

See table after number 3 below: Yes, boys play more time than girls.

3. Is there a significant difference between boys and girls in amount of exercise time?

There is not a significant difference in exercise in this sample between boys and girls.

Group Statistics f=1; m=2 Exercise

Hours

N

Mean

Std. Deviation

Std. Error Mean

1

22

8.7159

3.65661

.77959

2

18

7.5556

3.95429

.93204

1

22

7.6591

3.82780

.81609

2

18

11.1389

4.90290

1.15562

88 .

Independent Samples

Levene's Test for Equality of Variances

t-test for Equality of Means 95% Confidence Interval of the Difference

F Exercise

Equal variances assumed

Sig.

.288 .594

Equal variances not assumed Hours

Equal variances assumed

.950 .336

Equal variances not assumed

t

Sig. (2Mean Std. Error tailed) Difference Difference Lower

df 38

.342

1.16035

1.20539

1.27983

3.60054

.955 35.172

.346

1.16035

1.21509

1.30599

3.62670

38

.016

-3.47980

1.37991

6.27329

-.68631

-2.460 31.785

.020

-3.47980

1.41473

6.36228

-.59732

.963

-2.522

4. At what age in the sample do youth report the most video use?

The answer is age 13.

89 .

Upper

Case Processing Summary Cases Included N

Excluded

Percent

N

Total

Percent

N

Percent

Hours * AGE

40

100.0%

0

.0%

40

100.0%

Exercise * AGE

40

100.0%

0

.0%

40

100.0%

Report AGE 11

Hours Mean

7.2083

7.2500

6

6

Std. Deviation

2.96402

2.29674

Mean

9.6250

8.1875

20

20

Std. Deviation

5.24812

3.97515

Mean

10.7222

7.4444

9

9

Std. Deviation

4.12205

4.25327

Mean

7.3500

10.7000

5

5

Std. Deviation

4.11400

3.42053

Mean

9.2250

8.1938

40

40

4.63051

3.78911

N

12

N

13

N

14

N

Total

Exercise

N Std. Deviation

Chapter 19 Suggested Activity 3 Answers Please note that the answers found for the Chapter 19 questions will differ slightly depending upon data entry decisions made by the student. While the answers to the questions included in the suggested activities are provided, this dataset also allows for several other analysis, and can also be used to familiarize students with the functions of SPSS. As the instructor you might want to pose additional questions and allow for some time for the students to experiment and practice.

90 .

1. What is the correlation between HS GPA and College GPA? Answer r = .534 A Pearson r is appropriate to calculate the correlation as noted in Chapter 18. The analysis table is provided below.

As part of the exercise you might want students to calculate a Spearman rho correlation for some of the data. The correlation between marital status and gender is not significant. The table is provided below.

2. What is the correlation between age and weight? Graph this correlation with a scatterplot.

A Pearson r is appropriate to calculate the correlation as noted in Chapter 19. The analysis table is provided below. Answer: r = .20, The scatterplot is also provided below.

3. What is the relationship between height and weight?

Answer: r = .559 A scatter plot is also provided below.

4. Do males or females exercise more? Significantly more?

Males, yes significantly more. The means table and the t-test table are provided below.

This would be a good time to remind students of assumptions of statistical tests.

Correlations HS_GPA C_GPA Pearson 1.000 .534 Correlation Sig. (2-tailed) . .000 N 66 64 C_GPA Pearson .534 1.000 Correlation Sig. (2-tailed) .000 .

N 64 66 ** Correlation is significant at the 0.01 level (2-tailed).

HS_GPA

91 .

Correlations Spearman's GENDER Correlation rho Coefficient Sig. (2tailed) N MARITAL Correlation Coefficient Sig. (2tailed) N

GENDER MARITAL 1.000 -.013 .

.919

68 -.013

66 1.000

.919

.

66

66

Correlations AGE

Pearson Correlation Sig. (2-tailed) N WEIGHT Pearson Correlation Sig. (2-tailed) N

AGE WEIGHT 1.000 .200 .

67 .200

.104 67 1.000

.104 67

.

68

92 .

93 .

94 .

Correlations WEIGHT HEIGHT Pearson 1.000 .559 Correlation Sig. (2-tailed) . .000 N 68 68 HEIGHT Pearson .559 1.000 Correlation Sig. (2-tailed) .000 .

N 68 68 ** Correlation is significant at the 0.01 level (2-tailed).

WEIGHT

Group Statistics GENDER EXERCISE

1.00 2.00

N 41 26

Independent Samples Test Levene's Test for Equality of Variances F

EXERCI Equal SE variances assumed Equal variances not assumed

Mean

Std. Std. Error Deviation Mean 2.3293 2.2181 .3464 6.7500 5.4282 1.0646

t-test for Equality of Means Sig.

t

df

22.856 .000

-4.653

65

-3.94930.362

95 .

Sig. (2- Mean Std. 95% tailed) Differen Error Confidence ce Differen Interval of ce the Difference Lower Upper .000 -4.4207 .9500 -6.3181 -2.5234

.000 -4.4207

1.1195

-6.7059 -2.1356

Chapter 20: Qualitative Data Collection Activities embedded in the etext MyLab Education Application Exercise 20.1: Evaluating Research Articles: Collecting Data for a Qualitative Study.

MyLab Education Application Exercise 20.2: Collecting Qualitative Data MyLab Education Application Exercise 20.3: Designing Qualitative Data Collection for a Research Study MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. Send students ‘to the field’ in pairs. (This activity should be cleared with your IRB, but should not need approval since the data will not be used and students are practicing observations not interacting with potential participants.) Pairs should be in the field for approximately 10-15 minutes. Provide a common protocol for all of the students to use. An example might be one that includes the following selected aspects of the protocol found in Chapter 20, referenced as a simple protocol for observation.

Encourage students to ‘label’ their data with time, date, number, and notes as is found in Figure 20.2.

Simply, draw two columns to record descriptive notes and reflective notes.

• • •

Who is being observed? How many people are there? Who are they? What individual roles and mannerisms are evident?

What is going on? What are people saying or doing? How do the participants interact with each other? What beliefs, attitudes, values, etc are evident in the participants?

What activities or interactions seemed unusual or significant? What was the researcher doing during the session?

Students should be sent to places where they can observe without a lot of interruption from others regarding ‘what are you doing’ and where two extra people, taking notes, will not cause alarm. Do NOT allow students to audiotape or video record the exercise due to ethical protections. The exercise works better if the students do not know those they will be observing.

Places to send students to the field might include A shopping center An athletic event A public meeting (e.g., town council, school board, congregation) Student union Student sit-in Computer lab Library Student group meetings Coffee shop Political rally A carnival or fair

96 .

The activity works best when two students go to the same site but take field notes independently, so they can (and the class as a whole can) learn from each other’s notes and experiences. If you have students with varied skill, it might be appropriate for a more and less able student to go to the same site. Encourage students to use different colors of ink, or code the notes taken at the site and those added after the experience. Students should be encouraged to record while at the site, write (in a different color or with a code) additional notes directly after their fieldwork and again later or the next day (again in different color or with a different code). If the class is very large and sharing the notes aloud will be too time consuming, or if you think some students would be sensitive to sharing their less than ideal notes aloud, have pairs share their notes before class instead and have the students highlight the differences in field notes from two perspectives of the same situation.

Otherwise, have students share their field notes aloud (or portions of them) in class. Or, photocopy enough of each sample and distribute one to each class member. The field notes will likely be fairly short. However, through hearing several examples, students will gather some additional aspects of what can be effectively included in field notes.

Examples of your ‘field notes’ of a similar event or those of an experienced qualitative researcher are also really helpful to demonstrate description to the students. I have had an experienced qualitative researcher collect field notes at local musical events and share those with the class.

They provide a great example of amazing description.

2. Develop a mock interview between two of the students regarding a hot topic in the community or on campus. Recently at my institution there is some conflict between students and administration related to logos and companies producing materials. Be sure to select students who can facilitate the activity by talking and describing a lot, making stuff up ‘on the fly’ and by asking appropriate questions. You will want to provide a protocol or at the least some substantial coaching to the role-playing students.

It is ideal to provide both ‘good’ and ‘bad’ interview questions so the students can see what can happen when questions are not well crafted. Further, be sure to include both convergent and divergent questions so students can see how each ‘work’ in an interview.

Have all students take field notes of the interview. Preferably use the guidelines found in the text regarding field notes. These ‘field notes’ will be used to illustrate both interviewing notes and field notes as an activity. To encourage thick description, it is better to hold the mock interview somewhere other than where the class generally meets. Students will less likely record descriptive setting information if they feel the setting is a place that everyone is familiar with.

Compare the interview notes and develop a set of shared notes through class discussion. These notes, of course, will be exemplary because everyone will contribute. This exercise works well because all students share the same experience. Further, because several people contribute to the notes of the experience it illustrates both a positive example of effective field notes while at the same time illustrates how much would be missed without an audio recording if one person were to interview and take notes simultaneously.

3. Qualitative data sources that students generally think of are observations and interviews. Less thought is taken regarding documents and other sources. Further, students tend to believe that they will interview or observe specific individuals. Have students work in pairs or groups of three to consider one of the following examples what data sources might be available to them given the following topics, or one of your or their choosing, for description in a given district.

97 .

Handout 20.3 Working a small group or with a peer consider the following potential focal points for a qualitative research study to be conducted within a district.

School Climate Leadership opportunities Playground culture Physical Education as a context Consider the following questions: Who might be your informant(s)? Consider multiple levels of informants.

What settings might you observe? How often, how, and why?

Who might you interview? How often and why?

What other sources might be available as data (think deeply)? Consider documents including correspondence, policies, photos and other materials.

Are there any ‘things’ you would want to catalog or describe?

98 .

Chapter 21: Qualitative Research: Data Analysis and Interpretation Activities embedded in the etext MyLab Education Application Exercise 21.1: Analyzing and Interpreting Qualitative Data MyLab Education Application Exercise 21.2: Analyzing Qualitative Research Data MyLab Education Application Exercise 21.3: Evaluating Research Articles: Qualitative Data Analysis MyLab Education Application Exercise 21.4: Evaluating Research Articles: Qualitative Data Interpretation MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. One of the issues students generally question regarding qualitative research is how much researchers draw conclusions from their data and ‘what it takes’ to be able to make claims from their data. In a recent special issue of Educational Psychology Review, (June 2011), Jonna Kulikowich and Rayne Sperling tackled this question for data overall. An entry in that issue authored by Amanda Nolen and Tony Talbert specifically addressed qualitative assertions. The manuscript is accessible to even novice researchers and can be used for discussion in class. Have students read the article out of class and come prepared for discussion of the following questions.

Handout 21.1 Given the following article: Nolen, A., & Talbert, T. (2011). Qualitative assertions as prescriptive statements. Educational Psychology Review, 23, 263-271.

1. The authors suggest that there are necessary conditions that must be met prior to qualitative researchers stating prescriptive statements. Select one and provide a rationale for that condition.

2. How do the authors refer to data? How does this stance relate to your text?

3. One large focus of the manuscript is the role of the researcher. State and describe one characteristic or goal of qualitative researchers. Why is this characteristic or goal desirable or necessary?

99 .

4. In the end, what types of statements do the authors suggest qualitative researchers can make regarding claims from their evidence?

5. The text uses the term trustworthy research reports. How might the authors consider this term?

What to the authors makes a report trustworthy?

6. Share one surprising aspect of the manuscript.

7. Share one place where the manuscript extends or contradicts the text content regarding qualitative research.

Handout 21.1 (Discussion points) 1. The authors suggest that there are necessary conditions that must be met prior to qualitative researchers stating prescriptive statements. Select one and provide a rationale for that condition.

The authors note roles for both the reader and the researcher. Early in the manuscript (p. 264-265) the authors contend that the researcher must be fully transparent in his or her strategy of data analysis and reduction, the study must be derived from existing research literature and the relationship and roles of the researcher in relation to the researched is described.

I suspect these conditions would not be universally accepted by qualitative researchers with the second, the relationship of the study to the existing research base being one that might not be shared by all. The other two are clearly supported by content in the text. Transparency is clearly a theme of the first and third conditions. This is also a theme discussed in the text in each of the qualitative chapters. One discussion point from this might include how this stance relates to objectivity. The authors are careful to not advocate objectivity.

2. How do the authors refer to data? How does this stance relate to your text?

The authors use the term evidence instead of data. The text uses the term data. In fact, in a search of the text of the chapter evidence is not used at all in the chapter. How might this alter the approach to analysis suggested in Chapter 20? Could one simply replace data with evidence in the chapter and still follow the suggested strategies?

3. One large focus of the manuscript is the role of the researcher. State and describe one characteristic or goal of qualitative researchers. Why is this characteristic or goal desirable or necessary?

Several of the characteristics or goals for the researchers are consistent with discussion of the text.

However the authors discuss in some detail the need for theoretical and philosophical knowledge on the part of the qualitative researcher. Discuss the tension between this assertion and the downside of

100 .

too much knowledge. Although mentioned in the manuscript, this downside is discussed in more detail in the text.

4. In the end, what types of statements do the authors suggest qualitative researchers can make regarding claims from their evidence?

One point in the manuscript that bears additional discussion is the role of disconfirmatory evidence.

A discussion about how such evidence is incorporated into narratives would be worthwhile in class.

In addition, how qualitative research methods may circle to target disconfirmatory evidence is also worthwhile.

5. The text uses the term trustworthy research reports. How might the authors consider this term?

What to the authors makes a report trustworthy?

One large focus of the manuscript is the role of researcher transparency. Tie transparency to trustworthiness. Effective communication is a primary theme in the manuscript.

6. Share one surprising aspect of the manuscript.

I thought the discussion regarding the role of the reader very interesting and somewhat novel compared to the text content. This provides a slightly different lens to trustworthy reports.

7. Share one place where the manuscript extends or contradicts the text content regarding qualitative research.

Overall, contradictions are not directly present. However there are several places where the manuscript and the text emphasize different characteristics of qualitative research.

The main purpose of the manuscript really is about communication. This supports Chapter 20 content well.

2. As an introductory handout for practitioners who may or may not ever conduct research, a recent publication from the University of Wisconsin-Extension, Cooperative Extension office has received considerable publicity. The publication developed by Ellen Taylor-Powell and Marcus Renner (2003) presents a very brief ‘how to’ type of primer for analyzing qualitative data. It is very hands-on and was designed for those involved in program evaluation studies. The publication suggests or reviews different forms of qualitative data. In addition, the publication divides data analyses into a series of steps. The examples are explicit and the publication can easily be used as a class handout. One instructional strategy would be to compare the article suggestions and steps with those presented in Chapter 21, noting the similarities and differences. The publication can be ordered for $1.25 in a PDF format from the University of Wisconsin-Extension’s The Learning Store (learningstore.uwex.edu).

3. As the text notes, there are software programs available to assist with qualitative data analysis. For your own review, or for your students to read, consider two articles from the journal Social Science Computer Review. Both extend several important points made in the text regarding the appropriate roles for software in qualitative data analysis.

MacMilan, K., Koenig, T. (2004). The wow factor: Preconceptions and expectations for data analysis software in qualitative research. Social Science Computer Review, 22(2), 179-186.

101 .

Mangabeira, W. L, & Raymond, M., & Fielding, N. G. (2004). Computers and qualitative research: Adoption, use, and representation. Social Science Computer Review, 22(2), 167-178.

4. For related discussion to item 3 above and the text content, some articles address the use of more generic word processing programs to use for assistance when analyzing qualitative data. See the following two publications found in Field Methods in 2004. These articles present ‘low tech’ and small learning curve ways to use software to assist with data analyses.

La-Pelle, N. (2004). Simplifying qualitative data analysis using general purpose software tools.

Field Methods, 16, 85-108.

Ryan, G. W. (2004). Using a word processor to tag and retrieve blocks of text. Field Methods, 16, 109-130.

102 .

Chapter 22: Preparing a Research Report Activities embedded in the etext MyLab Education Application Exercise 22.1: Writing Research Reports Part 1 MyLab Education Application Exercise 22.2: Writing Research Reports Part 2 MyLab Education Application Exercise 22.3: Finding a Professional Journal MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. Journal publishers now provide exceptional support for authors. Taylor and Francis (tandf.co.uk), and Elsevier (elsevier.com) author instructions are excellent examples. These are an amazing resource for many emerging scholars. Assign students in pairs to explore these sites. Ask students to share three things they learned that they did not know about the publication process and also two things that they learned about writing. Of important note, journals often provide links to paid editorial services for second language learners. We have found that many potentially worthy manuscripts get rejected by peer review immediately because of English language use.

2. As noted in other chapters, APA and AERA and other organizations provide mentoring programs for students. Often these types of programs lead students to work with scholars who do research in their chosen area of interest. These mentoring programs often turn into great opportunities for students to receive constructive feedback on their writing. Encourage students to apply for these programs and also to take drafts of their work with them when they present at conferences. Often scholars are willing to help students by reading some of their work and providing feedback. It never hurts to ask (politely)!

3. The text describes three reference management software packages: RefWorks, Zotero, and EndNote.

It may be that your institution subscribes to one in particular. Often technology support personnel or librarians at your institution are willing (and excited) to demonstrate these programs for the students.

Students appreciate the hands-on demonstration. Should you not have this type of resource available, Endnote, for example, offers free trials and tutorials (endnote.com).

4. Show examples of your work in progress. Your notes, the IRB you submitted that described the study, the proposal or abstract that you wrote to present it at a research conference, your outline and every draft and notes—keep it all. Students, even doctoral students, seem to think that writing is a one-draft process. With the artifacts of a manuscript in progress you can illustrate otherwise. Share this manuscript in progress and ask them to provide feedback to you. This will allow you to model how to accept and use constructive criticism and peer review to make a manuscript better.

5. Similarly, many writers now use Google Drive to allow multiple authors to work simultaneously on a single document. Some caution, however, might be warranted because there have been questions about the use of these types of programs and copyright concerns. The help page (look for help on docs.google.com) provides a tutorial of the program.

103 .

6. Peer editing is often an excellent strategy to use with the class. If you have had the students do group projects and turn in an actual report, the groups can edit others’ writing. Discuss the peer-editing process that researchers go through before they submit work and emphasize its importance and examples of how your writing has been ‘hacked’ and how much better it has made your work to obtain valuable critiques. The use of an electronic discussion board where students can post parts of their papers to receive peer feedback is also a good strategy.

7. Some students have never considered the review and publication process. Providing exposure through the notes to authors found in several journals will introduce them to the expectations of journals regarding preparation of research for publication. Bring several to class and have students review them in small groups and discuss the criteria. Use the most typical journals for the disciplines represented by those in your class. Some general examples that should have broad appeal are the AERA journals (search for journals at aera.net). Examine several journals’ submission requirements: The American Educational Research Journal Educational Evaluation and Policy Analysis Review of Educational Research

104 .

Chapter 23: Evaluating a Research Report Activities embedded in the etext MyLab Education Application Exercise 23.1: Evaluating Research Articles: Evaluating a Published Study MyLab Education Self-Checks for Each Learning Objective Suggested Activities 1. For Chapter 23 content, group students by program of study and have each group read an article and critique it with the criteria provided in the text. Placing students who are from the same program works well with this exercise because students erroneously believe students in other programs are likely to possess relevant skills for review that they do not. Further, some students will be more inclined to select various types of research based upon the nature of their discipline’s work. After the critiques are complete, invite students to share them in class so students are exposed to more than one critique efficiently.

2. Share examples of manuscript reviews you have received and given. Students are not often privy to this information and it is quite helpful in showing them how the review process works. You can spend a little time talking about categories of review and feedback and describe the process. Students really like this activity, especially if you share painful reviews. One helpful experience for them is for you to include before and after manuscripts. This can also provide you the opportunity to illustrate appropriate documentation of revisions and modifications you have made that illustrate how you have modified the report to address reviewers’ concerns.

3. Almost every year at national conferences there are sessions dedicated to journal editor presentations.

These sessions provide a forum for journal editors to discuss their journals and also to share experiences regarding some of the common mistakes that researchers make when they report and submit their work for review. Accessing this information can be very helpful to your students. One way is to ask editors to share their slides from such sessions. Another option is to have journal editors Skype class. Select one editor from a review journal and one from an empirical journal. The editors prepared initial statements and then the students asked questions that they had prepared in advance. This provided an opportunity for me to make the reporting and review process a little more familiar to students who lacked experience with the publication process.

4. If you are on editorial boards another strategy is to request to the editor that your class review a manuscript. If the class is small all can be listed as reviewers. If not, usually the class is noted in a feedback letter from the editor. I have done this several times and the students really like this activity. It works best if you have them all independently review the article first and then work through it in class. This activity can also work with a review you have done in the past. This allows you better control of the quality of the manuscript that is reviewed.

5. With permission, if you are working on a project with a student, have them share their work before and after feedback. If the student is a slightly more able peer it is quite well received. Students then get to see the process of someone more similar to them than you are

105 .

writing, modifying, and submitting research work. I think hearing about the review and critique process from another student makes the writing of research less intimidating and more easily accomplished.

6. Similarly, have recent masters and doctoral students come and talk about the process with my class. I usually ‘host’ it as a social. The students may come to a portion of the class session and talk about their recent experiences. They may share lessons learned, and tell stories, such as the classic experience of revising multiple times only to have their advisors change things back to the way they had them in the first place. Students often have had the experience of not having enough data, or nonsignificant findings, or a host of other thesis ills. It helps the students in the research class to see that no study is perfect and the process, while long, is rewarding.

7. As this chapter comes at the end of the class and addresses the preparation of a research report, this is an excellent time to encourage the students to become VERY familiar with the APA manual. The APA manuscript submission checklist is found on line (apa.org/pubs/authors/manuscript-check.aspx).

Students will need a copy of the sixth edition of the APA manual to gain the most from the site.

8. A peer review process in class works well as an out of class assignment. Options include a blind review process or having students swap with peers with whom they feel most comfortable. By critiquing and reviewing another’s ‘submission’, the student is better able to attend to important details in their own writing. In addition to the criteria found in Chapter 23, the checklist in the APA manual can facilitate this peer review activity.

9. Students often submit versions of their research that are slightly different from a traditional research proposal or a report of completed research. For example, students often apply for dissertation support, for participation in a professional conference, or for selection into a special doctoral seminar hosted by a professional organization such as APA or AERA. These proposals or reports often require a research summary longer than an abstract but much shorter than that my students have written for any other purpose. These types of proposals and reports are challenging for the novice researcher to compose. As a class assignment, ask students to submit a summary, heavy on the methodology, which might fit such guidelines. AERA and the regional organizations’ calls for submission are rotating so it may be difficult to share any one call while you are teaching. AERA proposals are usually due in mid to late July and as such are good timing for many. Refer to aera.net for the call timeline and the requirements for the proposal submission. The Spencer Foundation (spencer.org) funds dissertation research and a call for proposals can be found at their website.

10. At the end of the semester, hold a poster fair for the last class where students can practice presenting their proposals or completed projects. This activity is transfer-appropriate as many students may have the chance to present research at conferences. Guidelines for poster format vary a bit by conference or exhibition. The current suggestions for poster presentation for AERA are online (aera.net). These guidelines are very helpful to the novice researcher.

106 .

APPENDIX What is your GPA? _______________ Class standing? ___________ What is your gender? ________ What was your SAT combined score? __________ Please consider the class you have directly AFTER EDPSY in which there is a test.

What is your current grade in that class? _____________ What types of tests are given? (Please Circle) Multiple-Choice Essay

Short Answer

Mix of items Other: ________

1= Never 2=Seldom 3=Sometimes 4=Often 1.

Test Anxiety decreases my test performance.

2.

Test Anxiety helps me to do better on exams.

The remaining items should be read as “I feel stressed” …

3.

The day before the test.

4.

The morning before the test.

5.

On the way to take the test.

6.

When I study a lot for a test.

7.

When I do NOT study a lot for a test.

8.

About how the test is going to affect my grade.

9.

About what my parents will think about my test performance.

10.

About what my friends will think about my test performance.

11.

About what my classmates will think about my test performance .

12.

About what my teacher will think about my test performance.

13.

When taking multiple-choice tests.

14.

When taking short-answer tests.

15.

When taking essay tests.

16.

When I can’t do part of the test.

17.

When there is a time limit.

18.

When I think about how I am doing on the test.

19.

When I receive the test.

20.

When I see other people turn in their tests.

21.

When I am the first to turn in the test.

22.

When I have trouble on the first problem or item.

23.

After I turn in the test.

24.

After talking to other people in the class.

25.

After hearing the teacher talk about the answers.

26.

When the teacher talks about the class average.

27.

When waiting for the test to be returned.

28.

When the test is being returned.

107 .

5=Always 1 2 3 4 5 1 2 3 4 5 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3

4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5 4 5

ORP USE ONLY: IRB# 20008 The Pennsylvania State University Office for Research Protections Approval Date: 04/25/05 M. Becker Expiration Date: 12/06/05 M. Becker Social Science Institutional Review Board

INFORMED CONSENT FORM FOR SOCIAL SCIENCE RESEARCH The Pennsylvania State University

Title of Project: Learning from Expository Text with Multiple Representations Principal Investigator: Rayne A. Sperling, PhD Other Investigator(s): P. Karen Murphy, PhD, Nicholas D. Warcholak, MS, and Jill Gushka, MA This study is designed to examine the differences in how people learn from text with different representations (graphs, prompts, and others). If you agree to participate, you will be asked to come to one of the specified classrooms at the specified times, complete a demographic inventory, read some text materials, and answer questions based on those materials. The materials are similar to those you work with in class everyday.

The session should take about 45 minutes. You will receive 3 extra credit points for your participation. If you decide not to participate in this study your grade will be unaffected. You may also request to do an article review instead of participating in the study to earn the 3 extra credit points. The reviews will be collected from a member of the research team who is not your instructor.

The information generated from this study will be used to inform other researchers and teachers about how best to deliver text materials to students.

You may ask any questions that you have about the research procedures by contacting Dr. Rayne Sperling at 232 CEDAR Building, 814-863-2261, rsd7@psu.edu or Nicholas Warcholak at 225 CEDAR Building, 814-863-4486, nlw124@psu.edu. If Dr. Sperling is your instructor, please contact Nicholas Warcholak.

Your participation is confidential. Only research team members will have access to the information, and your instructor will not know who participated in the study or who did the alternate project. Additionally, all personal identifiers will be replaced with code numbers, so that no confidential information will be stored. The Office for Research Protections and the Social Science Institutional Review Board may review records related to this project.

Your participation in this study is voluntary. You are free to withdraw from the research at any time or to decline to answer any questions without penalty. You will still receive the extra credit. This study involves no risk to your physical or mental health beyond those encountered in the normal course of everyday life. You may also contact the Office for Research Protections, 212 Kern Building, (814) 8651775, with any questions regarding the rights of participants.

You will be given a copy of this consent form to keep for your records. You must be 18 years or older to participate in this study. ______________________________________ Participant Signature

_____________________ Date

I, the undersigned, verify that the above informed consent procedure has been followed. ______________________________________ Investigator Signature

_____________________ Date 108

.

REFERENCES Achinstein, B., Ogawa, R. T., & Speiglman, A. (2004). Are we creating separate and unequal tracks of teachers? The effects of state policy, local conditions, and teacher characteristics on new teacher socialization. American Educational Research Journal, 41(3), 557-604.

The American Educational Research Association: aera.net American Psychological Association: apa.org Archer, T. M. (2003). Web-based surveys. Journal of Extension, 41(4). Retrieved from joe.org/joe/2003august/tt6.shtml Atof Inc. Big Five inventory (2009). Available at outofservice.com/bigfive/ Bailey, R.L., & Angell, M. E. (2005). Improving feeding skills and mealtime behaviors in children and youth with disabilities. Education and Training in Developmental Disabilities, 40(1), 80-96.

Barnes, S. B., & Whinnery, K.W. (2002). Effects of functional mobility skills training for young students with physical disabilities. Exceptional Children, 68, 313–324.

Baron, J. (2005). Key Items To Get Right When Conducting a Randomized Controlled Trial in Education.

Retrieved from coalition4evidence.org/wp-content/uploads/2012/05/Guide-Key-items-to-GetRight-RCT.pdf Bertot, J. C. & McClure, C. R. (1996). Electronic Surveys: Methodological implications for using the World Wide Web to collect survey data. Proceedings of the ASIS Annual Meeting, 33, 173-185.

Blass, T. (2004, July). The Stanley Milgram Website. Retrieved from stanleymilgram.com Boulineau, T., Fore, C., Hagan-Burke, S, & Burke, M.D. (2004). Use of story-mapping to increase the story grammar text comprehension of elementary students with learning disabilities. Learning Disability Quarterly, 27(2), 105-121.

Buros Center for Testing Mental Measurements Yearbook: buros.org Callahan, C., & Barisa, M. T. (2005). Statistical process control and rehabilitation outcome: The singlesubject design reconsidered. Rehabilitation Psychology, 50(1), 24-33.

Campbell, J. M (2003). Efficacy of behavioral interventions for reducing problem behavior in persons with autism: A quantitative synthesis of single-subject research. Research in Developmental Disabilities, 24(2), 120-138.

Capizzano, J., Bischoff, K., Woodroffe, N., & Chaplin, D. (2007). Ingredients of a successful summer learning program: A case study of the building educated leaders for life (BELL) accelerated learning summer program. ED497332 Carter, R. T., Scales, J. E., Juby, H.L., Collins, N.M., Wan, C.M. (2003). Seeking career services on campus: Racial differences in referral, process, and outcome. Journal of Career Assessment, 11(4), 393-404.

109 .

Chang, M., Singh, K., & Mo, Y. (2007). Science engagement and science achievement: Longitudinal models using NELS data. Educational Research and Evaluation, 13(4), 349-371.

Colorado State University Writing Center: Case Study Writing Guide: writing.colostate.edu Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework improve academic achievement? A synthesis of research, 1987-2003. Review of Educational Research, 76(1), 1-62.

Coryn, C. L., Schröter, D. C., & McCowen, R. H. (2014). A mixed methods study of some of the factors associated with successful school strategies for native Hawaiian students in the State of Hawai’i.

Journal of Mixed Methods Research, 8, 377-395.

Coulter, C. A., & Smith, M. L., (2009) The Construction Zone. Educational Reasercher, 38(8).

Crede, E., & Borrego, M. (2013). From ethnography to items: A mixed methods approach to developing a survey to examine graduate engineering student retention. Journal of Mixed Methods Research, 7, 62-80.

Davidovitch, N., & Soen, D. (2006). Class attendance and students' evaluation of their college instructors.

College Student Journal, 40(3), 691-703.

Debevec, K., Shih, M., & Kashyap, V. (2006). Learning strategies and performance in a technology integrated classroom. Journal of Research on Technology in Education, 38(3), 293-307.

Department of Housing and Urban Development grant proposals: portal.hud.gov Dick, B. (2010). Action research literature 2008-2010: Themes and trends. Action Research, 9, 122-143.

Educational Evaluation and Policy Analysis website: aera.net/publications/?id=476 Education Research Global Observatory. Retrieved from ergobservatory.info/ejdirectory.html Edutopia: edutopia.org eSurveypro esurveyspro.com/ Ethnography and Education Journal. Retrieved from tandf.co.uk/journals/titles/17457823.asp Fallon, K. A., Light, J., McNaughton, D., Drager, K., Hammer, C. (2004). The effects of direct instruction on the single word reading skills of children who require augmentative and alternative communication. Journal of Speech, Language, and Hearing Research, 47(6), 1124-1439.

G* Power website for links to download gpower.hhu.de/en.html Guay, F., Marsh, H., Boivin, M. (2003). Academic self-concept and academic achievement: Developmental perspectives on their causal ordering. Journal of Educational Psychology, 95(1), 124-136.

Glover, J. A., Ronning, R., & Bruning, R. H. (1990). Educational Psychology: Principles and applications (3rd ed.). Glenview, IL: Scott, Foresman, & Company.

110 .

Goree, C. T., & Marszalek, J. F. III (1995). Electronic Surveys: Ethical issues for researchers. College Student Affairs Journal, 15, 75-79.

Hart, L.C., Smith, S. Z., Swars, S.L., & Smith, M.E. (2009). An examination of research methods in mathematics education. Journal of Mixed Method Research, 3(1), 26-41.

Hamre, B. K., & Pianta, R. C. (2005). Can instructional and emotional support in the first-grade classroom make a difference for children at risk of school failure? Child Development, 76, 949 – 967.

Hartmann, T., Maluk, H. P., & Riffer, M. (2007). Teachers and students learning through service: A report on need in deed's developing work with teachers. ED499066 Harris, M. B., & Brooks, L. J. (1998). Challenges for older students in higher education. Journal of Research and Development in Education, 31, 226-235.

Heath, A.W. (1997). The proposal in qualitative research. The Qualitative Report, 3(1). Retrieved from nova.edu/ssss/QR/QR3-1/heath.html Heitzmann, R. (2007). Target homework to maximize learning. Education Digest: Essential Readings Condensed for Quick Review, 72(7), 40-43.

Helmke, A., van Aken, M. (1995). The causal ordering of academic achievement and self-concept of ability during elementary school: A longitudinal study. Journal of Educational Psychology, 87(4), 624-637.

Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., Wolery, M. (2005). The use of single subject research to identify evidence based practice in special education. Exceptional Children, 71(2), 165-179.

Igo, L.B., Riccomini, P.J., Bruning, R.H., & Pope, G.G. (2006). How should middle-school students with LD approach online note taking? A mixed methods study. Learning Disability Quarterly, 29, 89– 100.

Institute of Educational Sciences guidelines for federal funding. Retrieved from ed.gov/about/offices/list/ies/index.html Iowa Test of Basic Skills. Retrieved from riversidepublishing.com Jackson, B. (2007). Homework inoculation and the limits of research. Phi Delta Kappan, 89(1), 55-59.

Jones, C. D., & Schwartz, I. S. (2004). Siblings, peers, and adults: Differential effects of models for children with autism. Topics in Early Childhood Special Education, 24(4), 187-198.

Kim, J-H. (2006). For whom the school bell tolls: Conflicting voices inside an alternative high school.

International Journal of Education & the Arts, 7, 1–19.

Koerner, M.E., & Abdul-Tawwab, N. (2006). Using community as a resource for teacher education: A case study. Equity & Excellence in Education, 39, 37–46.

111 .

Kulikowich, J. M., & Sperling, R. A. (2011). Prescriptive statements: Philosophical, theoretical, and methodological considerations, Educational Psychology Review, 23,189-195.

Lammi, B. M., Law, M. (2003). The effects of Family Centered Functional Therapy on the occupational performance of children with cerebral palsy. Canadian Journal of Occupational Therapy, 70(5), 285-297.

La-Pelle, N. (2004). Simplifying qualitative data analysis using general purpose software tools. Field Methods, 16(1), 85-108.

Lenski, S.D., Crawford, K., Crumpler, T., & Stallworth, C. (2005). Preparing preservice teachers in a diverse world. Action in Teacher Education, 27, 3–12.

LimeSurvey limesurvey.org/ MacMilan, K., Koenig, T. (2004). The wow factor: Preconceptions and expectations for data analysis software in qualitative research. Social Science Computer Review, 22(2), 179-186.

Mangabeira, W. L, & Raymond, M., & Fielding, N. G. (2004). Computers and qualitative research: Adoption, use, and representation. Social Science Computer Review, 22(2), 167-178.

Marlow, L., Inman, D., and Shwery, C. (2005). To what extent are literacy initiatives being supported: Important questions for administrators. Reading Improvement, 42, 179–186.

Marsh, H.W. (1990). Causal ordering of academic self-concept and academic achievement: A multiwave, longitudinal analysis. Journal of Educational Psychology, 82(4), 646-656 Marsh, H.W., Bryne, B. M., Yeung, A. S. (1999). Causal ordering of academic self-concept and achievement: Reanalysis of a pioneering study and revised recommendations. Educational Psychologist, 34(3), 155-167.

Marsh, H. W., Hau, K. T., Kong, C. K. (2002). Multilevel causal ordering of academic self-concept and achievement: Influence of language of instruction (English compared to Chinese) for Hong Kong students. American Educational Research Journal, 39(3), 727-763.

Marsh, H.W., & Perry, C. (2005). Self-concept contributes to winning gold medals: Causal ordering of self-concept and elite swimming performance. Journal of Sport and Exercise Psychology, 27(1), 71-91.

Marsh, H. W., Trautwein, U., Ludtke, O., Koller, O., Baumert, J. (2005). Academic self-concept, interest, greades, and standardized test scores: Reciprocal effects models of causal ordering. Child Development, 76(2), 397-416.

Martin, G. L., Thompson, K., & Regehr, K. (2004). Studies using single-subject designs in sport psychology: 30 years of research. Behavior Analyst, 27(2), 263-280.

McCroskey, James website for developed and tested measures. Retrieved from jamescmccroskey.com/measures/ Milgram, S. (1969). Obedience. (Videotape of the classic research on obedience to authority, based on footage shot at Yale University.) Available for purchase from Penn State Media Sales at MediaSales@ psu.edu.

112 .

Mullis, I.V.S., Martin, M.O., Fierros, E.G., Goldberg, A. L., & Stemler, S.E. (2000). Gender differences in achievement. IEA’s Third International Mathematics and Science Study (TIMSS).

International Association for the Evaluation of Educational Achievement: Boston, MA. Retrieved from timss.bc.edu/timss1995i/TIMSSPDF/t95_gender_all.pdf Murphy, P. K., & Alexander, P. A. (2000). A motivated exploration of motivation terminology.

Contemporary Educational Psychology, 25, 3-53.

Myers-Briggs. Available at myersbriggs.org National Center Educational Statistics Early Childhood Survey-Kindergarten ECLS-K 98-99 nces.ed.gov/ecls/kindergarten.asp Nolen, A., & Talbert, T. (2011). Qualitative assertions as prescriptive statements. Educational Psychology Review, 23, 263-271.

Otis-Lennon Aptitude Test. Available at pearsonassessments.com Peltier, G. L., Laden, R., & Matranga, M. (1999). Student persistence in college: A review of research.

Journal of College Student Retention, 1, 357-375.

Plowright, D. (2014). To what extent do postgraduate students understand the principles of mixed methods in educational research? International Journal of Multiple Research Approaches, 7, 6682.

Practical Assessment Research and Evaluation Journal Website: pareonline.net Pritchard, I. A. (2002). Travelers and Trolls: Practitioner research and institutional review boards.

Educational Researcher, 31(3), 3-13.

Rudner, L. M. (1994). Questions to ask when evaluating tests. Practical Assessment, Research, & Evaluation, 4(2). Retrieved from PAREonline.net/getvn.asp?v=4&n=2 Ryan, G. W. (2004). Using a word processor to tag and retrieve blocks of text. Field Methods, 16(1), 109-130.

Saracho, O. N. (2008). A literacy program for fathers: A case study. Early Childhood Education Journal, 35(4), 351-356.

Schilling, D. L., & Schwartz, I. S. (2004). Alternative seating for young children with autism spectrum disorder: Effects on classroom behavior. Journal of Autism and Developmental Disorders, 34(4), 423-432.

Shadish, W. R., & Rindskopf, D. M. (2007). Methods for evidence-based practice: Quantitative synthesis of single-subject designs. New Directions for Evaluation, 113, 95-109.

Skaggs, A. N. (2007). Homework: A nightly ritual beginning in the elementary grades. Online Submission; 52p. ED497419

113 .

Skinner, C. (C. H. (2004). Single-subject designs: Procedures that allow school psychologists to contribute the intervention evaluation and validation process. Journal of Applied School Psychology, 20(2), 1-10.

Sleeter, C. (2009). Developing teacher epistemological sophistication about multicultural curriculum: A case study. Action in Teacher Education, 31, 3-13. doi: 10.1080/01626620.2009.10463506 Smith, M. Action Research in Encyclopedia of Informal Education. Retrieved June 2011 from http://www.infed.org/research/b-actres.htm Sommer, R., & Sommer, B. A. (2007). Credit for comments, comments for credit. Teaching of Psychology, 34(2), 104-106.

Sorensen, P., Twidle, J., Childs, A., & Godwin, J. (2007). The use of the internet in science teaching: A longitudinal study of developments in use by student-teachers in England. International Journal of Science Education, 29(13), 1605-1627.

Spencer Foundation: spencer.org American Educational Research Association. (2014). Standards for educational and psychological testing. Washington, DC: Author Standler, R. B. (1997). Nonconsenual medical experiments on human beings. Retrieved from http://www.rbs2.com/humres.htm Stanford-Binet Aptitude test. Available at riversidepublishing.com Stanford 9 Achievement Test. Available at pearsonassessments.com Survey Monkey: surveymonkey.com Taylor-Powell, E., & Renner, M. (2003). Analyzing qualitative data. University of WisconsinExtension, Cooperative Extension. Accessible from learningstore.uwex.edu/AnalyzingQualitative-Data-P1023C0.aspx.

Terenzini, P. T., Pascarella, E. T., & Blimling, G. S. (1999). Students’ out-of-class experiences and their influence on learning and cognitive development: A literature review. Journal of College Student Development; 40. 610-23.

This American Life website: thisamericanlife.org Tinto, V. (1998). Colleges as Communities: Taking Research on Student Persistence Seriously. Review of Higher Education; 21, p167-77.

Trautwein, U. (2007). The homework-achievement relation reconsidered: Differentiating homework time, homework frequency, and homework effort. Learning and Instruction, 17(3), 372-388.

University of Glasgow Faculty of Education Ethics Guidelines gla.ac.uk/services/dpfoioffice/ University of Southern California, Los Angeles. School of Education. (2007). Making the grade in college prep: A guide for improving college preparation programs. ED499275 114 .

University of Wisconsin Distance Education journals and magazines. Retrieved from depd.wisc.edu/html/mags3.htm Washington Univ., Seattle. Center on Reinventing Public Education. (2007). Inside charter schools: A systematic look at our nation's charter schools project update. ED495842 Watson, S. C. (1998). A primer in survey research. Journal of Continuing Higher Education, 46, 31-40.

Wessa, P. (2011), Free Statistics Software, Office for Research Development and Education, version 1.1.23-r7. Available from wessa.net Woodruff, A. L., & Schallert, D. L. (2008). Studying to play, playing to study: Nine college studentathletes’ motivational sense of self. Contemporary Educational Psychology, 33, 34-57.

Wolgemuth, J.R., Cobb, R.B., Winokur, M.A., Leech, N., & Ellerby, D. (2006). Comparing longitudinal academic achievement of full-day and half-day kindergarten students. The Journal of Educational Research, 99, 260–269.

Xin, Y.P., Jitendra, A.K., & Deatline-Buchman, A. (2005). Effects of mathematical word problem solving instruction on middle school students with learning problems. The Journal of Special Education, 39, 181–192.

Zimmerman, L., W., McQueen, L., & Guy, G. (2003). Connecting women’s voices: a symposium on the intersection of gender, race, and class. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. [ERIC Number ED479147].

115 .

HANDOUTS

116 .

Handout 1.3 For Practice… a. A study that investigates what experts think about while solving problems.

b. A comparison between achievement scores of learners with simulations embedded in their mathematics instructional materials and those without simulations in their materials. c. A study that examines one district’s response to the NCLB legislation.

d. A study conducted by the high school art education teachers regarding the use of various instructional materials and media to facilitate student understanding of district-level objectives for art education.

e. A study that compares student grades in self-paced and instructor-taught college psychology courses.

f.

A study that determines the relationship between years of teaching and incidence of student disruptive behavior in the classroom.

g. A study that investigates what happens on a typical Saturday in our school’s detention program. h. A study that targets a reading-disabled student’s response to intervention.

i.

A study of 5 first year students from rural communities on acclimation to a large University setting.

j.

A study that addresses how parents’ feel about the new district standards regarding calculators in math classes.

k. A study that examines the relationship between in class attendance and course grades.

l.

A study that addresses the role of ethnicity in college entrance examination testing in the 1950 and 1960’s.

m. A study to determine whether a summer reading program should be continued.

117 .

Handout 1.4 While some topics may be most often approached for study by a particular method or type of research some topics can easily be considered from many different types of research. One example of such a topic is school uniforms. Given the general topic of school uniforms, develop a research topic for each of the following approaches. a. Narrative: __________________________________________________________________ ___________________________________________________________________________

b. Ethnography: _______________________________________________________________ ___________________________________________________________________________

c. Case Study: _________________________________________________________________ ___________________________________________________________________________

d. Single-Subject: ______________________________________________________________ ___________________________________________________________________________

e. Survey: ____________________________________________________________________ ___________________________________________________________________________

f.

Correlational: _______________________________________________________________ ___________________________________________________________________________

g. Action research: _____________________________________________________________ ___________________________________________________________________________

h. Causal-comparative : _________________________________________________________ ___________________________________________________________________________ i.

Mixed methods : _________________________________________________________ ___________________________________________________________________________

118 .

Handout 2.1 Guided notes for Pritchard (2002).

Answer the following guided notes questions to support your reading and understanding of Prichard (2002). Bring these answers to class to facilitate our discussion of the IRB and human subjects review.

1. How does the author define practitioner research?

2. Why does practitioner research pose ethical concerns?

3. Why does the author suggest the operational definitions of research held by various constituents are problematic? a. What is the IRB’s definition of research? b. What is the practitioner researcher's definition of research? c. What is the research ethicist's definition of research? d. How do these three definitions compare and contrast regarding ethical concerns of practitioner research?

4. On page 5, the author states the three components of informed consent. How does this compare with your text’s discussion of informed consent?

5. What is the “educational misconception”? Why is it of concern for practitioner research?

6. Briefly summarize why, according to the author, anonymity and confidentiality are particularly problematic in practitioner research. Also address how coercion can be of particular concern in action research.

7. The author suggests that practitioner research may not ‘jibe’ with working IRB categories.

Explain.

8. What are the IRB’s concerns with reference to risk?

9. What is the ‘common rule’?

10. The author suggests several ways IRBs can be reformed. Select one and briefly summarize.

11. Do you think the IRB is a ‘troll’? Support your answer.

119 .

Handout 3.1 For each of the following general questions identify a research problem, and write a null, nondirectional, and directional hypothesis. a. How does increased standardized testing relate to student motivation?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: b. Are there gender differences in students’ perceptions of Inquiry-based instruction?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: c. Does reading-disabled students’ comprehension benefit from guided notes?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: d. Do puzzle experiences in the primary grades matter in students’ later science interest?

Research Problem: Null Hypothesis: Nondirectional Hypothesis: Directional Hypothesis: e. Are there differences in the number of discipline referrals per student in large versus small schools?

Research Problem: Null Hypothesis: 120 .

Nondirectional Hypothesis: Directional Hypothesis:

121 .

Handout 4.4 1. Provide two keywords that would be representative for this article.

2. Is this a primary source or a secondary source? How do you know?

3. Which database might be appropriate to use to find this article?

4. If your article represents a study, please answer the following. a. What was the problem statement of the study?

b. Who was studied?

c. How was the study conducted?

d. Is this a quantitative or qualitative study?

5. If your article does NOT present a study, please answer the following.

a. Is this a literature review? How do you know?

b. Is this an essay or a position piece? If so, is this research piece objective? How does the author present the critical issue presented.

6. Suggest one purpose for which you might use this article.

122 .

Handout 7.2 Identifying and Operationalizing Variables For each research topic identify the dependent and independent variable. Develop the topic by operationalizing variables.

Are there differences between rural and urban children’s attitudes regarding diversity?

Is there a relationship between years of post-secondary schooling and social competence?

How will learners enrolled in an intensive mathematics program over the summer achieve in math compared to those who are not enrolled in the program?

What are the academic variables that account for a successful collegiate experiences?

123 .

Handout 10.5 Generate a causal-comparative research question that one could examine to address high school class attendance.

What is the independent variable? Use Figure 10.1 in your text to label the independent variable.

Identify the levels of the independent variable.

What is the dependent variable? How will you measure the construct?

124 .

Handout 11.5 Use the Baron (2005) publication, Key Items to Get Right When Conducting a Randomized Control Trial in Education. Answer the following questions.

The publication is available at: coalition4evidence.org/wp-content/uploads/2012/05/Guide-Key-items-to-Get-Right-RCT.pdf 1. The first section of the publication notes key items to get right in planning the study. a. What does the author mean by surrogate outcomes? In a study of mathematical problem solving what might be an example of a surrogate outcome? Why are surrogate outcomes a concern? b. The author presents two reasons when it may be more advisable to randomly assign groups instead of individuals. In the first reason, ‘spillover’ effects are presented. How do such spillover effects relate to threats to validity as discussed in Chapter 10 of your text? Select two examples and explain. c. As discussed elsewhere, the author promotes the use of a power analysis. What key features of the design are necessary for including in an a priori power analysis? Pick one and provide an example.

2. The second section of the publication notes key items to consider during the random assignment process. a. Why would it be advisable to have someone not invested in the study conduct the random assignment? b. Random assignment is sometimes challenging. What does the author mean when he says make sure the random assignment is truly random? What is one example of a non truly random assignment someone might conduct? Why does example present a concern? c. Why would a participant need a unique personal identifier? d. The author suggests that researchers monitor what services the control group might be receiving. Suggest one reason why this can be a concern. Provide an example.

3. The third section of the publication notes key items to get right in measuring outcomes for the study sample. a. What are some of the key characteristics you would seek in your instrumentation?

Explain in your own words why each is important. b. What is one way you might be able to limit subjectivity in administration? c. Why might you want to keep track of those who fail to complete the study? How does this relate to the threats discussed in your text? d. What are some of the benefits and drawbacks of collecting long term outcomes?

125 .

4. The fourth section notes key items to get right in the analysis of study results. a. How should you analyze those individuals who may have benefited from cross-over effects? Why? b. In addition to statistical significance of the intervention’s effect what else should be reported?

126 .

Handout 12.1 Determine if each of these general questions could be addressed in a single-subject design. If so, how might be the study conducted? Please draw a representation of one proposed design.

1. Do students who sit at tables in science class achieve more than those who sit in desks?

2. Do students taught physics through direct instruction learn more than those taught through experiential learning?

3. Does providing students recess decrease out of seat behaviors?

4. Does providing students web-based practice increase math fluency?

5. Does structured peer-tutoring benefit second language vocabulary?

127 .

Handout 13.1 Coulter and Smith introduce this themed issue of Educational Researcher with a piece titled The Construction Zone: Literary Elements in Narrative Research.

1. What is the primary purpose of this article as suggested by the authors?

2. Compare the description found in the article to that presented in text for narrative analysis and analysis of narrative.

3. What is meant by literal truth and storied truth?

4. The authors present several literary elements. Select two and dissect at least one example of each from those shared from the texts they selected to illustrate in their article.

128 .

Handout 16.1 Please consider the following abstracts and answer the following: a. Identify the purpose of the study. b. Based upon the abstract, what is the basic design (explanatory sequential, exploratory sequential, or convergent parallel) used for the study? Provide a rationale for your answer. c. List one data source that is quantitative and one data source that is qualitative.

Plowright, D. (2014). To what extent do postgraduate students understand the principles of mixed methods in educational research? International Journal of Multiple Research Approaches, 7, 66-82.

In this article, the findings of a small-scale exploratory study investigating postgraduate students' understandings of the use of mixed methods as a research strategy are reported. The empirical research was in two stages. Firstly, an on-line questionnaire survey was completed by 118 postgraduate students. That was followed by small-group, face-to-face discussions of 13 volunteers from the first stage. There is evidence from the findings. That there is some confusion and inconsistencies in the views of participants about the underlying methodological and philosophical principles associated with carrying out research in general and mixed methods research in particular. One source of the confusion might be the continuing use of the qualitative/quantitative distinction. It is recommended that a reconceptualising of the research process is now needed, based on characteristics of integration and not just the mixing or combining of different approaches to research.

Crede, E., & Borrego, M. (2013). From ethnography to items: A mixed methods approach to developing a survey to examine graduate engineering student retention. Journal of Mixed Methods Research, 7, 62-80.

As part of a […] mixed methods study, 9 months of ethnographically guided observations and interviews were used to develop a survey examining graduate engineering student retention.

Findings from the ethnographic fieldwork yielded several themes, including international diversity, research group organization and climate, perception of value, and individual and group learning. In this article, the authors present the final themes from the ethnographic analysis and discuss how these data were configured into constructs and survey questions.

The authors discuss the final survey, including validity and reliability analysis, and how constructs were developed to test hypotheses for future studies. The article concludes with implications for mixed methods researchers interested in using qualitative methods to create new surveys.

Coryn, C. L., Schröter, D. C., & McCowen, R. H. (2014). A mixed methods study of some of the factors associated with successful school strategies for native Hawaiian students in the State of Hawai’i. Journal of Mixed Methods Research, 8, 377-395.

A […] mixed methods design, giving equal priority to both quantitative and qualitative methods, was used to investigate some of the factors that influence academic achievement of Native Hawaiian students in public schools in the state of Hawai’i. School-level instructional strategies, curricula, policies, and other variables that distinguish more successful schools from less successful schools were a primary focus of the study. Successful schools are those having collaborative school governance structures, linking decision structures to data, having a well-established and dedicated teaching force, having focused learning 129 .

communities, having engaged leaders, share accountability for their students’ learning, are committed to continuous learning, and that provide effective supplementary and after-school programming.

Sorensen, P., Twidle, J., Childs, A., & Godwin, J. (2007). The use of the internet in science teaching: A longitudinal study of developments in use by student-teachers in England.

International Journal of Science Education, 29(13), 1605-1627.

This paper reports on a longitudinal study of developments in use of the Internet by science student-teachers on Post Graduate Certificate of Education (PGCE) courses in five higher education institution-school partnerships in England. These are 1-year, full-time, teacher training courses for graduate scientists. The aim of the research was to examine changes in attitudes to, and use of, the Internet to support science teaching and the perceived challenges and barriers to practice in schools, against a background of high national expectations reflected in the qualification standards of the teacher education courses. The research has involved nearly 600 student-teachers, representing between 7% and 8% of those training on PGCE science courses in England, and has employed mixed methods, with questionnaires serving as the main basis for analysing trends, and focus groups and case studies used to gain deeper insight to the particular issues identified. The process has been an iterative one, with the outcomes of each year's research being used to inform further research and course developments in the institutions involved. The findings indicate that attitudes and confidence in use of the Internet have improved over the period, with evidence of increased application directly in the classroom. However, in addition to some of the generic technological issues that may hinder developments in the use of Information and Communication Technology in schools, there are continuing concerns relating to limited pedagogical guidance and availability of good role models. The implications of this for developments in science teacher education programmes are discussed. (Contains 6 tables.) [Appended to this article is a copy of the "Areas Examined in the Questionnaire Relevant to the Research."]

130 .

Handout 17.1 Action Research In his article in the encyclopedia of information education (http://www.infed.org/research/bactres.htm) Mark Smith presents an overview of action research.

1. In the article Smith presents two definitions of action research. How do each of these definitions compare to the definition presented in your text?

2. On page 2 of Smith’s article, he presents a spiral of steps in action research based on Lewin’s work. How does this sequence of steps correspond to the dialectic action research spiral presented in your text? Later in his article Smith includes basic phases of action research process (p.4). How do these three phases relate to the dialectic action research spiral?

3. In both the chapter and the article Action research rely on characteristics of action research based on Stringer’s work. List these characteristics and provide a novel illustration of how action research in a school might demonstrate these characteristics.

4. Provide one critique that Smith notes regarding Lewin’s work. Do you feel that the approach discussed in your text addresses this critique? Why or Why not?

131 .

Handout 18.1 (Data found below the questions) 18.1 Doug and David are researchers who examine youth health. Numerous recent studies have suggested that youth in the 10-15 year old range are increasingly engaged in many hours of video game playing, they are not active enough (as defined by at least one hour of exercise per day), and they are disconnected to their family units. However, in some of their own previous research Doug and David found that although students play a lot of video games they remain active and engage in sufficient exercise. In their current study they asked children to keep a log and then to report for a given week the number of hours of video game playing and exercise that they engage in and also to report the number of dinners that their families eat together as a measure of family connection. They also collected gender and age as demographic data.

1. Generate one descriptive research question that can be addressed by Doug and David’s data.

2. Generate one relationship research question that can be addressed by these data.

3. State one null hypothesis that can be tested with these data.

4. Identify the level of measurement for each of the following variables: Age (reported in years), Gender (reported as Female=1; Male=2), Hours of video game playing in the week (reported to the quarter hour), Exercise (reported to the quarter hour), Dinners taken together as a family (0-7 dinners in the week).

5. Please enter the data into a file after double-checking for accuracy.

6. Please provide descriptive information for all of the variables. Which is the best measure of central tendency for each variable? Why? 7. Are there more girls or boys in the sample? On average, do youth in the sample play more time playing video games or exercising?

8. Graph the dinner (pie chart) and gender (histogram) data. Graph age, hours of game playing, and hours of exercise data with frequency distributions.

132 .

ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

AGE 12 13 12 11 11 12 13 12 12 12 12 14 13 13 12 12 11 14 14 12 12 12 13 11 12 12 14 12 12 13 13 14 11 12 12 13 12 12 13 11

Gender 1 1 2 1 1 2 1 2 1 2 1 1 2 1 2 1 1 1 2 2 2 2 2 1 1 2 1 2 1 2 1 2 1 1 1 1 2 1 2 2

Hours 3 14 12 6 2.5 15 14 16 6 8 8 5 13 12 8.5 7.75 8.25 9 12 16.5 5.75 9.75 12.5 11.25 8.25 24 1.5 4 2.75 14 2.5 9.25 8.75 11.5 12.5 8.25 7.5 5.75 6.25 6.5

Exercise 14 10 2 7 8.5 6 5 6 8 9 12 14 12 3 7 6 5.5 8 7 11.25 10 4 3 4 4 0 14.5 12 12 2.5 13 10 8 7.75 6.5 7 12.25 14 11.5 10.5

Dinner 5 3 2 3 3 2 1 4 3 3 3 3 3 2 2 4 6 2 1 3 4 5 1 3 1 2 3 1 3 4 7 2 3 3 3 2 2 4 1 2

133 .

Handout 20.3 Working a small group or with a peer consider the following potential focal points for a qualitative research study to be conducted within a district.

School Climate Leadership opportunities Playground culture Physical Education as a context Consider the following questions: Who might be your informant(s)? Consider multiple levels of informants.

What settings might you observe? How often, how, and why?

Who might you interview? How often and why?

What other sources might be available as data (think deeply)? Consider documents including correspondence, policies, photos and other materials.

Are there any ‘things’ you would want to catalog or describe?

134 .

Handout 21.1 Given the following article: Nolen, A., & Talbert, T. (2011). Qualitative assertions as prescriptive statements. Educational Psychology Review, 23, 263-271.

1. The authors suggest that there are necessary conditions that must be met prior to qualitative researchers stating prescriptive statements. Select one and provide a rationale for that condition.

2. How do the authors refer to data? How does this stance relate to your text?

3. One large focus of the manuscript is the role of the researcher. State and describe one characteristic or goal of qualitative researchers. Why is this characteristic or goal desirable or necessary?

4. In the end, what types of statements do the authors suggest qualitative researchers can make regarding claims from their evidence?

5. The text uses the term trustworthy research reports. How might the authors consider this term? What to the authors makes a report trustworthy?

6. Share one surprising aspect of the manuscript.

7. Share one place where the manuscript extends or contradicts the text content regarding qualitative research.

135 .