Research Design Critique

1 Research Design Critique Paper Outline I. Research question II. Research hypothesis III. Independent and dependent variables IV. Confounding variables V. Research design A. Strengths and weaknesses of the design B. Control of internal validity VI. Statistical analysis and interpretation VII. Ethical considerations VIII. Controlling external validity IX. Sample size X. Justification of researchers' conclusions

Buy this excellently written paper or order a fresh one from ace-myhomework.com

2 Research Design Critique Introduction The article “The effect of extrinsic motivation on cycle time trial performance” is authored by Michiel Hulleman, Jos J. De Koning, Florentian J. Hettinga and Carl Foster and published in the journal of Medicine & Science in Sports & Exercise, 2007 publication; volume 39, number 4 from page 709 to 715. In this article, Hulleman et al.

(2007) evaluate the effect of motivation on pacing (specifically the cycle time) when motivation is provided prior to a cycling competition.

1. What is the research question?

The authors seek to know whether providing an external motivation a few moments prior to an exercise alters the pacing strategy or whether it interferes with overall performance in cycling time trials. The research question is based on the observation that athletes usually perform well in competitions such as Olympic Games when they apply the Olympic pacing strategy yet such athletes seem less promising to perform well in pre-Olympic competitions. As such, the influence of an extrinsic factor is suspected to cause the improvement in pacing and time.

2. What is the hypothesis?

The hypothesis in this study was that cyclists would use the all-out pacing pattern upon presentation of a monetary motivation during the final (fourth) time trial thus improving pacing.

3. What are the independent and dependent variables, and constants? Consider operational definitions, and validity and reliability issues.

3 Experimental research is usually characterized by the presence of both independent and dependent variables. An independent variable is considered to cause a change on the dependent variable (Wood & Kerr, 2010). The independent variables were the cycling time trials (performance levels). The cycling time trials for 1500m in this case were divided into four: an initial practice trial, two self paced trial and a trial where a motivation in form of money was offered prior to the competition. The dependent variables included time, total power output, power output from aerobic metabolic sources and power output resulting form anaerobic sources, VO 2 and heart rate.

4. What are the most important confounding variables?

According to Wassertheil-Smoller (2004, p. 111), “a confounding variable is one that is closely associated with both the independent variable and the outcome of interest in those unexposed.” Where a close association between a confounding variable and an independent variable exists, the likelihood of the observed relationship between these two variables being truly related is very high. Prior knowledge that the subjects would participate in the competition may have influenced the outcome of the results.

Furthermore, the knowledge that the participants were not doing so for a monetary reward from the beginning may have influence pacing. Hulleman et al. (2007) also mention improved tolerance to accumulating metabolites as well as having faster VO 2 kinetics as possible confounding variables.

5. Which design has been used? To what extent does it allow the researchers to answer their research question?

This study utilizes the quantitative experimental design whereby the authors experiment cycle time and pacing using experienced cyclists and they record

4 measurements on the various variables. Using this design, it is possible to define the effects of varying the independent variable on the dependent variable. Moreover, it becomes possible to manipulate the independent variable in the most appropriate way that can result into expected outcomes (Creswell, 2003).

6. What are the strengths and weaknesses of the design of the study?

An experimental design is powerful in the sense that it prevents experimental bias. The researcher is able to provide the same treatment to all the subjects since he is able to control the environment under which the research is conducted. Since it is possible to control external factors in this kind of design, the researcher comes up with a stronger claim of causality. In addition, experimental design is suited for hypothesis testing since t-tests as well as ANOVA tests can be easily conducted (Muijs, 2004).

Hulleman et al. (2007) ensure that experimental bias is avoided by having all the cyclists perform a habituation trial prior to the cycle time trial. The habituation test is done under the same conditions for all the participants as identified in the incremental exercise test for establishing the VO2peak. The same standards were also used in the time trials including the introduction of the monetary reward in the fourth trial for all participants. External variables that could have interfered with the outcomes were therefore effectively controlled. Since experimental design helps in establishing cause and effect, Hulleman et al. (2007) are able to come to a conclusion that at least extrinsic motivation seems not to determine cycle time or the pacing strategy in a 1500 m cycle distance.

Whereas Hulleman et al. (2007) are able to control for external factors, this study is rendered weak in terms of its suitability in being applied in real life. Transferring this

5 study into a real life situation becomes quite hard since it would be quite difficult to have cyclist first be subjected to the same practice/treatment prior to say Olympic competitions. In a real life setting, the influence of external factors would be hardly avoided thus one would expect to have different results from those established in this study.

7. How well does the design control for threats to internal validity?

The experimental research design use by Hulleman et al. (2007) ensures that internal validity is controlled in that the independent variable (cycle times) are all kept uniform for all the subjects. In addition, all the characteristics of the subjects have been put into consideration thus variations due to subject characteristics have been controlled. For instance, all the subjects are males and all of them have been trained sufficiently as cyclists. As such, sex or prior cycling experience is ruled out as possible nuisance factors.

8. Is the statistical analysis and its interpretation correct?

Repeated measures ANOVA is considered to be powerful statistically since it helps in noting systematic differences that may be present between subjects. Moreover, it is a simple analytical design yet it does not compromise in its power in testing hypotheses (Norman & Streiner, 2008). Despite the sample size for the study is being quite small (n = 7), repeated measures ANOVA is able to bring out powerful conclusions. This is because repeated measures ANOVA allows a small sample to be analyzed several times other than having a large sample that is measured once which would not allow for differences in dependent variables to be viewed as the independent variable varies. Hulleman et al. (2007) are able to establish variances in time trials and

6 power output for all the participants during the four trials. This was possible due to the application of repeated measures ANOVA as an analytical test. Through the correlation coefficients generated in the repeated measures ANOVA, the authors of this study are able to establish the relationship between cycle time trials for each subject whereas ANOVA helps in determining variances in the performance (output or time) for individuals within each trial.

9. What are the ethical considerations in conducting this research?

When conducting research involving human subjects, ethical considerations must be considered. Among the issues that are of concern in studies involving human subject is the obtaining informed consent from prospective participants. The subjects must provide an informed consent. In addition to obtaining informed consent, such studies require institutional review boards to review the intended study and provide a guideline for a go ahead upon assurance that human subjects will be protected appropriately (Mackey & Gass, 2005). In this study, Hulleman et al. (2007) have taken such ethical considerations. The authors state that they had informed consent from the seven participants involved in the study. Moreover, the University of Wisconsin-La Crosse review board approved the study as a way of protecting the subject.

10. How well does the design control for threats to internal validity?

The sample (n = 7) in this case was selected from a list of competitive cyclists in the laboratory. It is however notable that since this is an experimental research; a small sample size may still give important findings unlike non-experimental studies which require a large sample size (Holzemer, 2009). The experimental design use by

7 Hulleman et al. (2007) controls for external factors well thus it becomes possible to infer outcomes with possible causes.

11. How does the sample size for this study affect external validity?

According to (Freed, Hess & Ryan, 2002), external validity entails how much the findings of a study can be generalized. To ensure that external validity is not violated, the researcher is advised to conduct random sampling. Hulleman et al. (2007) however do not appear to increase external validity since the sample has not been selected randomly.

12. Considering the above and other issues, are the conclusions of the researchers justified?

Fichtel (2009) argues that a “strong internal validity refers to the unambiguous assignment of causes to effects” (p. 58). The study by Hulleman et al. (2007) has ensured that internal validity is not violated through the use of a stringent experimental design. As such, the conclusion that cycle time and pacing is not dependent on extrinsic motivation as stated by the authors of this study is highly justifiable. Moreover, the effect of confounding factors has been well controlled not to mention that the experimental design has ensured that external factors do not influence the results of the study. At least the researchers have cautioned that the hypothesis may have been rejected due to existence of confounding factors of tolerance to metabolite accumulation among others.

8 References Creswell, J. W. (2003). Research design: qualitative, quantitative, and mixed method approaches, 2nd edition. SAGE. ISBN 0761924426, 9780761924425 Fichtel, S. (2009). What is beautiful is good: insights on the impact of employee attractiveness on market success. BoD – Books on Demand. ISBN 3934491456, 9783934491458 Freed, M. N., Hess, R. K. and Ryan, J. M. (2002). The educator's desk reference (EDR): a sourcebook of educational information and research, 2nd edition. Greenwood Publishing Group. ISBN 1573563595, 9781573563598 Hulleman, M., De Koning, J. J., Hettinga, F. J. and Foster, K. (2007). The effect of extrinsic motivation on cycle time trial performance. Medicine & Science in Sports & Exercise, 39(4): 709-715.

Holzemer, W. L. (2009). Improving health through nursing research. John Wiley and Sons. ISBN 1405134119, 9781405134118 Mackey, A. and Gass, S. M. (2005). Second language research: methodology and design. Routledge. ISBN 0805842497, 9780805842494 Muijs, D. (2004). Doing quantitative research in education with SPSS. SAGE. ISBN 0761943838, 9780761943839 Norman, G. R. and Streiner, D. L. (2008). Biostatistics: the bare essentials, 3rd edition.

PMPH-USA. ISBN 1550093479, 9781550093476 Wassertheil-Smoller, S. (2004). Biostatistics and epidemiology: a primer for health and biomedical professionals, 3rd edition. Springer. ISBN 0387402926, 9780387402925

9 Wood, M. J. and Kerr, J. C. (2010). Basic steps in planning nursing research: from question to proposal, 7th edition. Jones & Bartlett Learning. ISBN 0763771791, 9780763771799