MGT 5000 - Evaluating Research in Management Chapters 5, 6, 7 May 2, 2017 | Alexis Lake Question 6-1: How do you define pseudo evaluations? A pseudo evaluation is essentially what the authors call evaluations that “fail to produce and report valid assessments of merit or worth to all right-to-know audiences,” (Stufflebeam & Shinkfield, 2007, pg. 145). Essentially this means that pseudo evaluations tend to selectively present information, over generalize particularly negative findings, or even falsify some findings or conclusions from an evaluation. These may look like real or sound evaluations, but they are incomplete, skewed to certain perspective or ideas, and are often narrowly focused. The textbook also mentions that pseudo evaluations are politically motivated - where the client or the evaluator are looking to amplify a certain result or achievement other others. There are five types of pseudo evaluations: public relations inspired (a feel good evaluation focused on the positives of a program, product, etc.), politically controlled (multiple truths uncovered), pandering (driven by the evaluators to receive more client business), evaluation by pretext (the client has a hidden agenda for conducting the evaluation that is unknown to the evaluator) and empowerment under the guise of evaluation (an external evaluator aims to validate an internal evaluation because he/she believes in the greater good of the product, service, program, etc.) Question 6-5: A distorted, very positive view of a program is reported, while problematic facts are withheld. Which form of pseudo evaluation applies to this statement? The type of pseudo evaluation that applies to this statement is the public relations inspired pseudo evaluation. In this evaluation there is an intention to utilize the findings in a way that convinces or persuades a target audience that a product, program, service, etc., is effective. As the textbook outlines, this pseudo evaluation could also be called “ideological marketing,” “advertising,” or “infomercials” (Stufflebeam & Shinkfield, 2007, pg. 146). This approach addresses or focuses on only the strengths and not the weaknesses of a program. Additionally, this approach may include biased samples, biased selection of testimonials, a massaging or softening of the evaluation results, and selective release of only positive results. An real-world example given in the textbook for pseudo evaluations is around a “Lessons Learned” document around the 1991 Gulf War. This document was released by the government and was titled as a study, but really contained misstated facts that aimed to dismantle military or weapons policies, while avoiding the negatives and debates around budgets and the role of the service. Question 6-10: Under what circumstances can evaluators legitimately give private evaluative feedback to clients? According to the textbook, evaluators can legitimately give evaluative feedback to the client or clients when the evaluation is sound in merit, conforms with all applicable laws, statues, policies (whether domestically or internationally) and appropriate contractual agreements on the release and any edits around the findings. Essentially the evaluation should be ethical, legal, and honest in the disclosure of the findings and who conducted the evaluation.
Question 7-5: What kinds of questions are likely to be asked in respect to the success case method of evaluation? The kinds of questions in the success case method of evaluation would focus on uncovering the strengths or successes of a program, and trying to identify what makes something successful or unsuccessful, and then using the results to create positive. These types of summative questions are easy to ask and could include: what is working, what is not working, why isn’t something working, etc. This overall evaluation is very feasible to implement in a timely, cost-effective manner. This evaluation and type of questions focus less on the overall program and more on the successes of certain competencies or areas. In this method, the evaluation illuminates instances of success and contrasts these results with what is not working in a program or the failures, and then suggesting areas for improvement. Internally this method can help build or increase employee morale while also trying to rejuvenate programs that could be in the back burner or marginalized. This evaluation could be conducted through a focus group, case studies, or interviews. Some of the negatives of this method could be that programs or projects continue even though they are not noteworthy or nonessential to maintain or continue. Also, as the textbook outlines, this evaluation is not comprehensive in terms of the evaluation definition for merit and worth.
Reference: Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation Theory, Models, and Applications (1st ed.). John Wiley & Sons.