Meaningful Feedback in Medical Education

Best Practice and Implications for Clinical Radiology Examinations

It is commonplace to hear differing views on feedback in medical education. Trainees argue that they don’t get enough and what they do get is not useful. Trainers argue that they give large volumes of feedback but trainees do not necessarily engage with it in the right way. A 2019 article in Medical Education suggests this disconnect arises due to differing conceptions of feedback itself by learners and educators (1) .

Recent developments in the feedback literature

The academic literature on assessment and feedback in higher education has shifted in recent times away from conceptualising feedback as a justification or explanation of marks, to a focus on feedback primarily being about the ability to impact and enhance future learning (2). Thus, current thinking is that feedback should be understood as a relational process that is enacted over time. Key aspects of good feedback are: encouraging dialogue between the giver and receiver of feedback; involving peers; explicitly encouraging self-assessment and reflection; and encouraging learners to be proactive in working with feedback (3) .

“Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work” (4) .

Two major components of feedback are the quality of the information or data that is presented to the trainee, and the requirement for active engagement by the trainee. One without the other results in a mismatch, and an inability for meaningful feedback to occur. For this reason, it is suggested that the term ‘feedback’ itself should be reserved for a process that is dynamic and coconstructive. ‘Dynamic’ simply means that the feedback is not a static object, report or piece of data. ‘Co-constructive’ means that the process of feedback should be collaborative, mutually respectful and negotiated in a shared social space.

Thus, the most effective place that a good quality and meaningful feedback process can occur is day-to-day in the workplace. The nature of these collaborative conversations may be influenced by many factors, such as the trainer-trainee relationship, the organisational culture and the context in which they occur. Good quality feedback therefore relies on having an ‘educational alliance’: a shared sense of purpose and goals, working together on shared activities, and a strong mentorlike bond (5) .

In another recent paper, it is argued that “involving the learner in feedback processes is the best way to navigate the emotional responses that are reported in teacher-led feedback rituals characterised by one-way information transmission” (6). A safe and mutually respective culture is essential to this. Of course, praise and reassurance are a crucial part of the act of feedback, but the supervisor must also be prepared to challenge the trainee in ways that will help the trainee strive to become the best practitioner they can be. Difficult conversations (that are truly a dialogue, rather than a ‘telling off’) are an essential part of feedback.

Implications for high-stakes examinations

The above conception of feedback is in tension with many of the elements of feedback that typically arise from higherstakes assessments like examinations. Information that is necessarily ‘static’, such as results, grades, ratings and even comments from examiners, cannot in and of itself engage the learner dynamically. The trainee can choose to interpret the information from their own viewpoint and, if they wish, ignore information that challenges them.

This does not, however, imply that static feedback provides no value. With a point-in-time assessment, meaningful information can still be collected and fed back to the learner. But it is important to recognise the role of this type of feedback and ensure that it is used in a meaningful and effective way. The quality of the assessment data is therefore key.

In consultation with the Australian Council for Educational Research (ACER), the College is committed to providing feedback to trainees that is ‘diagnostic’ and reported in a way that is meaningful and useful. Taking the new 2022 Phase 1 Anatomy examination as an example, several options for candidate results letters are currently being explored. The first option on page 21 provides candidates with their score in relation to the passing standard based on the standard setting process.

Options for Candidate Results Letters

Option 1: Simple Pass/Fail reporting scores

Anatomy Exam

Your score: 52.3% Passing standard: 58.6% Outcome: Fail

Option 2: Pass/Fail overall and by format

Anatomy Exam

Your total score: 52.3% Passing standard: 58.6% Outcome: Fail

Your scores by format: Labelling: 84% MCQs: 75% VSAs: 40% SAQs: 20%

Option 3: Performance in relation to the standards by curriculum topic area

Anatomy Exam

Your total score: 52.3% Passing standard: 58.6% Outcome: Fail

Your scores by format: Labelling: 84% MCQs: 75% VSAs: 40% SAQs: 20%

Performance by topic area: Brain – at standard Head & Neck – well below standard Spine – below standard Thorax – at standard Abdomen & Pelvis – well above standard Limbs – above standard

Our view is that there are more meaningful ways of reporting to candidates, such as identifying areas of strength and weakness in an examination report—even if they pass the examination. Option 2 does this by breaking the candidate’s scores down by item format.

This shows that the candidate did well in the Labelling and Multiple Choice Question (MCQ) items, but struggled with Very Short Answer (VSA) questions and Short Answer Question (SAQ) items requiring them to draw upon their deeper understanding of anatomy.

Option 3 above is, in our view, better still, as more diagnostic information is provided across item types and mapped to the curriculum itself. The trainee clearly has a deficiency in their knowledge of anatomy of the head and neck and the spine, but has a strong command of their knowledge of abdominal and limb anatomy. This trainee now has more nuanced information with which to target their learning going forward. They should be proactive in seeking out more opportunities for exposure to head and neck and spinal anatomy in their training.

The College welcomes feedback from trainees and trainers alike on these presented options. If you would like to provide feedback, please email the team at CRtraining@ranzcr.edu.au.

Finally, remember it is a two-part equation

It is important to remember that the diagnostic information provided in examination letters and examiners’ reports is only one side of the feedback equation. Examination reports can provide detailed and meaningful information to the trainee, but it is how the trainee actively engages with this information that is crucial to the process of feedback. This means trainees entering into a dialogue with supervisors and directors of training in the spirit of co-construction, and reflecting on their own learning progress in light of the information received from the College.

Dr Jacob Pearce

Principal Research Fellow Specialist & Professional Assessment Australian Council for Educational Research (ACER)

References

1 Ajjawi R & Regehr G. (2019). When I say… feedback. Medical Education. 53(7):652-654.

2 Jackel B, Pearce J, Radloff, A & Edwards D (2017). Assessment and Feedback in Higher Education: Higher Education Academy, York UK.

3 Orsmond P, Maw S, Park J, Gomez S & Crook A. (2013). Moving feedback forward: Theory to practice. Assessment and Evaluation in Higher Education. 38 (2):240–52.

4 Boud D & Molloy E. (2013). What is the Problem With Feedback? In Feedback in Higher and Professional Education: Understanding It and Doing It Well. London: Routledge:1-10.

5 Telio S, Ajjawi R, Regehr G. (2015). The 'educational alliance' as a framework for reconceptualizing feedback in medical education. Academic Medicine. 90(5):609-614.

6 Molloy E, Ajjawi R, Bearman M, Noble C, Rudland J & Ryan A. (2020). Challenging feedback myths: values, learner involvement and promoting effects beyond the immediate task, Medical education 54(1):33-39.