Assessment Introduction When people think about assessment the first thing that comes to mind is probably sitting in an exam on a hot summer’s day or staying up all night drinking coffee to finish that essay due in the next morning. Assessment is often seen as the hurdle at the end of the course that has to be overcome. However, assessment can be much more than this if used in an innovative way. Formative practices, that is, assessments that do not necessarily count towards end of course or module grades, have a significant role to play in enhancing learning but are all to often marginalised and underused. It is of little surprise then that this frequent, almost utilitarian, approach to assessment is an area of concern and source of dissatisfaction for students. Across the country, regardless of course, institution type of background, students have resoundingly rated assessment and feedback as the poorest area of their academic experience in the undergraduate National Student Survey, NUS/HSBC Student Experience Report and the Higher Education Academy’s Postgraduate Taught Experience Survey. By outlining the key issues surrounding assessment, numerous case studies and NUS’ Ten Principles of good practice, this section is designed to give students’ unions and students a thorough understanding of this important academic area. The briefing will then address the role of the QAA, and finally the future, considering new and radical approaches to assessment that places it where it needs to be, directly at the heart of learning. This section is designed to ignite debate and thought, enabling meaningful discussions to take place between unions and their individual institutions that are focused on creating effective change in the area of assessment. We hope you find it useful.
Purposes of Assessment Assessment can serve a variety of purposes. Students may be most familiar with summative forms of assessment, a practice whereby assessment is used as a tool to ascribe an achieved level of learning, usually at the end of a course/module or period of learning. This commonly takes place in the form of an exam, or some other task that results in a grade or mark. In more formal contexts, for example A-levels or undergraduate final exams, this is not simply given to the student for their own personal information but also acts as a signal for wider audiences, for example employers or universities, allowing them to make judgements and decisions about that individual. However there are other, very different uses for assessment; many of which are underused. Broadly called formative assessments, this method actively engages students in their own learning, and combined with the provision of effective feedback, positions assessment as an aid to learning, enhancing understanding and progress rather than as simply a measure of it. Moreover, this does not simply apply to the subject matter itself but also to the development of transferable skills. This includes reflective and peer review, both of which are increasingly considered important and necessary skills for graduates hoping to
43
44
progress through other learning opportunities, or enter employment. Assessment practices can also be used by tutors at the beginning of a course to assess current levels of understanding and knowledge, in turn informing them how they should approach teaching the course content. Similarly, tutors can use students’ performance in assessments to reflect on teaching methods and strategies. Purposes of assessment: To prescribe a mark or grade to reflect or establish learning achieved. This can often impact on how a student progresses onto future learning or employment. A tool to enhance future learning outcomes and performance by providing constructive feedback (or “feedforward�) to students. To develop and refine key skills, such as reflective and peer assessment. Empowering students to become active participants in their learning journey. An aid allowing tutors to measure current knowledge and understanding and adjust their teaching content and methods accordingly.
Assessment: Student Opinion Assessment practices and methods, regardless of their purpose or nature have long been a concern in the higher education sector. This is true for both students and academic professionals. It was therefore little surprise that assessment and feedback was ranked the lowest area by students in all of the National Student Surveys since its introduction in 2005. As table 1 indicates, the survey found students experiences to be significantly less positive in assessment and feedback compared to the other six areas of: quality of teaching; academic support; organisation and management; learning resources; personal development and overall satisfaction.
Table 1: section results for the NSS 2005 and 2009 2005
2009
5 - The criteria used in marking have been clear in advance.
65
70
6 - Assessment arrangements and marking have been fair.
68
72
7 - Feedback on my work has been prompt.
54
58
8 - I have received detailed comments on my work.
56
63
9 - Feedback on my work has helped me clarify things I did not understand.
51
57
Table 2 breaks down these results by question allowing you to see more precise figures. Whilst the results do show an increase between 2005 and 2009 the figures are still well below the results in other sections of the Survey.
Table 2: Results for the assessment and feedback questions in the NSS in 2005 and 2009 2005
2009
The teaching on my course
77
83
Assessment and feedback
59
65
Academic support
67
74
Organisation and management
69
72
Learning resources
76
80
Personal development
75
79
Overall satisfaction
79
81
45
The NUS Student Experience Report 2008 published in November also shows that students want to be more involved in shaping all areas of their learning experience. Whilst only 23% of students currently feel involved in shaping their content, curriculum or designing their course (including assessment methods), 57% said that they wanted to be. Please visit page 62 for more information on engaging students in assessment development.
Different Types of Assessment The practice of assessment is a varied field which cannot be simply confined to narrow definitions. There are various types of assessment practices that have very different aims and outcomes. This section will discuss these broad areas and relate them to the various ways in which they interact with the learning process.
Summative Assessment: Assessment of learning. This is the type of assessment that most students will recognise. It is traditionally taken at the end of period of learning and intended to measure whether the student has reached the academic standard required for this part of the qualification and more generally whether they have got to grips with a particular subject area. A common example is an end of module exam or essay that results in a formal mark or a pass/fail. This stratification of students is a core feature of education and considered essential, not just for individuals involved, but also for employers and wider society to be able to differentiate between students. Unlike the other forms of assessment, what is distinctive about summative methods is that they are usually undertaken as a finale to a period of study, meaning the act of assessment itself is purely to ascribe a value to the students achievement rather than to foster future learning.
Common summative assessment methods: A-levels, GCSEs. Graded musical performances, practical art exams. End of module essays. Summative assessment outcomes are used by: Individual students to gauge progress and make decisions about future progression. Employers and universities to make decisions about a person’s ability, aptitude and potential, such as entry onto a course. Prospective students and their advisers as a tool to judge how well an education institution does. Institutional managers to assess teaching methods or organisational management procedures. Summative techniques are the most commonly used methods of assessment but there are concerns that there is too much reliance on them. Firstly there is a criticism that such methods, for example exams, measure exam technique just as much, or if not more, than that extent to which the subject area has been mastered. Therefore, regardless of how much someone is knowledgeable in a subject area, if they are not schooled in that particular assessment method, or do not have a natural aptitude for it, they will never perform as well as they might. It can be argued that acquisition of the skills needed for success in examinations, such as time management, working under pressure, speed of writing, are worthy; but during assessments it must be clear as to what is being assessed, the learning or the assessment technique – this is fundamental to any valid, reliable and transparent system. There are also worries that an overreliance on summative assessment can damage the morale and self-esteem of
46
some students. For example, if work is only marked at the end of a module, then any feedback or mark that is given may come too late for a student to learn from it. As summative assessment is finite, and often corresponds with little qualitative feedback, students receiving lower marks may be adversely affected and even disengaged from future learning. NUS calls for feedback to be provided on exams but it is also important to look at the greater use of other assessment methods, and it is to these methods that we turn to next. Concerns with summative assessment methods include: Used too frequently to the exclusion of other forms of assessment. Disproportionally measure how an individual has mastered the assessment technique, rather than subject understanding and knowledge. Can damage self-esteem and motivation. May not include any feedback for students to reflect upon e.g. examinations.
Formative Assessment: Assessment for learning. Whereas summative assessment aims to reflect what has been learned, formative assessment is quite different. Its primary focus is on the substantive learning itself and providing means by which progress can be made. The three characteristics of formative assessment are: It does not contribute to formal grades. It usually includes substantial and effective feedback. It is an integral part of day to day teaching rather than an isolated, finale event. Formative assessment practices are essentially similar to summative methods, including tests, projects, exhibitions,
laboratory reports and essays or similar. However, crucially they differ as they are generally conducted during the learning period, rather than at the end of it. They also may not provide the student with a finite mark but rather with a framework of what needs to be done in order to progress and an explanation of where the student has gained/lost marks, alongside an explanation of why and how to improve on this. In this sense it can help students understand where they are in the learning process, what progress they should aim for and how to make that progress. In essence therefore, this type of assessment is at the heart of the learning process and is undertaken primarily as a learning tool. It has the potential to encourage real learning and understanding of an issue rather than simply learning the skills of regurgitating information as can be required in summative assessments. It should be noted, however, that some summative assessments can give formative feedback; the two may not be mutually exclusive. However, a key issue with feedback is that it may not actually help the student to learn, but rather tells them what they have got wrong or omitted. Feedback is an area that is receiving much attention from the sector at the moment and more work needs to be done to ensure that students and staff work together to ensure that it is provided and used effectively. For more information on this area please go to the Learning and Teaching Hub on NUS Connect. Formative assessment is increasingly seen as underpinning effective teaching and learning within the state school system and is now a core feature of the teacher training syllabuses. However there is concern that these practices do not necessarily have a significant presence in the higher education sector and thus students are missing out on a valuable teaching and learning technique.
47
Uses of formative assessment: Diagnose reasons for good and poor performance. Identify strengths and weaknesses Identify problems with knowledge or skills at an early stage. Introduce students to assessment processes before the final summative assessment. Shows students where they can lose or gain marks. Helps students to apply the learning from one piece of work to another and their future learning. Improve teacher effectiveness. Strengthen dialogue between student and tutor. Examples of formative assessments include problem-solving group work, project work and presentations, often with an element of peer and/or reflective assessment. Rather than simply assign a grade or mark, outcomes here would involve greater engagement with the learning, embedding understanding and ultimately fostering further progress.
Reflective and Peer Assessment Reflective and peer assessment are a distinct part of assessment for learning. They have become increasingly significant in recent years and commonly considered key skills for graduates. Using the assessment criteria, students are encouraged to reflect, review and assess their own work, as well as the work of their peers. They are then asked to consider and discuss why they came to the decisions they did. The act of doing this encourages students to consider different perspectives and different approaches, which both embed the learning more deeply and broaden understanding.
Furthermore peer review provides the individual with numerous amounts of feedback on their work. Multiple feedback can be effective as it exposes people to differing perspectives and approaches. In addition, it can allow students to become more familiar with assessment practices, as well as strengthening bonds between students which in turn improves the learning environment. The benefits of reflective and peer assessment. When you apply the assessment criteria to your own or others work: Self-reflection encourages you to learn more deeply about that subject area. Has the benefit of exchanging ideas with others – you may find a discussion with peers is more effective than just having your tutor explain something. When you are able to explain something to someone, the act of explaining is one of the most effective ways of learning it. Empowerment of the learner in a learning environment. Development of learner ability to selfevaluate and reflect. Greater understanding of what is required by tutors for assessments at degree level. Reflection on recently completed assessments with full explanation of the answer (improving information and understanding). Seeing standards set by peers as well as mistakes of others (and avoiding them in the future). Gaining an ability to ‘stand back’ from own work and reflect.
Diagnostic Assessment Diagnostic assessment is another type of assessment for learning. Its purpose is specific; it is aimed at identifying
48
a student’s knowledge, skill set, understanding and strengths/weaknesses prior to a period of learning, for example a course or module. This usually takes place in the form of a short test or discussion group. The subsequent information gives the tutor a greater understanding of those they are about to instruct, allowing them to adapt the curriculum, teaching methods and resources to best suit individual and group needs, encouraging greater development and learning. Very little diagnostic assessment takes places in higher education institutions and often students are taught with the assumption they have a uniform level of understanding. Diagnostic tests are easy to administer and have the potential to contribute a great deal to ensuring all students are allowed to progress, regardless of the level at which they began.
Do students actually want to engage with formative assessment? When discussing different formative types of assessment it is sometimes raised that students are reluctant to engage with this type of assessment as it does not lead to a tangible summative outcome. NUS believes that the key way to involve students with any part of their learning process is to create a pro-active learning environment. In this context, student involvement should be at the heart of formative assessment design, this including: engagement with the student. relevance to the student. ownership by the student.
The Role of the Quality Assurance Agency (QAA) The QAA has produced a Code of Practice for the assurance of academic quality and standards in higher education. This Code of Practice offers guidance to universities and colleges for maintaining quality and standards and section 6 relates to the assessment of students which includes 15 precepts of assessment which institutions should take note of when formulating their own policy and practice. These precepts are: 1. As autonomous bodies, institutions are responsible for the academic standards of awards made in their name, and consequently are responsible for1: designing, approving, monitoring and reviewing the assessment strategies for programmes and awards. implementing rigorous assessment policies and practices that ensure the standard for each award and award element is set and maintained at the appropriate level, and that student performance is properly judged against this. evaluating how academic standards are maintained through assessment practice that also encourages effective learning. 2. Publicise and implement principles and procedures for, and processes of, assessment that are explicit, valid and reliable. 3. Encourage assessment practice that promotes effective learning. 4. Publicise and implement effective, clear and consistent policies for the membership, procedures, powers and accountability of assessment panels and boards of examiners. 5. Ensure that assessment is conducted with rigour, probity and fairness and with due regard for security.
1. http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section6
49
6. Ensure that the amount and timing of assessment enables effective and appropriate measurement of students’ achievement of intended learning outcomes. 7. Have transparent and fair mechanisms for marking and for moderating marks. 8. Publicise and implement clear rules and regulations for progressing from one stage of a programme to another and for qualifying for an award. 9. Provide appropriate and timely feedback to students on assessed work in a way that promotes learning and facilitates improvement but does not increase the burden of assessment. 10. Ensure that everyone involved in the assessment of students is competent to undertake their roles and responsibilities. 11. Ensure the languages used in teaching and assessment are normally the same. If, for any reason, this is not possible, institutions ensure that their academic standards are not consequently put at risk. 12. Provide clear information to staff and students about specific assessment outcomes or other criteria that must be met to fulfil the requirements of professional, statutory and regulatory bodies (PSRBs). 13. Review and amend assessment regulations periodically, as appropriate, to assure themselves that the regulations remain fit for purpose. 14. Encourage students to adopt good academic conduct in respect of assessment and seek to ensure they are aware of their responsibilities. 15. Ensure that assessment decisions are recorded and documented accurately and systematically and that the decisions of relevant assessment panels and examination boards are communicated as quickly as possible. Along with institution’s use of external
examiners, the QAA ensures that comparable standards are maintained in higher education. It is encouraging that several of the recommendations from the Code of Practice give particular reference to the importance and need for more assessment for learning. Institutions are strongly encouraged to incorporate the QAA’s Code of Practice into their institutional policy. This section of the Code is important as it sends a clear signal that a refocus of assessment to give it a greater emphasis to present it as a learning tool is desired.
Quality Assurance in the European Higher Education Area The European Association for Quality Assurance in Higher Education (ENQA) aims to promote European co-operation in the field of quality assurance by sharing, developing and disseminating good practice. Sector bodies involved in this process include the various European quality assurance agencies, public authorities and higher education institutions. In 2005, ENQA published a report that included standards and guidelines as to how assessment practices should be designed and implemented. This report, titled ‘Standards and Guidelines for Quality Assurance in the European Higher Education Area’, was “adopted” by the 45 Ministers from across Europe responsible for higher education at the Bergen Summit of the “Bologna Process”. 19/20 May 2005. The Guidelines state that “assessment of students is one of the most important elements of higher education. The outcomes of assessment have a profound effect on students’ future careers. It is therefore important that assessment is carried out professionally at all times and that it takes into account the extensive knowledge which exists about testing and examination processes. Assessment also provides valuable information for institutions about the effectiveness of teaching and learners’ support.” The full report, including the set of guidelines can be found at http://www.enqa.eu/files/ ESG_v03.pdf
50
Scottish Enhancement Themes Enhancement Themes in Scotland are a series of investigations in order to inform and enhance institutional policies to improve the experience of learners in Scottish HEIs. The Scottish Funding Council, Quality Assurance Agency, Higher Education Academy and NUS Scotland work in partnership to research and develop innovative practice within the higher education sector. The topics of the Enhancement Themes are identified through consultation with both staff and students, and a steering committee leads the work on the projects. The activities and outcomes of the themes are published on the Enhancement themes website (www. enhancementthemes.ac.uk). Although this information is collated in Scotland, the research draws from international practices, and the outcomes can be applied to any higher education institution in the world. So far seven themes have been completed which are: Research-Teaching Linkages. The First Year. Integrative Assessment. Flexible Delivery. Employability. Responding to Student Needs. Assessment. There are also two themes currently in progression: Graduates for the 21st Century: Integrating the Enhancement Themes. Quality Cultures and Systems & Structures for Enhancement. Details of all of the work carried out on these themes can be downloaded form the Enhancement Themes website including
presentations, group activities and recommendations. Graduates for the 21st Century, The first Year, Integrative Assessment, Flexible Delivery, and Assessment have all had a specific focus on assessment and/or feedback. Here are some of the publications which have addressed the issues of assessment and feedback.
Graduates of the 21st Century Theme Four recent papers on assessment and feedback with significant implications for practice - Professor David Nicol.
First Year Enhancement Theme Curriculum design for the First Year - Dr Catherine Bovill, Dr Kate Morss, Dr Cathy Bulley. Transforming assessment and feedback Professor David Nicol.
Integrative Assessment Theme Monitoring Students’ Experiences of Assessment. Balancing Assessment of and Assessment for Learning. Blending Assignments and Assessments. Managing Assessment Practices and Procedures - all written by Professor Dai Hounsell, Dr Rui Xu and Miss Chun Ming Tai.
Flexible Delivery Theme Flexible Delivery: the flexible curriculum - Ruth Whittaker, Hazel Knox, Vince Mills, Paula Cleary.
Assessment Theme - various contributors
Streamlining assessment- how to make assessment more efficient and more effective. Using assessment to motivate learning. Constructive alignment of learning outcomes to assessment methods
51
Developing a variety of assessment methods, including self and peer assessment. Assessing online. Issues of validity, reliability and fairness. Improving feedback to students (link between formative and summative assessment). Assessing personal transferable skill. As you can see there is a wealth of information on the Enhancement Themes website for you to use when formulating proposals to your institution for policy changes. Make sure you give yourself enough time to explore the website, and remember that as well as reports and policy documents, there are also presentations and activities which could help you with your chosen strand of assessment practice.
52
Grading Criteria A paper by Anthony Vickers, Lecturer at University of Essex and UK Bologna Expert
Programme Specifications and Learning Outcomes Currently in the UK all degree programmes are defined by programme specifications. This is a requirement of all universities. You should be able to find these at any UK university. Within these programme specifications you will find the programme level learning outcomes. These state what a student should be able to do as a result of completed the programme. All programme learning outcomes should be mapped to the modules to indicate where they will be assessed. This mapping may or may not be publicly available. Within modules you may or may not find module learning outcomes. These outcomes should be quite specific and in an ideal world they should be mapped to specific assessment within the module. In some universities a check is made by the academic responsible for a module to demonstrate that all the module learning outcomes map to specific assessments. It is not however a requirement for universities to use module learning outcomes. Where they are not used it should be possible to understand the relationship between the programme learning outcomes and the modules through the module description. Students should be aware of their programme learning outcomes. However this is often not the case in particular when module learning outcomes are not used. Where learning outcomes are used they should be written with careful consideration to the proposed learning of the student and the assessment. All three, the
Learning Outcomes, the Learning, and the Assessment should be designed together. A simple way to remember this is through the acronym LOLA.
The writing of learning outcomes can use a range of methodologies. For example one can use the cognitive section of Bloom’s Taxonomy2 to choose the appropriate verb associated with a particular aspect of learning such as knowledge (name, list, etc) or synthesis (predict, propose, etc) to pick two examples from the Bloom’s taxonomy. A new version of cognitive taxonomy was developed by Anderson and Krathwohl in 20013. Try using either cognitive taxonomy to construct for yourself a set of key learning outcomes using a module description of your choice as a starting point.
Assessment The assessment of any piece of coursework or examination is quite complex and often a personal judgment to some extent as it is never possible to anticipate all the ways in which a student will choose to answer a set question. There is considerable literature written on this subject and you may like to browse the list of references at the end of this section. From a student prospective they would like to see a clearly defined assessment both in terms of the question asked, the grading criteria used, and the feedback given. The important point is that all aspects should be transparent. One rarely encounters problems with the transparency of a set question although of course there can be problems with interpreting questions. You might often have said to yourself, “What does s/ he mean by that?” and gone headlong to the academic responsible for the course for further clarification. It could be argued that the clarity of the question would be enhanced if a clear grading criteria is given, indicating how marks will be assigned to particular parts of a given answer. How might we define clear grading criteria? It sits
2. Benjamin Bloom (editor), M. D. Englehart, E. J. Furst, W. H. Hill, and David Krathwohl., “The Taxonomy of Educational Objectives, The Classification of Educational Goals, Handbook I: Cognitive Domain”, 1956 3. Anderson, L. W. and David R. Krathwohl, D. R., et al (Eds..) “A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives.”, 2001
53
somewhere between nothing and a fully worked answer showing the distribution of marks. Neither of these are useful in terms of assisting the student in undertaking the assessment. Clear grading criteria indicates to the student where they should concentrate their efforts in order to secure the marks available. They can be written in a specific way or in a generic way. For example, “5% is awarded for writing down the appropriate formula and defining all the terms in the equation”, is a very specific grading criteria, whereas, “successful completion of all requirements of the assignment/report, with initiative used to carry out some extensions to the basic requirement. Clear evidence of understanding all aspects of the work. Excellent presentation, using concise clear English (good grammar and spelling) and diagrams (where appropriate).” is more generic and in this example refers to the 1st class honours criteria for a final year project. Grade distributions for modules are interesting to analyse. These are automatically generated by most Management Information Systems in universities and can be used to monitor anomalies in grading. They can also reveal the extent to which the full range of grades (0-100) is used. It is not uncommon to find broad distributions for science modules and narrow distributions for arts and humanities modules. It is often said that this is due to science modules having very definite answers and therefore it is possible to award the full range of grades. This may in part be true but it may also be the case that the practice of providing grading criteria in science based modules is more common than in arts/humanities modules. It may be the case that some module grading criteria (published or unpublished) preserve top grades (80%+) for the rare occasion when a student provides an answer of outstanding quality. If this is the case then at least this should be transparent by publishing the grading criteria. It is not acceptable to surround these top grades in mystique.
Feedback Lack of feedback is one of the main complaints by students at all levels of education in all parts of the globe. There are many reasons for the lack of feedback but none of them are acceptable. Feedback is essential if a student is to progress. The main thing to note about feedback is that it needs to be planned from the outset. How feedback will be given should be known at the time the assessment is designed and indeed is better seen as a component of the design of the assessment. In a learning outcomes approach the assessment is designed to measure to what extent a student has met the learning outcome(s) and the feedback should indicate how the student has or has not met the learning outcomes(s).
Conclusion In assessing a module students should look for the following components. A clear set of aims and objectives or learning outcomes. An indication of the modes of learning. On issuing the assessment the grading criteria that will be used should be stated set against the local scale which is commonly 0-100 in the UK. A description of the method of feedback.
References O’Donovan & Price & Rust, “Developing Student Understanding of Assessment Standards: A Nested Hierarchy of Approaches” 2009 Price & Carroll & O’Donovan & Rust, “If I was going there I wouldn’t start from here: a critical commentary on current assessment practice ”, 2010 Sadler, “Indeterminacy in the use of preset criteria for assessment and grading ”, 2009 Sadler, “Interpretations of criteria-based assessment and grading in HE”, 2005
54
Liberating the Curriculum There are a number of benefits that “equality proofing” the curriculum can bring. For society at large, it ensures that further and higher education continue to act as an agent for liberal and democratic values, encouraging social cohesion and mutual understanding of different viewpoints. It also benefits the individual by allowing them to reflect on who they are and their place in society. Indeed, if individuals are not able to do this, it can be argued that it stunts their personal growth and reduces their employability.
So how exactly do we liberate the curriculum? There are 2 strands to looking at equality in the curriculum: 1. Ensuring all students regardless of their profile, have access to the same resources and facilities and ensuring students feel safe, giving students equal chances of success. 2. Bringing up issues of equality as part of the curriculum. Ensuring that case studies and resources reflect our diverse population and a curriculum which aims to encourage social inclusion and education about different profiles of people. One or both of these things may be something which you may want to think about when running a campaign on assessment, your campaign could investigate how your institution ensures that their curriculum considers equality in their curriculums and help to shape policy in this area or actively encourage opportunities for students to bring their experiences and perspectives into the classroom and use them as an integral
part of the learning process though engaging them in curriculum design. Whatever path you choose to take, we hope these next few pages will give you some indication on the type of resources already available. After the Equality Act 2010 passed in April, new measures will be introduced over the coming years which will have a direct impact on institutions and students’ unions. The Equality Act 2010, consolidates an streamlines current discrimination laws and aims to strengthen the law to support the progress of equality. The Equality Challenge Unit have a dedicated page on their website on how institutions and students unions will be affected by the new legislation, including a specific briefing on the implications for students’ unions: http://www.ecu.ac.uk/subjects/ equality-act-2010. The Equality Challenge unit has also done research on ethnicity, gender and attainment. As well as finding an achievement gap, its findings suggested that further linkage and coordination is needed at governance, strategic and curriculum development levels between principles and practice of equality and diversity, and learning, teaching and assessment functions…More research and development activities are needed to strengthen demonstrably fair, inclusive and helpful assessment and feedback regimes for all students.4
Diversity Audits Dr. Charles Gore (Lecturer in the History of African Art) and Deb Viney (Diversity Advisor and former lecturer in Human Psychology) of the School of Oriental and African Studies, University of London, anticipated in their 2006 HE Academy Symposium that diversity audits may become more widespread within the HE
4. ECU and HE Academy Ethnicity and Degree Attainment Project Report, 2008. http://www.ecu.ac.uk/ publications/files/ethnicity-gender-and-degree-attainment-project-final-report.pdf/at_download/file
55
sector although they recognise that “It is, naturally, impossible to achieve inclusion of an infinite range of issues, but units and their participants can benefit from
a process of consideration of what more could be done.”5 In their symposium, they outline some examples of questions a diversity audit may ask:
1
Are the learning outcomes expressed in a fashion that does not introduce unnecessary or unintended barriers? Are they sufficiently flexible to permit “reasonable adjustments” to meet specific needs when necessary?
2
Does the unit (explicitly or implicitly) address an appropriate range of diversity issues in connection with the subject material?
3
Do the supporting materials (course handbooks etc.) include information about the institutional resources and systems available to support students who have additional needs? Do they include an “Equal opportunities / diversity statement”?
4
Are the teaching materials presented in formats which are easy to read?
5
Are teaching materials made available in alternative formats (e.g. electronic)?
6
7
What use is made of electronic aids such as posting material in a Virtual Learning Environment [VLE] such as Blackboard? If case studies are used do they draw on a full range of examples from all strata of society? Are assessments all dependent on reading / writing skills? What other skills could / should be assessed?
8
Does the teaching draw on the full range of learning styles and utilise a range of accessible resources?
9
Does the programme recruit from the full range of students?
10
11
Do all students have positive experiences on the unit? Are there any differences in experience between student groups? How do staff experience the unit? Are there diversity issues for teaching staff?
5. http://www.heacademy.ac.uk/assets/York/documents/ourwork/inclusion/wp/web0393_infinite_diversity_ in_infinite_combination.doc p2
56
They used these questions to do an audit on their African Art Part 1 course and used the following process to answer the questions above:
6. review course recruitment, retention and achievement (e.g. diversity variable statistics) and associated action plans.
1. review course learning outcomes to check for any unintended barriers or diversity issues and for flexibility to make reasonable adjustments as required.
7. review course assessment strategies for skills / knowledge being tested; variety of assessment; availability of alternative forms of assessment; etc.
2. check course materials – modify to make easier to read. i. font – sans serif preferable e.g. arial. ii. font size – preferably a minimum of 12 unless electronic version is easily available. iii. justify – to the left margin only, right margin ragged. iv. spacing (min 1.5 line spacing but also general spacing of material) and text layout – for ease of reading. v. highlight important information such as assignment questions. vi. paper – off white or pale colours. vii. organisation – use page numbers, headings and labels. viii. is electronic format available? ix. is appropriate use made of a virtual learning environment [VLE]? 3. check supporting materials (course handbook, publicity materials etc.) for inclusive language and appropriate inclusion of diverse images and equal opportunities statements etc. 4. evaluate case histories / other course content for representation of men / women, minority groups and diversity issues. 5. consider diversity issues which may / should arise in the course content (see discussion document attached).
8. staff and student feedback consider the means by which feedback is obtained and whether these have / are likely to provide opportunities to express views which may lead to genuine changes. Institutions in Scotland asked for guidance on how to comply with the then amended Race Relations Act 2000 and the Scottish Funding Council and Universities Scotland produced a useful toolkit for institutions to monitor compliance. The Toolkit is not prescriptive. It does not provide a blueprint for how race equality should be addressed in learning and teaching but it encourages the academic staff to self‑evaluate, and to review the curriculum and their teaching and assessment methods, in order to create as inclusive a learning environment as possible.6 Although the toolkit deals mostly with racial equality, it gives institutions the opportunity to review all aspects of their institution and offers many potential enquiries a review can take as well as tips of how to integrate discussions into the curriculum. Its appendices also offer great insight into websites and further reading around equality issues. With particular regard to assessment the toolkit asks: 5.1 How can procedures be put in place to ensure assessment requirements are understood by all students?
6. Race Equality Toolkit, 2000, Page 5 http://www.heacademy.ac.uk/assets/York/documents/aboutus/ scotland/Race_Equality_Toolkit2006.pdf
57
5.2 Can assessment instruments and procedures be adapted to promote race equality? 5.3 Can assessment instruments and procedures be re-examined to encourage inclusion? 5.4 Does scheduling of assessment take into account religious observances and demonstrate awareness of the needs of students in class as far as possible? 5.5 Do all students understand what plagiarism means?7 From p55 onwards it also asks institution a whole host of useful questions, from curriculum design to staff training. Another useful resource when looking at equality issues in the curriculum is the QAA code of practice for disabled students. Page 25 of the code states: Staff should be aware of their responsibility to design inclusive programmes and should seek training and ongoing support where necessary. Staff should be given access to sources of advice both from within the institution and externally.8
NUS Women’s Campaign Liberating the curriculum and anonymous marking Gender segregation does exist in education and there are many subjects that are dominated by either men or women and therefore minimises the opportunities for students to take up courses that would not traditionally be aimed at them, based on their gender. Liberating the curriculum and ensuring that women and men feel they have access to all courses would be a big step to ensuring that equality of opportunity was offered to all genders in education. In the same vein, to ensure that women and men are able to take up a course and be treated fairly during the assessment process, anonymous marking is essential to ensure that no gender bias exists and attitudes and perceptions cannot be a part of the process.
“Research has demonstrated that disabled students are less satisfied when compared to their non disabled peers when it comes to receiving adequate support from their intuition. An accessible environment is crucial for disabled students to fulfil their highest potential and therefore fairly marked exams and assessments are crucial in order to remove discrimination.”
From the perspective of women students, whilst legislation might support equality for women, we know from research and evidence that women still suffer from sexism in everyday life. In education, we recognise that there is a lack of women role-models i.e. only 13% of vice chancellors are women (The Educational Backgrounds of Vice Chancellors, The Sutton Trust, November 2008) and there is also gender-bias in the material used by students to carry out there course i.e. Recommended literature for studies is more likely to have been produced by men (with some notable exceptions such as nursing as it is a course that is dominated by women, due to gender-stereotyping). Liberating the curriculum would change this gender-bias and ensure that women felt they could go on to be academics and vice chancellors, without being a minority or seeing these roles as being reserved for men.
Rupy Kaur, National Disabled Students’ Officer
Olivia Bailey, NUS Womens Officer
The code also talks about ensuring the accessibility of feedback as well as ensuring a flexible approach to assessment, making sure that all students have the same opportunities to succeed.
7. Race Equality Toolkit, 2000, Page 49 http://www.heacademy.ac.uk/assets/York/documents/aboutus/ scotland/Race_Equality_Toolkit2006.pdf 8. Code of practice for the assurance of academic quality and standards in higher education, Section 3: Disabled students, 2010 page 23
58
Useful Resources The Association of American Colleges and Universities (USA): Advice on Effective Curriculum Transformation (The Diversity Web)
www.diversityweb.org/Digest/W97/advice.html This web page links into a larger website (www.diversityweb.org). The site is designed to act as an interactive resource hub for higher education and claims to be ‘the most comprehensive compendium of campus practices and resources about diversity in higher education that you can find anywhere. The Education Alliance at Brown University: The Diversity Kit – An Introductory Resource for Social Change in Education
www.lab.brown.edu/pubs/ diversity_kit/index.shtml This publication focuses on human development and cultural diversity. It explores issues of diversity in education that are essential for teachers who are committed to diversity and quality education for all students. Leeds University Anti-Racist Toolkit
www.leeds.ac.uk/CERS/ toolkit/Section%20One.htm Seven Principles for Good Practice: Enhancing Student Learning
(Winona State University) www.winona.edu/president/seven.htm University of Strathclyde: TEACHABILITY
www.teachability.strath.ac.uk The Teachability project promotes the creation of an accessible curriculum for students with disabilities through making freely available informative publications for academic staff. For more information please contact David Malcolm, Head of Social Policy, NUS david. malcolm@nus.org.uk
59
Assessment, feedback and engagement: challenges and opportunities for students of the arts
especially when those supporting the creation of work are also the primary judges of quality. Lack of clarity and transparency can be a problem with assessment criteria in the arts and some students said that they would benefit from greater support enabling them to use criticism to positively inform the development of their work rather than just offering a summative judgement. It can be difficult for some students to know how feedback on work translates to assessment judgements and how to use feedback to improve their marks.
With the vocational nature of many courses, borne out by the employment statistics, there can be huge benefits in terms of preparation for working in specific industries. Some course, however, may take this approach to an extreme with insufficient positive reinforcement for students. Opportunities for curriculum and assessment co-design might provide additional opportunities for students to develop skills which complement those gained through academic activities.
The Arts Group exists to represent the interests of students of arts subjects in the UK. The Art Design Media Subject Centre of the Higher Education Academy works across its named disciplines to support positive academic development. While all courses are different in their organisation and delivery, there are many common themes which run across many arts disciplines and should be considered when evaluating courses at discipline level: Personal interaction between students and academic staff is often a great positive feature of arts courses but this can also have its downsides: this can create a perception of excessive subjectivity in the assessment process,
Quality of facilities can vary widely both between, and within, institutions which can have a significant impact on the success of an individual student or a cohort. Students should be involved in decisions about the best use of the resources available to them: some students said that sense of space was a very important part of their educational experience. There is also a tension between students and their institution with regards to who supplies the art materials. The cost of materials can be expensive, and many institutions do not make clear which materials will be provided and which ones will have to be sourced by students. Information given to students (handbooks, induction sessions, training, reactive support) should be clear about standards and expectations
60
and avoid using confusing or unclear language. Peer assessment in the form of critiques can pose challenges for students in intimidating and highly competitive environments and further training could be given to support students in making the most of these opportunities.
The Future of Assessment There are a variety of aims and uses of assessment, some of which students will be more familiar with than others. The unequal application of assessment methods, along with poorly managed implementation has resulted in students expressing deep concern and dissatisfaction, on occasions damaging self-esteem and motivation. As highlighted in the NSS it is clearly a worry for students and professionals alike. This is a particular concern as we know how integral assessment is, not only to learning outcomes but also to the ability to fulfil learning potential. Current practices fail to recognise the value of assessment for learning and this is to the detriment of all students. If learning is to be prioritised then assessment practices need to reflect this and become more applicable and relevant to the modern education system. Change can be slow in the higher education sector but a new approach is required if meaningful outcomes are to be realised. NUS calls for a clear and radical change within the higher education sector, repositioning assessment to encompass wider notions of learning. It is only when assessment for learning is fully embedded into practices across the sector that this will be achieved and the potential for student learning be developed and maximised. The following page outlines NUS’ ten principles of effective assessment practices. These can act as a guide, enabling the higher education sector to rediscover assessment in its fullest form and enhance the learning experience for the vast majority of students.
NUS' Principles of Effective Assessment Practice Assessment‌ 1. Should be for learning, not simply of learning.
This positions assessment at the heart of learning rather than a simple add-on at the end of a learning period.
2. Should be reliable, valid, fair and consistent.
This is crucial for staff, students and employers to have confidence in the assessment processes and their outcomes.
3. Should incorporate effective and constructive feedback.
Effective feedback on assessment is a crucial aspect of assessment processes and a key feature of enhancing the learning process.
4. Should be innovative and have the capacity to inspire and motivate.
Formative assessment practices have the potential to inspire and motivate and this aspect can be captured by innovative approaches such as with the use of technology.
5. Should measure understanding and application, rather than technique and memory.
Assessments need to have a holistic approach that transcends the particular method being used; only this will truly test and reflect levels of learning.
6. Should be conducted throughout the course, not simply positioned as a finale event.
Positioning assessment as an integral part of the course to facilitate learning throughout the course.
61
7. Should develop key skills such as peer and reflective assessment.
Not only do these mechanisms allow the student to receive extra feedback on work beyond that of their tutor, it also develops the key skills of self-reflection.
8. Should be central to staff development and teaching strategies, and frequently reviewed.
Assessment processes need to be innovative and responsive to learner needs, as such they need to be central to staff development and teaching strategies.
9. Should be of a manageable amount for both tutors and students.
While assessment should be placed as a central role to learning, a level of balance is required for it to be effective. Over burdening the tutor or the student is not conducive to this.
10. Should encourage dialogue between students and their tutors, and students and their peers.
It is important that students and staff share the same definitions and ideas around standards and this can be fostered by increased dialogue and engagements.
62
Engaging Students in Assessment Development Engaging students in their education can take a wide variety of forms: an individual student might take a particularly strong interest in their study, might represent their peers on a staff-student committee, fill in feedback forms or participate in a periodic review. One key way to involve students is through sharing the design process for the curriculum, including assessment methods. In practice, engaging students in assessment design means negotiating with a cohort and/or its representatives to establish the ways in which all sides believe that the most effective assessment of skills and knowledge can be achieved. While there may be fears that students ‘will choose the easy option’, experience shows that this is not necessarily the case: the process of designing assessment can be a tool for skills development and reflective learning in itself and by setting a clear framework for the rigour of the outcome of the negotiated process, the quality of the process can be assured. In many instances, minor changes to programmes can be the result of a staff-student dialogue based on course representatives and module evaluation feedback as well as national survey such as NSS and informal discussions. More major curriculum review might require a more separated approach which engages in-depth with a larger number of students, such as using focus groups to probe issues and identify solutions where impasses occur. Students can be tasked with leading this process: a course rep, for example, might undertake research into the views of their peers on the most effective forms of assessment, what they benefit from most in terms of marking criteria and the timing of practicals, exams or coursework. When involving students in designing assessment, thought should be given to the evaluation of the processes involved and the
resulting methods: it should be possible to identify practices which have worked well in parts of an institution and translate these to other departments. With the assessment itself, there should be feedback on its implementation in order to inform future negotiations of the curriculum. There may be students who do not like the outcome of a negotiated approach to curriculum design. While a thorough and wide-ranging consultation should minimise any negative impact, it may be necessary to provide evidence for how the result was arrived at to ensure that there is a compelling case for experimenting with variations in assessment.
Student Engagement Toolkit The Student Engagement Toolkit will be launched in September 2010, alongside additional resources on NUS Connect. The materials will help institutions and students’ unions enhance their engagement activities through gathering feedback, supporting representatives and involving students in curriculum design. This work is part of the ongoing NUS-HE Academy project, which will also be hosting two conferences in November 2010.