Hilary Read
The best
assessor’s guide Apprenticeships edition
Co m an plet d el up y r da ev te ise d d
Essential knowledge and skills for vocational assessors
Contents Foreword............................................................................................................................................................................................................... 3 Introduction........................................................................................................................................................................................................ 5 Vocational assessment....................................................................................................................................................................................... 6
Part I Principles and practice....................................................................................................................................................................... 9 1 What is vocational assessment?............................................................................................................................................................ 10 The main types of assessment........................................................................................................................................................................ 10 The assessor’s role..............................................................................................................................................................................................12 Becoming an assessor........................................................................................................................................................................................15 The learning and assessment processes....................................................................................................................................................... 16 Quality assurance.............................................................................................................................................................................................. 19 2 Assessing standards and qualifications.............................................................................................................................................. 21 Who’s who in qualifications delivery............................................................................................................................................................22 The assessor’s main responsibilities..............................................................................................................................................................23 The main responsibilities of internal quality assurers ..............................................................................................................................25 The main responsibilities of external quality assurers..............................................................................................................................26 Qualifications frameworks.............................................................................................................................................................................. 27 Level descriptors................................................................................................................................................................................................29 3 Assessment planning................................................................................................................................................................................. 31 The principles of holistic planning................................................................................................................................................................. 31 Planning assessment with learners................................................................................................................................................................32 Assessing prior experience, learning and achievement............................................................................................................................32 Recording the assessment plan......................................................................................................................................................................34 Planning for formative assessment............................................................................................................................................................... 37 What the standards say.................................................................................................................................................................................... 37 4 Carrying out assessment.........................................................................................................................................................................38 Using different assessment methods............................................................................................................................................................38 Making assessment decisions.........................................................................................................................................................................40 Learning theories: domains and taxonomies.............................................................................................................................................. 41 Giving learners feedback .................................................................................................................................................................................43 Keeping records..................................................................................................................................................................................................46 Carrying out summative assessment............................................................................................................................................................49 The appeals procedure.....................................................................................................................................................................................50 What the standards say.................................................................................................................................................................................... 51 5 Keeping it legal............................................................................................................................................................................................. 52 Discrimination.................................................................................................................................................................................................... 52 Safeguarding and the Prevent Duty .............................................................................................................................................................56 Data protection.................................................................................................................................................................................................. 57 What the standards say....................................................................................................................................................................................58
1
6 Improving your practice...........................................................................................................................................................................59 Taking part in standardisation of assessment.............................................................................................................................................60 Keeping a reflective account........................................................................................................................................................................... 61 What the standards say....................................................................................................................................................................................63
Part II The assessment methods toolkit...........................................................................................................................................65 7 Observation of performance..................................................................................................................................................................68 Key considerations for using observation....................................................................................................................................................69 How to carry out observation........................................................................................................................................................................69 Recording observation evidence.................................................................................................................................................................... 70 Observation of practical tests........................................................................................................................................................................ 75 8 Professional discussion............................................................................................................................................................................. 76 The key principles of professional discussion.............................................................................................................................................. 76 When to use professional discussion............................................................................................................................................................ 78 Planning discussions.......................................................................................................................................................................................... 78 Carrying out discussion....................................................................................................................................................................................80 Giving feedback.................................................................................................................................................................................................. 81 Recording discussion outcomes.....................................................................................................................................................................82 9 Examining work products and questioning.......................................................................................................................................83 How work products and questioning work together.................................................................................................................................83 Examining work products................................................................................................................................................................................84 How to use questioning...................................................................................................................................................................................85 Work products and questioning in end-point assessment......................................................................................................................88 How not to use questioning............................................................................................................................................................................88 10 Using others and learner statements................................................................................................................................................89 When to use others...........................................................................................................................................................................................89 Who can be a witness?.....................................................................................................................................................................................90 Briefing others.................................................................................................................................................................................................... 91 Recording another person’s account.............................................................................................................................................................94 Using learner statements.................................................................................................................................................................................94 11 Projects, assignments and case studies............................................................................................................................................ 97 Definition and purpose.....................................................................................................................................................................................98 Assessing projects, assignments and case studies.....................................................................................................................................98 Devising a formative assessment scheme.................................................................................................................................................100 Designing projects, assignments and case studies..................................................................................................................................102 12 Simulations, realistic working environments and testing........................................................................................................109 Using simulation and RWEs...........................................................................................................................................................................109 Using tests..........................................................................................................................................................................................................112
2
Index...................................................................................................................................................................................................................117
Vocational assessment Assessment is an important part of the teaching–learning process at all levels of education. In workplace learning, assessment plays a key role in every learner’s programme, which means that your knowledge and competence as an assessor are crucial for encouraging learners to succeed. Because you are responsible for judging the performance of learners, you need to make sure that your assessments are sound. It is thus vital for you to develop your expertise, becoming familiar with all aspects of the assessment process so that you can make fair and accurate assessment decisions. This includes understanding the many different methods of assessment, their advantages and limitations, and how to use them. This handbook will guide you through all aspects of assessment, from the early planning stage to making final assessment decisions. There are features linked to apprenticeships if you need to focus on these, highlighted by an icon alongside them. The guide is based on best practice in assessment, and will help to equip you with all the skills you need as you develop your role as an assessor.
How to use the guide For ease of use, the guide is divided into two main sections. Part I, called ‘Principles and practice’, contains information on: • who does what in relation to assessing and quality-assuring assessment • the essential steps in the assessment process: planning, methods of assessment, carrying out assessment, and judging progress • how to explain and record your decisions, and give feedback to the learner • the law as it relates to assessment practice • your role in the standardisation of assessment practice • reflecting on your performance and keeping up to date. Part II, called ‘The assessment methods toolkit’, tells you in more detail about different assessment methods, and includes help with: • the correct method to use in a given context • the stages involved for each method • how to carry out the assessment effectively • how to record and assess the evidence gained from the assessment method you used. Throughout the guide are hints, tips, key points and activities to test your knowledge, along with real-life commentaries and examples of assessment practice. The following activities will help you identify which parts of the guide are most relevant to you. If you are new to vocational assessment, answer the questions in the first activity. If you are looking to update your practice in training and assessing apprentices, answer the questions in the second activity.
6
Key terms Standardisation of assessment: The process of ensuring that all assessors reach their assessment decisions in the same way. Vocational assessment: The process of assessing skills, knowledge and behaviours linked to a craft, trade or profession. Internal quality assurance: The process of monitoring and improving assessment practice at an approved assessment centre. Approved assessment centre: An organisation approved by an awarding organisation for offering its qualifications. Apprentice assessment organisation (AAO): An organisation approved to offer endpoint assessment of apprentices. External quality assurance: The process used by awarding organisations to approve and monitor assessment and internal quality assurance practices at centres that offer their qualifications. Reflective practice: The process of critically appraising one’s performance and making changes to practice as a result. Continual professional development (CPD): The process of keeping your skills, knowledge and practice up to date. As an assessor, this means keeping both your occupational competence and your assessment practice current.
Qualifications frameworks Qualifications in the UK are subject to various different regulatory frameworks. The frameworks are designed to help learners and those advising them decide which qualifications to take. The frameworks may look different according to where you assess, but there are common principles behind them. Each framework contains a set of levels along with level descriptors. The level descriptors tell you the level of demand associated with a particular qualification in terms of teaching, learning and assessment. Refer to the diagram on page 22 for who does what in the regulation and delivery of qualifications.
Remember The RQF replaced the Qualifications and Credit Framework (QCF). As an assessor, you may still assess QCF qualifications because candidates may still be registering for and completing them until the end of 2017.
1 The Regulated Qualifications Framework (RQF) In England and Northern Ireland, all regulated qualifications are in the RQF, including vocational qualifications, A levels and GCSEs. Under the Framework, awarding organisations are free to design qualifications as they see fit and regulate their own activities, as long as they meet Ofqual’s General Conditions of Recognition. This diagram shows the levels at which the different types of qualification sit. Some qualifications span more than one level.
Level 8
Doctorates
Level 7
Level 6
Masters degrees
Degree apprenticeships
Academic degrees
Level 5
Level of challenge
Level 4
Level 3
Level 2
Level 1
Higher apprenticeships
A levels and advanced apprenticeships
GCSE (grades A*–C), Functional Skills level 2 and intermediate apprenticeships
GCSE (grades D–G), Functional Skills level 1
Entry Level 3 Entry Level 2 Entry Level 1
There is a variety of qualifications at these entry levels, designed to prepare learners either for study at higher levels or for work Qualification size
The level of demand on the learner increases as you go up the levels. You can also have smaller, in-depth qualifications at higher levels and larger qualifications at lower levels. The time it takes to achieve a qualification depends on its size.
27
2 The Scottish Credit and Qualifications Framework (SCQF) The SCQF includes all academic and vocational qualifications offered within Scotland and contains 12 levels. As with the RQF, the levels indicate the level of difficulty or challenge of each qualification. This table shows where Scottish qualifications sit within the SCQF. National courses
Awards
National progression awards
National certificates
Higher education qualifications
Professional development awards
SVQs/MAs
Award Level 12
Doctorate
PDA Level 12
Professional Apprenticeship
Award Level 11
Masters
PDA Level 11
Professional Apprenticeship SVQ 5
Award Level 10
Honours degree
PDA Level 10
Professional Apprenticeship
Award Level 9
Degree
PDA Level 9
Technical Apprenticeship SVQ 4
Award Level 8
Advanced Higher
Award Level 7
Scottish Baccalaureates Higher
Higher National Diploma
PDA Level 8
Higher National Certificate
PDA Level 7
Modern Apprenticeship SVQ 3
PDA Level 6
Modern Apprenticeship SVQ 3
SVQ 4
Award Level 6
NPA Level 6
NC Level 6
Award Level 5
NPA Level 5
NC Level 5
Modern Apprenticeship SVQ 2
Award Level 4
NPA Level 4
NC Level 4
SVQ 1
Award Level 3
NPA Level 3
NC Level 3
National 2
Award Level 2
NPA Level 2
NC Level 2
National 1
Award Level 1
NPA Level 1
NC Level 1
Skills for Work Higher National 5 Skills for Work National 5 National 4 Skills for Work National 4 National 3 Skills for Work National 3
3 The Credit and Qualifications Framework for Wales (CQFW) The framework for Wales is called the Credit and Qualifications Framework for Wales (CQFW). Within the CQFW are entry levels 1–3 and levels 1–8 covering all qualifications.
28
Technical Apprenticeship
64
Part II The assessment methods toolkit It is important to use a variety of assessment methods formatively, not only to ensure that the learner knows what progress they are making towards their learning objectives, but also to tell you what they need to do next as they develop their skills. Similarly, good summative assessment also means using a variety of assessment methods to reach a decision about a learner’s competence, as measured against the standards. This section of the guide describes in more detail the different assessment methods, what they involve and how and when to use them. The methods are:
• observation of performance discussion – either one-to-one or in a • professional panel interview • examining work products and questioning • using others (witness testimony) and learner statements • work projects, assignments and case studies • simulations, realistic work environments and testing. If you are involved in assessment and delivery of the on-programme element of apprenticeships, you can use these assessment methods to measure progress towards gateway requirements and to prepare apprentices for end-point assessment. For end-point assessment, you can use the appropriate methods as specified in the assessment plan against the apprenticeship standard.
65
Activity: How well do I know the assessment methods? To help you get the most from the toolkit, answer the following questions honestly in relation to each of the main assessment methods. Questions
Taking account of what learners already know and/ or can already do • Do you know what to do if a learner claims they can already meet the standards you are assessing? Observation of performance • Do you know what observation involves?
Yes
No
Not sure
Turn to the following section Taking account of prior experience, learning and achievement, page 32
Observation of performance, page 68
• Are you 100 per cent confident about using it? • Can you record the results effectively? • Do you know how observation is used in practical tests? Professional discussion • Have you been trained in how to use professional discussion with learners?
Professional discussion, page 76
• Do your learners understand their role in professional discussion? • Do you and your learners know what to do before, during and after a professional discussion? • Do you know how to use professional discussion during a panel interview, and how to arrive at a joint assessment decision? Examining work products and questioning • Are you confident about using evidence the learner has produced as a result of their work?
Examining work products and questioning, page 83
• Do you know how to use work products with other assessment methods? • Do you know how to use effective oral questioning with work products? Using others (witness testimony) and learner statements Do you know… • when to use contributions from other people? • how to identify suitable people? • how to brief others? • the best ways to capture others’ accounts? • what a good written contribution from another person looks like (or sounds like if it’s recorded)? • when and how to use learner statements (for example, in end-point assessment of apprentices)?
66
Using others and learner statements, page 89
External assessment is where the awarding body sets the assignment and assessment scheme, and you follow it. For example, end-point assessment of apprenticeships is delivered independently by an external, approved assessment organisation (although it may be possible for different parts of an organisation to provide external assessment within large organisations). Closed assessment is where the criteria for assessing are very specific: there is only one right answer, or the learner has to give a specific example (from a list that you or the awarding body have specified) to illustrate a given point. Open assessment is where there is room for the learner to demonstrate what they know in different ways, and you can use your discretion as the assessor or person marking the assignment. Marking schemes are guidelines for assessors or markers on what to look for in learners’ answers and how to allocate marks. Total marks lead to overall grades.
Using projects, assignments and case studies formatively Consider the following points when deciding whether to use a project, an assignment or a case study when gauging progress in learning (formatively) or to develop learners’ knowledge, skills and behaviours.
Projects are good for: • combining and developing a wide range of skills, and for integrating activities both within and across units in a qualification, in-company standards or apprenticeship standards • developing learners’ cognitive skills to do with analysis, synthesis, evaluation and reflection • learners who possess the necessary skills to carry them out successfully (such as research skills, report writing and the ability to organise themselves) • group work, provided you are clear about who does what.
Don’t use them… • in groups to assess high-order skills (such as leadership abilities, taking the initiative and decision making) unless you are prepared to observe each learner’s contribution and give feedback in detail • as a short cut: good practical projects are time-consuming to prepare and to assess.
Assignments are good for: • applying practical skills and related knowledge and understanding, and for situations that involve task management and/or problem solving • gauging whether or not learners can work within time limits • situations where learners have to show initiative, or demonstrate that they can work without, or under limited, supervision • covering several standards, units and/or learning outcomes at the same time.
Don’t use them… • unless they are set at a suitable level for the learner (for example, you aren’t testing the learner’s ability to read and write) • as a way of avoiding teaching learners the generic skills that underpin them – for example, skills needed for the world of work, such as report writing, problem solving and working with other people; learners don’t learn these by osmosis.
'My company has just invested heavily in a project carried out by our engineering apprentices. They were tasked with the design and manufacture of a mass-flow air measurement system to assist in the development and improvement of aero engine performance. A budget of £100K was allocated to the project, which they controlled by producing, installing and helping to commission the system. Other major companies have since expressed interest in our data with a view to securing hire time.’ Lead IQA, Engineering
99
Case studies are good for: • developing and assessing process skills such as problem solving and decision making in the work-related environment • for situations where there are no right or wrong answers • showing learners the relevance of a particular set of circumstances to their work environment • introducing learners to situations they are likely to encounter in the working environment – as a development tool.
Don’t use them… • as a way of saving time or avoiding the real working environment: they are not a valid means of assessing learners’ performance under working conditions • on their own: they only tell you a small part of what the learner knows and understands. In addition, assessing or marking them can prove a challenge because of the range of approaches and responses that learners might give, so there needs to be a robust assessment scheme accompanying them.
Devising a formative assessment scheme If you are one of a team of assessors and it is up to you to devise the assessment scheme for your project, assignment or case study, you will need to agree the details of the scheme with your colleagues. The assessment scheme should take account of: • the skills, knowledge and/or behaviour you want learners to demonstrate • the condition(s) under which learners need to perform • any degree of variation in performance that would be acceptable • any marks or grades that may be allocated and the criteria to be used to arrive at these. To help you define your checklist, use the learning outcomes in the modules or units of the qualifications you are assessing, or the knowledge, skills and behaviours in the apprenticeship standard. Work across several units, learning outcomes and/or standards at a time, so that you develop a checklist that integrates the standards and reflects the way in which the learner would naturally approach a task.
Planning for the on-programme element of apprenticeships If you are responsible for planning teaching, learning and formative assessment towards the on-programme element of apprenticeships, you may find the following stages helpful. You can see where you might include assignments and projects (numbers 4 and 7). The same stages apply to planning any programme of teaching and learning that involves on-job and off-job vocational training. 1 List the skills, knowledge and behaviour elements of the apprenticeship standard. When you have completed your plan, go through it and make sure you have covered all of these. 2 Group the skills, knowledge and behaviours into common topic areas.
100
Don’t be surprised if you have several that don’t fit anywhere, or if you discover generic skills, behaviours and/or areas of knowledge that underpin all others.
Example 1
104
105
Index A achievement, assessment of prior (APA) 10, 32–3 affective learning domain 42 AOs (awarding organisations) 22, 49 APA (assessment of prior achievement) 10, 32–3 APE (assessment of prior experience) 10, 32–3 APL (assessment of prior learning) 10, 32–3 appeals procedures 50–1 apprenticeships apprentice’s learning journey 18 planning of training and formative assessment 100–2 standards 13, 32, 42 see also end-point (synoptic) assessment approved assessment centres 6, 22 approved end-point organisations 6, 22 assessment, main types of 10–12, 16 assessment decisions appeals against 50–1 feedback on 44–5 making 40–1 national occupational standards and 51 recording 47 assessment episodes 31 assessment for learning see formative assessment assessment of learning see summative assessment assessment methods deciding on 24, 38–40 observation 39, 68–75 professional discussion 39, 76–82 projects, assignments and case studies 40, 97–108 simulation and RWEs 40, 109–12 tests and testing 40, 112–16 using others and learner statements 39, 89–96 work products and questioning 39, 83–8 assessment plans and planning assessing prior experience, learning and achievement 32–3 end-point assessment plans 13 holistic planning principles 31 holistic planning tips 36 national occupational standards and 37 planning discussions 78–9
planning for formative assessment 37 planning with learners 31 planning for tests 115–16 recording plans 34–5 assessors becoming an assessor 15 main responsibilities 23 roles of 12–14 assignments 40, 98 assessing 98–9 designing 102–6 reasons for using 108 using formatively 99, 100 awarding organisations (AOs) 22, 30, 49 B Bloom, Benjamin 41, 42–3 C case studies 40, 98 assessing 98–9, 100 designing 102–3 reasons for using 108 using formatively 100 certificates 33 closed questions 86 coaching process 14 cognitive learning domain 42 continuous professional development (CPD) 59–66 Counter-Terrorism and Security Act 2015 56 Credit and Qualifications Framework for Wales (CQFW) 28 credits 30, 33 D Data Protection Act 1998 57 decisions see assessment decisions diagnostic assessment 11, 16 direct racial discrimination 54 direct sex discrimination 53 disability, discrimination on the grounds of 55 discrimination 52–5 discussions see professional discussions domains of learning 41–2 dual professionals 15 E end-point (synoptic) assessment 12, 17, 18, 74
approved assessment organisations 22 assessment methods and 39, 40, 65, 73, 77, 79, 88, 98, 110 and assessment planning 13, 16, 32, 35, 102 assessors and 13, 17, 23, 37 carrying out 49–50, 94 practical test and 75 quality assurance in 25–6 use of portfolios in 48 Equality Act 2010 52, 53, 55 evidence and assessment decisions 40–1 gathering through observation 70 and questioning 87–8 work products 84–5 experience, assessment of prior (APL) 10, 32–3 expert witnesses 14 see also witness testimony external quality assurance and EQAs 22, 26 F feedback to learners 43–6, 70, 81 formative assessment 11, 16 planning 37, 99, 100–2 records 46 within apprenticeships 17 G gateway requirements 17 guided learning hours (GLH) 30 H holistic planning 31, 36 hypothetical questions 87 I indirect racial discrimination 55 indirect sex discrimination 53 individual learning plans (ILPs) 10, 16, 18 initial assessment 10–11, 16, 18 internal quality assurance and IQAs 19–20, 22, 25, 60, 61 internal verifiers (IVs) 19 interviews 39, 79 L leading questions 87 learner statements 39, 94–6 learning, assessment of prior (APL) 10, 32–3
117
learning journeys 16, 18, 20 learning theories 41–3 legislation complying with 58 data protection 57 and discrimination 52–5 national occupational standards and 58 safeguarding and Prevent Duty 56–7 level descriptors 29–30 M mentoring process 14 methods of assessment see assessment methods O observation 39, 68 how to carry out 69–70 key considerations 69 of practical tests 75 recording evidence 70–3 using formatively 74 using summatively 74 on-programme assessment 17 on-programme element of apprenticeships 17, 18, 65, 79, 100, 110 open questions 85–6 P panel discussions 39, 79 personal learning plans (PLPs) 10, 16, 18 plans and planning see assessment plans and planning portfolios 33, 48 Prevent Duty 56–7 probing questions 87 professional discussions 39, 76 carrying out 80–1 giving feedback on 81 key principles of 76–7 planning 79–80 recording 82
118
when to use 78 projects 40, 98 assessing 98–9 designing 102–3 reasons for using 108 using formatively 99, 100 psychomotor learning domain 42 Q qualifications for assessors 15 frameworks 27–8 level descriptors 29–30 who’s who in delivery 22–6 Qualifications and Credit Framework (QCF) 27 quality assurance 19 of apprenticeships 25–6 external 22, 26 internal 19–20, 22, 25, 60, 61 of teaching and learning 20 questions and questioning 39 how not to use 88 how to use 85–8 and work products 83, 88 R racial discrimination 54–5 radicalisation 56–7 realistic working environments (RWEs) 40, 109–10, 112 records assessment documentation 46–9 of discussion outcomes 82 of observation evidence 70–3 of others’ accounts 94 reflective accounts 61–3 of simulation results 112 reflective practice 61–3 regulated qualifications frameworks 22, 27–8, 29–30 regulatory bodies for national qualifications 22 reliability of assessment 41
S Safeguarding Vulnerable Groups Act 2006 56, 57 Scottish Credit and Qualifications Framework (SCQF) 28, 30 Scottish Qualifications Authority (SQA) 22 sex discrimination 52–4 simulation 40, 109–12 SMART targets 37 standardisation of assessment 60 summative assessment 12, 16, 17, 46, 49–50 see also end-point assessment summative assessors 12–13 synoptic assessment see end-point assessment T taxonomies of learning 42–3 tests and testing 40, 75, 112–16 total qualification time (TQT) 30 trainers, assessors as 14 training providers 14 V VACS criteria 40–41, 85, 94 valid and invalid assessment 37 vivas 39, 79 vocational assessment 6 see also assessment vocational qualifications and standards 32 vulnerable adults 56, 57 W witness testimony 33, 39, 89 briefing others 91–3 recording another person’s account 94 when to use others 89–90 who can be a witness? 90–1 work products 39 examining 75, 84–5 and questioning 83, 88 work projects see projects
Hilary Read
The best
assessor's guide
Essential knowledge and skills for vocational assessors Apprenticeships edition The second edition of this comprehensive, easy-to-use handbook is for everyone with responsibility for vocational assessment, whether you are new to the role, an experienced assessor or responsible for training assessors. Completely revised and updated to reflect recent changes in assessment practice, this guide explains the role of the assessor in apprenticeships (England), how to approach the different stages of assessment, and how to prepare learners for vocational qualifications. You will find advice on how and when to use the different assessment methods, including new methods used in apprenticeships. Throughout the guide are new sections on formative strategies, to help you with both on-programme and end-point assessment. The guide brings together the expertise of many experienced assessors, with case studies and real-life examples including those related to new apprenticeships. It is an ideal companion for everyone working towards an assessor qualification or who wants to improve their skills. Hilary Read has been training vocational trainers, teachers, assessors and internal quality assurers nationally for over 20 years. She strongly believes that everyone assessing in the further education and skills sector should be equipped with the skills they need, and her guides have enabled thousands of people to feel confident that they are implementing good practice. This second edition of The best assessor’s guide follows her highly acclaimed The best vocational trainer’s guide, The best quality assurer’s guide and The best initial assessment guide.
Read On Publications Limited PO Box 162 Bideford EX39 9DP enquiries@readonpublications.co.uk
To order more copies of this publication, ring the orderline on 0844 888 7138 or go to the website, www.readonpublications.co.uk