DETERMINING ATTAINMENT OF CAPABILITY

Assessment of ICT capability is vital as it allows for you to determine and track the progress of students and then to use that information to plan appropriately a path for them to progress. As ICT is predominantly associated with practical outcomes it needs to be embedded into meaningful subject-related context. Therefore, the best way to assess a student’s ICT capability is to give them something interesting to do and then to monitor the approaches they use when completing the task. It is through your own observation of the student in all contexts as an ICT-integrated curriculum is vital.

There are several ways of measuring the achievement of students in ICT capability. These include:

• Observing how a child goes about a piece of work; • Diagnosing difficulties which become apparent over a series of lessons; • Observing which planning strategies appear to work and allow the student to succeed in a given area; • Collecting significant pieces of work in a portfolio of development; • Noting the context of the work and any factors which were significant: the grouping, the time taken, the level of concentration etc.; • Noting the view of the student about the piece of work and asking her or him what made the activity so successful/significant; • Feeding the information back into the planning process; • When appropriate, making a judgement about the student’s level in terms of the level descriptions in the attainment targets for ICT capability in the Learning Continuum of the Australian Curriculum (at the end of the key stages in the primary school – Foundation, Level 1-4); • At all times keeping a clear focus on the objectives; this is very important as having a general awareness of other learning taking place. (Adapted from Potter, Sharp, Turvey, & Allen, 2012)

How to I design quality assessment activities?

The finished product does not represent a fair picture of a student’s ICT capability. It is only the final element of a more complex process. In this section, I will discuss how important it is to design ICT activities that promote the learning of ICT techniques and growth of higher order skills. A properly constructed and designed activity will enable you to effectively plan and assess a student’s ICT capability and track their progress.

ICT capability is more than just the ability of a student to use ICT techniques. It also involves students having conceptual understanding and making use of higher order skills. For a student to be ICT capable they need to be able to carry out the sets of processes defined in the target level descriptions of the Learning Continuum of ICT capability and this means having a good understanding of the routines, concepts, processes, techniques and higher order skills they have used.

How do the ICT Capability components combine in practice?

Imagine you are producing a worksheet for students in your class. The worksheet would contain an eye-catching heading, images to illustrate the key points and some empty boxes for students to fill in the answers. There would be at least seven steps involved. Below Kennewell (2004, pp111) outlines what is involved.

Figure 5. ICT Capability components in practice. When designing your ICT activity or task for assessment purposes these are the steps that are needed to be considered.

How do I structure activities for assessment?

In addition to the above steps, Lewis (2008) demonstrates an ideal approach for structuring activities. Using the example of a multimedia software program such as Microsoft PowerPoint she lays out what I believe to a good representation of the key assessment items. The development of ICT capability stems from students being able to demonstrate two of the main components discussed earlier. ICT techniques and higher order skills need to be highlighted and it is my belief that by singling them out at the beginning you will have a clear idea of what you need to assess and give the students a better picture of what is needed to be achieved.

Higher Order Skills

Plan how ICT techniques might be used

Monitor the progress of the activity

Recognise when the use of ICT might be effective Insert text in a text box, modify text.

Use the arrow key to move the insertion point.

Press the Insert key on the keyboard (with help from an adult)

Highlight and delete text.

ICT Techniques

• Recognising when the use of ICT might be appropriate or effective; • Planning how ICT resources, techniques and processes are to be used in a task; • Conjecturing, discussing and testing the strategies and data to be used; • Monitoring the progress of problem-solving activities; • Making and testing hypotheses; evaluating the outcomes of using ICT for a task; • Explaining and justifying the use of ICT in producing solutions to problems; • Reflecting on the leaning that might have occurred during the task.

What methods can be used to capture the application of student’s ICT capabilities?

It is important that the approach you choose for assessment fits in well with your teaching approach. Your understanding of what constitutes as progression in ICT capability should form part of this approach as it not only lays the foundations for effective and efficient assessment but will result in significant gains in attainment.

Summative or Formative methods? Which is best?

There are two different categories of assessment practices: summative and formative. Summative assessment is to do with representing students’ knowledge, skills and understanding over a long period of time. A good example of this would be the end-of-year tests. They are useful for making broad comparisons of performance between individual students within a year. Despite this, research indicates that the main disadvantage is that they “aggregate too many quite different facets of attainment to provide students and teachers with information concerning how they might improve attainment” (Kennewell S. , 2004, p. 161).

Ideally, it is formative assessment practices that cater to the needs of an effective and efficient assessment of ICT capability as it involves a more detailed and qualitative evaluation of students’ ability and progress. It also provides more benefits for the day-to-day teaching and learning. Formative practices must occur continuously if there is to be improved standards of achievement through maximised learning and student progress. It is assessment for learning as opposed to assessment of learning and is “vital because of its role in individualising learning for individual students and groups” (Finger, Russell, Jaimeson-Proctor, & Russell, 2007, p. 222).

Assessment for Learning: Why assess a student’s ICT capability?

As discussed, this approach enables you to pitch your teaching appropriately. The key assumptions about it are highlighted below (Kennewell S. , 2004, p. 162). Assessment for learning:

• Is embedded in the teaching and learning process of which it is an essential part; • Shares learning goals with students; • Helps students to know and to recognise the standards to aim for; • Provides feedback, which leads students identify what they should do next to improve; • Has a commitment that every student can improve; • Involves both teacher and student reviewing and reflecting on students’ performances and progress; • Involves students in self-assessment.

• As processes subsumes all other elements, it might be thought that simply making a judgement in terms of what processes a student can undertake is all that is needed i.e. all we need to do is to assign a level.

However, just knowing a level, or even a description of the student’s ICT capability in terms of processes, would not really help in designing appropriate learning experiences. A teacher would normally want to keep track of the student’s knowledge of techniques; • Assessment solely in terms of processes is not sufficient but necessary as you need to have some idea of the student’s position in relation to progression in ICT capability in the primary phase of schooling.

As discussed earlier, progression in the Australian Curriculum takes the form of what is in the Learning

Continuum.

Using ‘WALT’, WILF’ and ‘TIB’

As a primary educator, you would be familiar with this system in work. Using this system would be one of the key things that you can do to successfully assess students’ ICT capability. It will ensure that even the younger children in the classroom will have clear understanding of the lesson objectives and what their role will be. You can further support this by using open-ended questioning techniques. Remember to leave enough ‘waiting time’ for students to encourage them to provide longer answers and this will supply you with more evidence of deeper understanding. Allowing them time to reflect on their learning is important as it further develops their higher order skills. In addition, “consider as to what extent the students discussed the activity, whether each student had a fair part in the activity, whether any problem-solving techniques were used, and how long the children were on task” (Williams & Easingwood, 2007, p. 30).

What do I need to know about students and why?

In order for you to plan appropriate learning experiences it is essential that you maintain a reasonable amount of information relating to the composition of ICT capability. In other words, every component that constitutes ICT capability – routines, techniques, concepts, processes and higher order skills – needs to be assessed individually in order for you to ensure progression and continuity of student capabilities.

The best way to assess a student’s ICT capability is by monitoring and observing these components and using different strategies for each of them.

Techniques

For this component, it is best to demonstrate them either in a whole-class or small group setting. The reason is that you want to know if a student can perform a technique after such a discussion. For example, when demonstrating how to use a bee bot I went through and discussed with the whole class how to do so and then I divided them into smaller groups and briefly went over it again. Then I allowed the student to undertake the task themselves.

In this situation, it is ideal to have a method for you to be able to record their ICT skills in order for you to keep track of which students are confident in using the technique. It is important to remember to provide the minimum amount of support to students who need help and that this support to be withdrawn as soon as possible. Use a coding system to identify the three different stages acquisition (see below for an example).

Need considerable help Need some help Confident user

Routines

Routines are techniques being done without any conscious thought and they are recognised by the ease and speed with which familiar techniques are executed. For example, while assessing student techniques when using a bee bot, I included three columns that would let me know if a student was hesitant, steady or fluent. I then used this information to plan opportunities to move student from hesitant to steady or steady to fluent in a technique. All my judgements were obviously based on observation.

It is a good idea to have for each age group a brief checklist of techniques that the students will be able to use routinely.

Concepts

As discussed earlier, conceptual understanding forms an integral part of ICT capability. You would need to know then if a student has any misconceptions and this can be achieved by using an error analysis technique. In other words, note down any mistakes or misconceptions and further check for understanding using effective questioning and discussion.

Processes

When a student is able to carry out a process then they are also capable of making decisions. This is part of their higher order skills and can be judged on the extent to which scaffolding was necessary. According to Kennewell et al. (2000, p95), “an understanding of relevant concepts is needed order to analyse a situation and identify the particular techniques required to reach the desired goal.” Therefore, a simple checklist or tick box is not appropriate and you would need to know, for example, if a student was able to make a decision about:

• Which media to combine, and in what way, in order to present some particular information to a specific audience; • Which series of techniques to use in order to follow a line of enquiry to prove or disprove an hypothesis; • Which techniques to use in a graphics program in order to produce a portrait that visually represents feelings?

This type of information would help you to ensure that students understand the concept of ‘audience’ and not just being able to use techniques like adding text, graphics or sound to a page.

If you intend to asses a process, then you will need to focus on a group and observe how they can execute a task. You may need to at times intervene and ask key questions.

Self-Assessment: How effective are they?

Another way of tracking of the skills is by using self-assessment. Self-assessment is a useful life skill and can aid in the reflection and development of general metacognitive skills (Kennewell, Parkinson, & Tanner, Developing ICT capability, 2000). For senior primary in their final year, these assessments needs to include more detail than the formal teacher assessment record so that they know and understand what is expected. In doing so it will provide valuable information to the secondary school of their choice. I will discuss this in more detail later in Using Assessment Effectively. If you provide students with more freedom to use ICT in appropriate ways and make them responsible for recording their progress, then many of them are likely to be more motivated. In addition, by giving them an indication of the range of ICT techniques that they will need to demonstrate over a period of time will point them in the direction in which they need to be heading.

professionally produced and be tabular in structure. It is important that the language and terminology you use be appropriate for the students involved. It has have spaces for the students to indicate that they have demonstrated the particular technique as well as space to show the activity that was involved. Furthermore, you need space to write comments and to confirm that the student has actually completed the task (Ager, 2003).

Ten tips to remember about how to use self-assessments (Higgins, Packard, & Race, 1999, pp. 84-85):

1. Identify small steps: break down the ICT task that you have planned into a series of relatively easily attained targets that the students will need to achieve; 2. Make these ‘can do’ statements; 3. Involve the students: tell the students what they are going to be doing and introduce them to the steps they need to take; 4. Make sure they can do it; 5. Agree on targets: very useful as you may find that your students are more capable than you think and you might have to set higher challenges like more difficult techniques or processes for them to achieve; 6. Make a record sheet for the students to complete: design an easy way for them to record these targets and when they achieve them; 7. Remind them of the targets; 8. Help students to decide success; 9. Did they get it right? : evaluate how accurate you think their judgements are; 10. Continue the process: continue this as a cycle of target setting and evaluation.

How do I assess higher order skills?

A student must use their higher order skills in order to develop their capabilities in ICT. It is the decision by the student to determine whether a software or hardware is appropriate, a technique is appropriate or both are appropriate to use for an ICT solution to a problem. Questioning and discussions about how and why a student did something is vital in any assessment.

There are three main principles involved when assessing higher order skills. According to Brookhart (2010, p23), these are:

• Present something to students to think about, usually in the form of introductory text, visuals, scenarios, resource material or problems of some sort; • Use novel material – material that is new to the student, not covered in class and thus subject to recall; • Distinguish between level of difficulty (easy versus hard) and level of thinking (lower-order thinking or recall versus higher order thinking), and control for each separately.

In the development of ICT capability, the use of ICT demands higher order skills such as: • Recognising when the use of ICT might be appropriate; • Planning how ICT resources, techniques and processes are to be used in a task; • Conjecturing, discussing and testing the strategies and data to be used; • Monitoring the progress of problem-solving activities; • Making and testing hypotheses; • Evaluating the outcomes of using ICT for a task; • Explaining and justifying the use of ICT in producing solutions to problems and; • Reflecting on the learning that have occurred during the task. (Kennewell, Parkinson, & Tanner, 2000)

Another approach that you may consider when assessing a student’s ICT capability is that offered by Bennett (2007, p. 59). It is an information-handling system that focuses on statistical investigative methods. His method is based on what he believes to be assessment cycle for ICT (see diagram).

Figure 4. The Assessment Cycle for ICT (Bennett, 2007, p58)

How I identify what I want the students to learn?

It is important to identify your learning objectives for every ICT activity and subtasks in terms of the routines (skills), techniques, concepts and higher order thinking. To best clarify this further, I will use an example from the Australian Curriculum (English) Year 1 and create an activity that may suit its criteria. This approach is adapted from Bennett’s (2007) detailed advice.

Year 1 English (Literacy Strand) Creating Texts Create short imaginative and informative texts that show emerging use of appropriate text structure, sentence-level grammar, word choice, spelling, punctuation and appropriate multimodal elements, for example illustrations and diagrams.

Construct texts that incorporate supporting images using software including word processing programs.

Activity/Project: Students are to recreate a page of a story they have read using a word processor.

By the end of this activity, students will be able to achieve the following:

• Routines/Skills: Enter text with a keyboard; embolden headings; save and print their page; • Techniques: Import images and position them appropriately for the text; enter, format and edit text to conform with their section of the story; • Concepts: Explain how text and images can be used to communicate an episode or event in a story;

• Higher order skills: Decide which image(s) from a limited section could best illustrate an event; choose the most appropriate font styles for text to convey meaning (e.g. bold to indicate shouting, italic for emphasis).

How do I decide what evidence is needed?

Using the same example from above, you will need to ensure that the desired and expected outcomes exemplify the learning they represent Bennet et al. (2007). The learning objectives for this type of activity might include the following:

• Routines/Skills: Expected outcomes demonstrating successful learning will include: relevant text entered from keyboard, including use of Space bar, enter key, Backspace and Shift and/or Caps Lock; headings emboldened appropriately; • Techniques: Expected outcomes demonstrating successful learning will include: images imported and positioned appropriately in the text; text entered, formatted with a range of font styles (i.e. bold, italic and indent) to make the story easy to read; • Concepts: Expected outcomes demonstrating successful learning will include: clear explanations as to how text and images can be used to communicate an episode or even in a story and reasoned justifications for the layout of their page; • Higher order skills: Expected outcome demonstrating learning will include: appropriate image(s) selected for a page; appropriate styles used to convey meaning with explanations as to why the styles have been chosen.

According to Bennett et al. (2007) the type of evidence that you will require will depend on the learning objectives of the ICT activity, task or project. He states that the content of the text will be important particularly “if a skeleton text has been improved or the text copied from a web page has been summarised to show key points” (p. 61). The appearance of the text will also provide good evidence.

In relation to planning, it is important that you consider what the students will need to produce to provide you evidence of their learning. Bennett et al. (2007) also points out that a highly structured writing frame could prevent students from developing their capabilities especially if you are trying to assess whether students can edit text independently for a given purpose. By modifying the task to a skeleton text or word bank the emphasis to using editing and formatting tools as opposed from just entering text. As a consequence the students do not spend too much time on keying in words and focus their efforts more on improving the appearance and thus the readability of their text. Such a method will provide with substantial evidence of the differences in your students’ capabilities in editing.

How do I gather the evidence?

There are many ways for you to gather the evidence you need to assess their ICT capability. The most obvious when it comes to using ICT in the classroom is through the finished product. However, as stated earlier this does not give a complete picture of what they are capable of doing. The finished product is not irrelevant but is known as tangible evidence. Ephemeral evidence is short lived and include the techniques use to complete a finished product. Observation and monitoring student’s work with ICT is the most effective way. The most feasible way to achieve in the primary classroom includes by:

• Saving or printing documents at various stages of completion; • Asking students to log their decision-making at key points in an activity; • Using photographic evidence; • Informal observation and;

Saving or printing documents

Saving your work on ICT is great practice because you never know what will go wrong with technology. If something can, it will go wrong. It means continually backing up your work to ensure that there is no loss involved and when it comes to assessment of capabilities in ICT this is quite significant. Programs such as MS Word can be set to automatically save work at various times and I usually like to first encourage my students to save their work after they have just completed something that they are happy with but also I will demonstrate to them how they can change their program to save automatically at 10 minute intervals (I find this ideal as 5 minutes is too short and it may slow down the progress and efforts of students if it is too short). So while this does not capture the techniques used, it is a fantastic routine to establish for students in your classroom. Printing work is good too, but it cost paper and time if there is a que and if something is not saved it could create problems if technical issues arise.

Logging decision-making at key points

An ICT capable student is someone who is able to decide and knows if a technique, software or hardware is the right ICT tool for the solution. Bennett et al. (2007) makes a fantastic point here as higher order skills is the main component of a student’s ICT capability that you need to assess. So having students log their decision making at key points in activities is ideal. He further explains how this is best achieved when students are working in pairs or groups and this is a common occurrence when using ICT in the classroom. In situations like this, one student could complete the activity log by explaining the reasoning behind a particular technique. Can you recall what I said about assessing reasoning? It could also just be a tick box to show which approach was selected could also be used.

Another approach that Bennett et al. (2007) discusses is the self-assessment option. While this assessment method does contain valid points, the main disadvantage here is that over-confident students may over rate their capabilities. It does provide a useful method of assessment as it helps students to understand the purpose of the activity. However, in my opinion by asking the students to actively log their decision you will be to effectively and efficiently assess their higher order skills – they ‘why’ and ‘how’ they chose a technique/ software/hardware for this particular solution. Self-assessment by-passes this process and students sometimes are not as confident and competent in techniques as they think they are.

An important factor which you also need to take into account here is the level of support provided to students. Bennett et al. (2007) raises a valid point by highlighting that a class may produce similar looking outcomes and this is accurate when using ICT. There will be students who would have received substantial guidance and help from you, or a teacher assistant, another student and even an Adult helper in the classroom. It is advised that you log this down in your notes when recording the student’s progress.

Photographic Evidence

Not all ICT activities lends itself to this method of evidence collection for assessment. The ones that do have a practical component about them like using a programmable toy or logging data. Screenshots is an alternative to this approach and it can also be used to train students in this technique. Taking screenshots is a great way of recording evidence at key points of time during an activity, task or assignment and can be used in conjunction with the students logging their decision making. Any screenshot can be pasted in their word processor like MS Word to be used as part of your assessment of their capabilities in ICT.

Informal Observation

The best way to assess a student’s ICT capability is by observation but it has to be carried out in way that you will be able to use it for future planning. To achieve this you need to monitor the way a student responds to your explanations continuously. Determine the way they:

• Answer your questions; • Apply themselves to the task you have set.

Also monitor:

• The changing moods and relationships between different students or groups of students and; • Whether you have pitched the level of demand appropriately. (Bennett, Hamill, & Pickford, 2007)

The information gathered through this type of observation will be useful to you to change your approaches to teaching in order to meet the needs of the students. It will also provide you with information as to the opportune moments to intervene with apposite instructions, explanations or simply just to increase the level of challenge provided by the activity. You will notice that some students will be very confident and can carry on the activity with little help. These are the students that you can assign additional roles to as ‘student helpers’.

It is important to find the time at the end of every ICT lesson to recall how the students responded to the ICT activity you assigned them. By making clear learning objectives you will be able to quickly decide who was able to complete the activity without support, with minimum support and those who needed considerable support. A coding system as discussed earlier would be an ideal way of doing this. As with most of your primary lessons, you may have a teaching assistant or an Adult helper with you. Ask for their help in the informal assessment especially if they had been supporting a particular groups of students. Log the progress of the students in your class as soon as possible after (or if you can, during the lesson) a lesson (Bennett, Hamill, & Pickford, 2007).

Structured Observation

The only difference between structured and informal observation is the preparation. There are three main ways this can be carried out:

1. Observe the whole class to gain an overview of its progress; 2. Observe a different group closely each week to gather more detailed information on individual attainment; 3. Target your observation on particular skills. (Bennett, Hamill, & Pickford, 2007, p. 64)

Once again, using a coding system will be the best way to monitor and observe their progress in capabilities. Take for example a class learning how to record and edit video clips. Focusing on the routines (skills) and techniques you might code in the following way:

A – Applied with confidence B – Needed some help C – Needed considerable help D – Attempted but unsuccessful E – Not attempted X – Not observed

Always ensure that you include the final category so that you will know for the next lesson who to prioritise for the next observation. It won’t be possible to complete the observation sheet during the lesson so be patient and ensure that in the end all students have been adequately assessed. In my experience, it is best to provide the students with ample time to complete the task or activity. This is where effective planning comes into its own. By taking into account the fact that during the lesson there will be students who will need considerable support and time for classroom management, you will need to allow yourself enough time to adequately assess their progress and doing this ensure that they finish what they started.

Record the Evidence

It is vital that even if you are an experienced teacher in the field of ICT that you record your structured and informal observations. It will help you track the students’ progress of a longer period of time. This will be covered more in the next module Using Assessment Effectively.

To gain access to additional resources click on the image.