What Really Happens When Educators Make and Evaluate TEL Innovations? Claire Raistrick University of Warwick, Coventry, UK c.g.raistrick@warwick.ac.uk Abstract: This social practice research focuses on what educators in higher education (HE) actually do when they make technology enhanced learning (TEL) innovations. Using the concept of self‐evaluation presented by Saunders et al. (2011) I focus on the self‐evaluative practices educators use to make judgements about the worth or value of their TEL innovations, rather than the technology itself. One prominent view, regarding TEL in HE is “that effective change will emerge by equipping the main participants with a proper understanding of their needs, and then with the ability to use technology effectively to meet them” (Mayes, 2009:46‐47). However, how is this “effective change” to be achieved? Perhaps self‐ evaluation might contribute? The methodological approach in this qualitative study is co‐construction – the researcher and participants working together to generate data through dialogical conversation before also using RUFDATA (Saunders, 2000) (an established evaluation tool). The rich data we generate provides evidence from which I typologise features of self‐evaluative practices. Participants display recurrent behaviours as they engage iteratively with the processes of self‐ evaluation; their questioning, responsive acts involving both stakeholders and reflexivity. These efforts cohere to move their TEL projects forward, from one provisional state to the next, via a process of evaluative creep which achieves pedagogical change. The significant output of this research is the SEPT4TEL (self‐evaluative practices typologies for TEL) framework which offers a robust understanding of what is meant by self‐evaluative practices in this study’s context. This new knowledge exemplifies Reckwitz's (2002) description of people as carriers of practice within their social mêlée. The SEPT4TEL framework is a means to guide and inform educators’ judgements about their TEL innovations; thereby supporting the transformative potential of TEL. It promotes authentic, systemised use of self‐evaluation within evaluative cultures where educators’ learning about their professional practice is fundamental. Keywords: self‐evaluative practices, technology enhanced learning, higher education, innovation
1. Introduction A landmark report regarding Technology Enhanced Learning (TEL) in higher education (HE) suggests “that effective change will emerge by equipping the main participants with a proper understanding of their needs, and then with the ability to use technology effectively to meet them” (Mayes, 2009:46‐47). But, how is this “effective change” to be achieved? Perhaps self‐evaluation might contribute? Overall, four domains of evaluative practice are recognised in HE: national/systemic, programmatic, institutional and self‐evaluation (Saunders et al., 2011). It is, however, “unusual to focus on evaluation practice as an object of study” (Saunders, 2011:1). Additionally, though examples of educators’ TEL innovations are widespread, educators’ self‐evaluative practices are not the focal point. Therefore this study’s focus on self‐evaluative practices when making TEL innovations is unusual. This paper concentrates on one aspect of a social practice research study which asks: ‘How do educators in HE explain what they do, and why, when evaluating TEL innovations?’ In short the answer to this question is encapsulated in the SEPT4TEL (self‐evaluative practices typologies for TEL) framework which is the significant output of this research and is detailed presently. Accordingly, the object of study is the self‐evaluative practices used by educators (teachers, researchers who teach and technologists) when they introduce a TEL innovation to enhance the learning environments in which they work and how they are convinced that their TEL innovations are worthwhile. I use the concept of self‐evaluation presented in Saunders et al.’s (2011) fourth domain of evaluative practice: self‐evaluative practice – evaluations which take place to inform the professional practice of either solitary practitioners or groups of practitioners. The nub of self‐evaluation being: the development of new knowledge “as people engage in a process of reflection related to real problems and issues in their own context” (Saunders, 2011:14). One prominent barrier to the transformative potential of TEL in HE is that “the data needed for informed judgements about the way in which e‐learning provision is actually used in real learning and teaching at module level are very hard to pin down” (Mayes, 2009:52). Therefore, for TEL to be transformational educators need to find effective ways to “pin down” data to inform their judgements. By illuminating the fine detail of self‐evaluative practices when educators implement TEL innovations this study contributes to filling this gap. As Saunders asserts: “[i]f we are to improve the capacity of evaluations to be used (to contribute to
393