SHOWING THE PATH TO SUCCESS -ASSESSMENT FOR LEARNING IN CONTEXTUAL PEDAGOGICAL LEARNING ENVIRONMENTS Maarika Piispanen, Merja Meriläinen Kokkola University Consortium Chydenius (FINLAND) Kokkola University Consortium Chydenius (FINLAND)
Abstract Most teachers and researches agree that assessment for learning can help raise teaching quality, improve the learning experience for students and increase student`s learning outcomes. (Fjortoft, 2012; Meriläinen & Piispanen, 2013; William 2012) According to Finnish National Core Curriculum for Basic Education (2004, 260), the task of assessment during the course of studies is to guide and encourage studying and to depict how well the student has met the objectives established for growth and learning. It is the task of assessment to help the student form a realistic image of his/her learning and development, and thus to support the student´s personality growth, too. In transformational pedagogical settings, the assessment will be seen as learning itself contrary to traditional pedagogic where assessment is seen as information about learning. According to Piispanen & Meriläinen (2013) in the model of Contextual-pedagogical approach to learning the assessing criteria will be visible and well known already at the beginning of the learning process. It is important to begin the planning process by identifying the learning goals and criteria for successful learning. After that the task and the activity designing process is easy to accomplish so that the tasks and activities will help students to achieve these goals. Assessing will act as a tool for guiding students through the learning path –the learning aims will come true through the learning tasks based on assessing criteria. This is, as Meriläinen & Piispanen (2012) states, particularly important in order to make students understand and recognize what the learning expectations are and how will the assessment come true. In essence, assessment occurs at all stages in the learning process and is beneficial for both the students and the teacher. This article examines pedagogical changes in different learning contexts as well the questions of assessment in these contexts with the help of the model of Contextual pedagogical approach to learning (Meriläinen & Piispanen 2012; Meriläinen, Piispanen & Valli 2013). When working in learning environments based on Contextual pedagogical approach to learning, the teacher will emphasize transdisciplinary approach to curriculum, enhance student’s individuality and creativity in different learning situations and support both content knowledge as well as 21st Century civil skills knowledge to develop hand in hand. The assessment will occur at all stages in the learning process and guide students to assess themselves to improve learning throughout the learning process. st
Keywords: assessment, contextual pedagogical learning, 21 century civil skills pedagogical content knowledge
TOWARDS WELL KNOWN DESTINATIONS The teacher has a great responsibility as a guard of that treasure chest that is called evaluation. According to some opinions; the teacher stands with the handle of the power and decides what the grades (numbers) are that students will get. The other could say that it is just to give each one what one earns. In either case it is all about the discussions, considerations and questions of power, responsibility and grounds and - hopefully - the pupil's rights. The Finnish National Curriculum (2004) namely requires educational institutions to define the fundamentals of evaluation in their curriculum. The pupil has a right to know to what kind of criteria his evaluation is directed and what kind of knowhow the different grades require. If the subject of the evaluation were ready operating functions of some kind of some kind of machine according to the original settings, the criteria for evaluation would be a relatively simple matter. When the evaluation target is aimed at learning, that will happen accord-
ing to individual abilities within people, who grow individually and learn individually in various growth environments, the programmed functions' cannot be expected to be similar to everyone. Can we expect that in the same way taught matters would be manifested similar to everybody and would be outlined into the memory alike? Every human being constructs information through his own worldview which also affects how new information combines to the data structure in the background. The central actors, who are usually a pupil and a teacher and to some extent also parents, will bring an additional challenge to learning and assessing. The fact, that the teacher would be able to teach and to understand learning, growth and development of each pupil, requires deep understanding about students' abilities of learning, planning focused on that learning as well as ability to create enthusiastic learning projects, lots of supporting targeting individual learners and finally ability to evaluate everything. The fact that in turn the pupil, for example a primary school pupil, would reach that information and skills, which he is expected according to the descriptions of the good know-how of grounds for the curriculum, during a learning project, requires of course good teaching and support, but also the clear visibility of the assessment criteria so that the pupil will be aware of what he is expected to understand, do and be after the project as well as how he is able to achieve these expectations. This realizing is a precondition for the fact that the pupil can set for himself targets towards which to attempt. It is not enough that the pupil is requested to learn "well" or to get good grades from the test or for example to certificate - "nobody will find out from the unknown big city without a map, navigator or the directing traffic signs" unless he happens to have good luck with him. The success of the learning must not depend on good luck. The child must be taught according to his age level to navigate towards the set learning targets and consider carefully with which methods he is able to reach them. This is one central stage of the evaluation. The student is actively participating in his own learning process by setting himself individual learning aims. Noticing that he himself can influence his knowhow and its evaluation during the whole learning process, one will become motivated and engaged towards learning.
Expanding the perspectives The teacher's challenge in today’s education is to strengthen learner´s natural ways to learn and produce information in different learning environments. Learning is thus seen as something happening in connection with an individual and his environment. Norrena, Kankaanranta and Nieminen (2011) argue that there has to be a significant pedagogical change in teaching routines and pedagogical operations to move from teaching to learning and towards 21st century requirements. How will this change become true in primary school learning contexts – what are those pedagogical changes in the field of curriculum, planning and implementing as well as in the roles of teachers and students? The Figure of 21st Century Civil Skills Pedagogical Content Knowledge (21st Century CSPCK) (Fig. 1) attempts to identify the nature of vast pedagogical knowledge required when turning learning from traditional to transformational i.e. blending the 21st century civil skills in to the authentic learning contexts and the curriculum.
Figure1. The 21st Century Civil Skills Pedagogical Content Knowledge (Meriläinen & Piispanen, 2013, follows Mishra & Koehler 2006, 2010)
The basis of the framework is the understanding that teaching itself is a highly complex activity that draws on many kinds of knowledge. This knowledge, as Ashe and Bibi (2011) highlights, is diverse and includes both content and pedagogical knowledge. In recent years the new type of knowledge has been raised to attention that of 21st century skills or 21st century civil skills as Finnish National Board of Education has named that knowledge in curriculum renewing process. The 21st Century CSPCK – figure articulates the role of 21st century civil skills in the process of teaching and learning in a really blended manner. In 21st Century CSPCK –model the emphasis is put on competency, performance and capabilities and the key question in learning situations is rather how the information will be used than what the information is. At the heart of the 21st Century Civil Skills Pedagogical Content Knowledge framework, is the complex interplay of three primary forms of knowledge: 21st Century Civil Skills Knowledge (21st Century CSK), Pedagogical Knowledge (PK), and Curriculum Content Knowledge (CCK). It is essential as a teacher to find the 21st Century Civil Skills Pedagogical Content Knowledge point of intersect, where the three primary forms of knowledge meets each other and use that essence as a starting point when creating innovative and enthusiastic learning as well as authentic learning situations. (cf. Mishra & Koehler 2006, 2010) As Meriläinen & Piispanen (2012) highlights, the planning process is to be viewed from at least three different angles as pictured in Fig.1. What do we mean by that is that the emphasis of learning should not lie on curriculum contents (subject contents) themselves, but these contents should act as tools for accomplishing 21st century civil skills by arranging learning situations and environments as authentic as possible to support vast and deep understanding of every day phenomena. The 21st century civil skills should also not be seen as isolated skills or learning targets, but they should be examined as visible parts of a learning context. Together all the three knowledge areas will create a successful and pedagogically meaningful learning process produced by students and supported by teachers. (Meriläinen & Piispanen 2012, 2013) In a Contextual-pedagogical approach towards learning, a special attention is paid to the growht of 21st Century CSPCK knowledge. The skills, context and pedagogic will have a crucial significance in all learning situations. Where traditional pedagogic and multidisciplinary approach to integration emphasizes pedagogy and curriculum as tools for creating learning situations, the transformational pedagogic connects the three knowledge areas together. The learning situations will be discovered in the heart of the expanded knowledge acquisition as you can see in Fig. 2.
Figure 2. Dimension of pedagogical approach to learning (Meriläinen & Piispanen, 2012.)
The student, the pedagogical expert (the teacher), the content expertise from real life contexts, the society and the curriculum will settle in the heart of the Contextual – pedagogical approach towards learning –model. The planning begins from the premise of individual student and his/ her skills, knowledge, interests and enthusiasm (comparison to traditional planning where the planning is made to fit to the school constructions; timing, text books, classrooms, etc.). In this model, the teacher observes surrounding world to the understanding and skills those primary school students will need in authentic operation cultures. Afterwards she/he mirrors and connects the curriculum contents with these real life phenomena. The real life phenomena will help learners to understand and link the curriculum contents with the life outside and develop 21st century civil skills blended to authentic learning situations. The teacher's role, as Meriläinen and Piispanen (2012) states, is to be a pedagogical expert, who creates learning situations based on the 21st Century CSPCK framework i.e. identifies the individual needs of each student, designs creative, authentic learning tasks and supports the growth of multiple civil skills needed in real life. (Meriläinen, Piispanen & Valli 2013.) According to Finnish National Core Curriculum for Basic Education (2004, 260): The task of assessment during the course of studies are to guide and encourage studying and to depict how well the student has met the objectives established for growth and learning. It is the task of assessment to help the student form a realistic image of his/her learning and development, and thus to support the student´s personality growth, too. As Bookhart (2004 5) states, the evaluation means, widely defined, the collecting of the information for benefit. It is more than a mere measurement even though the evaluation can also contain the measurement. The fact, that the “collecting of the information” for the benefit, does not define who the collector is - the pupil as well as a teacher or some other person can act as the information collector makes the evaluation interesting. The collecting of the information for evaluation use is not defined to take place at the end of the learning process, but the collecting is rather seen as active operation which takes place during the whole learning process. The pupil can be seen interpreted this way as the actor who reflects information and estimates himself and directs his learning actively during the whole learning process. At the same time, he is able to get supportive feedback from the teacher, from other pupils and experts along the whole learning process. Thus the assessment will be directed not only to the final result of the process but to the whole learning process. Once we have defined what we want our students to learn, we need to determine how we will evaluate their learning at the end as well as assess their progress to find their way through the content. To be able to that, you, as a teacher, need to know what assessment options are available and suitable to accomplish that. You also need to consider, how to construct or select an appropriate assessment from these options, how to get these assessments to yield good-quality information, how to interpret the information and help students to interpret it, and how to use the information yourself and help students (and sometimes others) to use it (Bookhart 2004,7). You also need to follow this cycle through the learning process to get the collected information used. Otherwise, as Bookhart (2004) states, the students’ time and yours are wasted. Isn´t it time to rethink the role of assessment in the successful learning process? What uses of assessment are most likely to maximize the students` learning and wellbeing? How best can we use assessment in the service of the student learning and wellbeing? If you look at the planning process from traditional points of view, where assessment is seen rather assessment as learning than assessment for learning, the whole planning process from planning to implement can be created without focusing on individual learners. In Fig. 3, you can see the difference between the traditional and transformational planning processes in the perspective of a child and assessment.
Fig. 3. Individual learners and assessment in traditional and transformational planning processes. The planning process is build up from several variables which join together more or less tight and with a little or lots of interaction between them. When you think of teaching and learning in primary school, the variables in the planning processes consists at least of six different variables as follow: a teacher, a child, a curriculum, learning tasks, learning environments and evaluation/ assessment. Traditionally, the teacher begins the learning process by seeking information from books and the curriculum. The learning contents and theme areas will rise up from the curriculum or from the books and text books. At its worst, when talking of assessment, the ready-made summative evaluation tests can be found behind the book and used as evaluation criteria of the learning process. The interaction between the process and child is slight if any. The child can be seen as stable variable –the one who accomplish the given tasks without knowing where are these tasks guiding me, where do they come and how to accomplish them at my best? In the end of the learning unit, as Beyer (1987) represents, the child will accomplish an exam, the test, which has been kept secret, is administered, and students fill in the answer in absolute quiet. The teacher watches carefully to make sure that no students refer to their notes or ask classmates for help. This common method of assessment is familiar to most students, teachers, parents, and administrators, but it fails to provide teachers or students with the information they need to promote deep understanding of the subject. Assessment can be seen as information of learning where the student lacks possibility to show his best. In transformational pedagogical settings, the child and his learning is the core activator in a planning process, where the assessment will be seen as learning itself. In the model of Contextual-pedagogical approach to learning the assessing criteria will be visible and well known already at the beginning of the learning process. Assessing will act as a tool for guiding students through the learning path –the learning aims will come true through the learning tasks based on assessing criteria. This is, as Meriläinen & Piispanen (2012) states, particularly important in order that students will understand and recognize what are the learning expectations and how will the assessment come true. Bayer (1987) describes this supportive model of assessing in the following way: As students progress through the unit, the teacher continually provides opportunities for them to think about their learning and to ask questions. She designs a performance task which requires students to show that they understand the concepts associated with the unit. In a Contextual pedagogical approach to learning, pedagogical planning begins by paying attention to the child's individuality, which directs choices which are related to a learning context as well as pedagogic. In turn, in the traditional pedagogical settings, in which the structures, schedules, class premises and books often create the framework for the learning, where the student and decisions which apply him must settle. Phenomenon that will relate to child’s everyday life will rice to the central position in planning and learning processes. This phenomenon will be mirrored through the curriculum and created to an enthusiastic learning process directed to child’s interests. The child's individuality and the interdisciplinary examining of the phenomenon require thinking and the explosion of the evaluation already at the beginning of a learning project. The fact, that each student will look through the phenomenon with
individual “glasses” does not make it possible to give a similar exam to everyone, at least it is not ideal.
Comprehensively towards contextual and memorable learning Let´s look at the contextual pedagogical approach in opening of the curriculum closer. How does it influence your planning process? Where to begin, how to put emphasis on needed skills, what is the connection between subject contents and real life in practice, what means authentic learning environment? Above all, how will the assessing process support and guide the students to achieve their best of learning? These are some of the questions that you as a teacher will have to pay attention to when moving from traditional pedagogic towards transformational pedagogical settings. Flipping the sight from society to curriculum puts emphasis of teaching curriculum contents to emphasis of learning real life phenomena and skills that we need in authentic learning environments and learning situations. When using assessment as a tool for guiding students to master the learning process at its best, you will have to emphasize at least three different kinds of learning as follows: content knowledge and st understanding, mastery of 21 century civil skills and the contextual pedagogical learning process. (Fig. 1) In order to succeed with assessment for learning, the learning aims have to be presented clear enough at the beginning of the learning process, not only for the teacher but also for the students. (Tynjälä, 1999, 174; Virtanen 2007; 16) That will give the assessment a clear objective during the learning process.
1.2.1 Learning in contextual pedagogical learning environments The Contextual pedagogical approach to learning is based on comprehensive, overall pedagogical knowledge as presented in Fig 1. The pupil's role as an information handler and information construction worker will rise to a central position. The pupils are given concretely the “actor's role” with the help of which he dives among the information on the subject matter. The subject matter, phenomenon, is raised from the child's everyday life. In a Contextual pedagogical approach to learning the pupils usually act in the small groups in which case situations and problems of the real life phenomena are solved in collaboration with others (peer students, teacher, experts etc.). The focus will be in the learning process and process based learning where the 21st century civil skills will develop hand in hand with content knowledge. (Meriläinen & Piispanen 2012.) The following example of a typical Contextual pedagogical learning process will combine the transdisciplinary planning process, the 21st century civil skills pedagogical content knowledge and the child and his connections to life outside the school. In the example, assessment for learning is a valuable tool for a student to learn what he has done well, what to do next and how to improve to achieve even better grades from the process. Table 1. Contextual-pedagogical study plan in a nutshell (5th grade). Phenomenon (authentic/ outside the curriculum/ familiar to children (connections to real life)
Students role (authentic – rises from the phenomenon)
Junior Master Chef competitions
Competitor, editor
designer
and
Task (authentic –supports 21 tury civil Skills to develop)
st
Cen-
Introduction: “Congratulations! Your group has been chosen for the Junior Master chef competition among many participants! It is your task to draw up the three-course menu according to the instructions below, to make a sample portion and demonstrate it to the honored council which will choose the winner on the basis of the evaluation criteria. I wish you all
good luck for the competition! You will, as a small group, take a part to Junior Master Chef – competition. The task is to plan a three course menu for 20 persons. Be sure that the menu is based on the food circle and the menu consists of all primary and space colors. You will cook the sample portion for one person. Your second task is to create a calling leaflet and webpage to Gourmet Festival –blog. Your third task is to create an enthusiastic commercial video and add it to your blog. This video will be sent to competition judges. Competition council: School cooks, two peer class students, school principal and head editor from the local press. This Contextual pedagogical example has connections to the curriculum as follows: MATHEMATICS multiplication measures/conversions ten and hundred crossings ART
the primary colors and space colors mixture of colors modeling
MOTHER TONGUE AND LITTERACY skills in producing text interaction skills CROSS CURRICULAR THEMES Media skills and communication Participatory Citizenship and Entrepreneurship Technology and the individual The Contextual pedagogical approach to learning needs versatile means of assessment to give students the possibility to see the learning as a process and look for versatile information and support to meet the aims set for the learning. Rubrics and scoring have become useful tools for giving students understanding of what has been assessed. The focus of using this kind of evaluation method is to monitor and support the progress rather than assess the final end of the project. In the Junior master Chef-learning project, the evaluation criteria have been divided into six different aspects of assessment and scored to four different levels of learning as you can see in Table 2. The assessment is targeted to the planning process, to collaboration, to shared responsibility, to products, to appearance and finally to the presentation. The scoring is set from excellent to poor and the criteria for each scoring are well known to everyone already from the beginning of the course.
Table 2. Junior master Chef Assessment rubric Assessment targets Planning
Excellent
Good
Average
Poor
Learning process shows good planning. The group uses creativity in the planning process The responsibility is shared equally. Everybody knows individual responsibility areas.
The planning process needs some focusing to meet the aims. The ideas are lent from others. The responsibility is light shared but the work still goes further smoothly. Some work might be done overlapping some might be missing. The responsibility is shared, but not equally.
The group needs a lot of guidance –the ideas comes mostly from the teacher
Collaboration
The group work is based on contribution, helping, listening and cooperation. There is a harmony of the work.
The group works towards set aims, but there is a lack of discussion. There is still a harmony of the work.
The group forgets the target often. There is a lack of harmony and the teacher has to support the harmony. There is a lack of discussion targeted to the actual work.
Product
The product shows creativity and is done according to the guidelines. All the discipline contents are seen in the product and carried out correctly. The final production represents students´ excellent knowledge. The appearance is calling, neat and lack writing mistakes
The product is well done according to the guidelines. Discipline contents are seen in the product with some lack of information. You can see development in students´ knowledge.
The product is done careless. There are some missing information and some mistakes in the final product. The product doesn´t show much improvement in students´ knowledge.
The planning process is lost to other directions. The ideas are not connected to actual work. The responsibility is not shared at all. The teacher will point the responsibility areas and take care of the work all the time. Some group members are not involved of the work at all. Tasks will not be finished. The group forgets the target more than often. There is a lack of harmony and the group disturbs the others. There is a lack of discussion targeted to the actual work and some bad words are used during working. The final product does not involve all the tasks. A lot of information is missing and might be wrong. The final product doesn´t show progress in earlier knowledge.
The appearance is calling and neat. There are some mistakes in writing. The presentation is good and has been well practiced before. The presentation gives
Part of the appearance is messy and there are some mistakes in writing. The presentation has been done without practicing beforehand. The presentation nei-
Shared responsibility
Appearance
Presentation
The presentation is enthusiastic and has been excellent practiced before. The
The responsibility is partly not shared. The teacher is often needed to share the responsibility. Many tasks might be missing.
The appearance is messy and there are lots mistakes in writing. The presentation has not been done.
presentation gives lots of information a is excellent argued.
information and is argued a bit.
ther gives some information nor is argued at all.
In the contextual pedagogical learning process, the evaluation criteria will be visible already at the beginning of the learning process together with the task that is based on the assessment criteria. The given task will tell the students what they are expected of the learning process and the task is divided into several smaller tasks that will together gather the whole production needed to pass the course. The evaluation criteria give the students the path through the learning process and the aims for learning will come true in the learning tasks which are based on the evaluation criteria. In this way, the pupils understand what they are expected to learn and how the evaluation in practice comes true. (Meriläinen & Piispanen, 2012.) By constantly evaluating students work throughout the course, the teacher will identify the aspects of his teaching that students have not understood. It is then possible to repeat certain aspects again, until students have understood them. Fjortoft (2012, 6) emphasizes the timing as a key for successful learning. He states (2012, 6) that the feedback should be delivered in in connection with the actual work, before the final grade has been given, so that each student has an opportunity to use the given feedback to improve his work. Without visible assessment criteria students will be left out of the evaluation process and individual right to influence to one’s evaluation. By this kind of reflecting extract to the own work teacher also assesses and improves one´s own teaching all the time.
SYNTHESIS Chaltain (2011) represents a few examples of the change of thinking: according to traditional opinion the pupil is responsible for his learning primarily whereas the responsibility for learning in the transformational pedagogic is seen as shared for everyone involved in the learning process. The traditional idea emphasizes according to Chaltain (2011) good marks, as pupil's achievements and the transformational pedagogic focuses on the learning process as the target and tool for evaluation. The aim of the evaluation system in transformational pedagogic is to make every pupil get excited by the learning so that it can become even the passion. (Meriläinen & Piispanen 2012) According to Finnish National Core Curriculum for Basic Education (2004, 260): The task of assessment during the course of studies are to guide and encourage studying and to depict how well the student has met the objectives established for growth and learning. It is the task of assessment to help the student form a realistic image of his/her learning and development, and thus to support the student´s personality growth, too. In transformational pedagogical settings, the assessment will be seen as learning itself contrary to traditional pedagogic where assessment is seen as the information of learning. In the model of Contextual-pedagogical approach to learning the assessing criteria will be visible and well known already at the beginning of the learning process. Assessing will act as a tool for guiding students through the learning path –the learning aims will come true through the learning tasks based on assessing criteria. This is, as Meriläinen & Piispanen (2012) states, particularly important in order that students will understand and recognize what are the learning expectations and how will the assessment come true. Because the learning concentrates on above all the learning processes, problem solving and the preparing of challenges, the whole learning process is evaluated versatile. The focus from the evaluation of the final result switches to the evaluation of the whole learning process. Before the beginning of the learning project a teacher must think what will be assessed and how to define the evaluation criteria. This process has to be visible also to students. This also is stated in the Finnish National Curriculum for Basic Education (2004 162): The pupil and his parents have to be given information about the grounds of the evaluation beforehand.
It is important that the pupil will realize and experience that he is in interaction with his own evaluation st and he is also able to influence on that. This will develop the planning and thinking skills, which in 21 century pedagogy is seen as central areas of learning.
.REFERENCES [1]
Ashe, D. & Bibi, S. (2011). Unpacking TPACK and students' approaches to learning: Applying knowledge in pieces to Higher Education teaching and learning. In G.Williams, P. Statham, N. Brown, B. Cleland (Eds.) Changing Demands, Changing Directions. Proceedings ascilite Hobart 2011. (Pp.128–132).
[2]
Beyer, B. K. (1987). Practical strategies for the teaching of thinking. Newton, MA: Allyn and Bacon.
[3]
Bookhart, S. 2004. Assessment theory for college classrooms. http://www.gwu.edu/~fellows/GTAP/Online%20Makeup/TL%20Presentation%20Readings/Assessment%20Theory_2004-Winter_p5.pdf (Read: 6.6.2013)
[4]
Chaltain, S. 2011. http://www.samchaltain.com/blog (Read: 6.6.2013)
[5]
Finnish National Core Curriculum. (2004). Opetushallitus. Vammala: Vammalan kirjapaino Oy.
[6]
Fjortoft, H. 2012. Managing assessment for learning. How learning platforms help busy teachers manage assessment for learning. Itslearning AS. Bergen. Norway.
[7]
Koehler, M. J., & Mishra, P. (2010). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education (CITE), 9(1), 60-70.
[8]
Meriläinen, M. & Piispanen, M. 2012. Learning as a Phenomenon - Manuscript of Phenomenon Based Learning. Toim: L. Gómez Chova, A. López Martínez, I. Candel Torres. Edulearn12 4th International Conference on Education and New Learning Technologies 5447–5454. IATED A4.
[9]
Meriläinen, M., Piispanen, M. & Valli, P. 2013. The voyage of the dawn treader – turning learning to m – era. Toim: P. Kommers & P. Isasías. E-society. International association for development of the information society.437-442. IADIS A4.
[10]
Mishra, P. & Koehler, M,J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108 (6), 1017–1054.
[11]
Norrena, J. Kankaanranta, M. Nieminen, M. 2011. Kohti innovatiivisia opetuskäytänteitä. Teoksessa Kankaanranta, M. (toim.) Opetusteknologia koulun arjessa. Jyväskylän yliopisto: Koulutuksen tutkimuslaitos, 77–100.
[12]
Piispanen, M. & Meriläinen, M. 2013. Phenomenon Called Learning! -Turning Learning from Traditional to Transformational. 15th Annual International Conference on Education. The Education Research Unit of ATINER.
[13]
Tynjälä, P. 1999. Oppiminen tiedon rakentamisena. Konstruktivistisen oppimiskäsityksen perusteita. Helsinki:Tammi.
[14]
Virtanen, P. 2007. Arviointi, arviointitiedon luonne, tuottaminen ja hyödyntäminen. Porvoo: Edita.