SENIOR UNIVERSITY TEACHING QUALIFICATION 17 Effect the flipped classroom in learning Effect ofof the flipped classroom in learning EFFECT OF THE FLIPPED clinical research methods clinical research methods CLASSROOM IN LEARNING CLINICAL RESEARCH METHODS Dr. C.J.M. Doggen, HealthHealth Technology and Services Research, Faculty BMS, University of Twente of Twente Dr. C.J.M. Doggen, Technology and Services Research, Faculty BMS, University Rationale Rationale Effect Effect of of the the flipped flipped classroom classroom in in learning learning Examination results Examinationmethods results clinical clinical research research methods Dr. C.J.M. Doggen, Health Technology Services Research, Faculty BMS, University Twente Table 1.and Presence of all 118 students duringof all 6 tutorials (T) - Student-driven learning is encouraged and Research, of 1. Presence of allBMS, 118University students during all 6 tutorials (T) Dr. C.J.M. C.J.M. Doggen, Doggen, Health Health Technology TechnologyTable and Services Services Research, Faculty Faculty BMS, University of Twente Twente - Student-driven learning is encouragedDr. (customized, on-demand learning, self-paced) T1 T3 T4 T5 T2 T7 (customized, on-demand learning, self-paced) T2 T7 T1 T3 T4 T5 Examination - Online educational materials available Rationale Examination results Study Researchresults Frequency association experimental observational Rationale Examination results Online educational materials available Rationale Research Frequency association experimental observation Study - Flipped classroom: basic knowledge moved to design question measures measures design design Table 1. Presence of all 118 students during all 6 tutorials (T) -- Student-driven learning is encouraged - Flipped classroom: knowledge moved to Table of all during all question measures design design design Student-driven learning is encouraged RATIONALE EXAMINATION pre-class, application andbasic syntheses in-class Table 1. 1. Presence Presence ofRESULTS all 118 118 students students duringmeasures all 6 6 tutorials tutorials (T) (T) -(customized, Student-driven learning is encouraged on-demand learning, self-paced) T1 T3 T4 T5 T2 T7 (customized, on-demand learning, self-paced) pre-class, application and syntheses in-class T1 T3 T4 T5 T2 T7 (customized, on-demand learning, self-paced) - - Integrating online materials order to actively Student-driven isinencouraged (customized, T1 95 (80.5) T2 T3 T4 T5(44.9) T747 (39.8) Present, N(%) 79 (66.9) (66.1) 68Study (57.1) 53 -- learning Online educational materials available Research Frequency78association experimental observational educational available Study association experimental - on-demand Integrating online materials in materials order actively -- Online Online educational materials available association78 (66.1) experimental Research Frequency Study observational Present, N(%) Research 79 (66.9)Frequency 95 (80.5) 68design (57.1) observational 53 (44.9) 47 (39.8) engage studentslearning, and enhance learning isknowledge ato challenge classroom: basic moved to self-paced) design question measures measures design -- Flipped Flipped classroom: basic knowledge moved to design question measures measures design design Flipped classroom: basic knowledge moved to question measures measures design design design engage students and enhance learning isclinical a challenge pre-class, application and syntheses in-class - - Effects on learning effectiveness in course pre-class, application and syntheses in-class Online educational materials available application and syntheses in-class --pre-class, Integrating online materials in order to actively - Effects on learning effectiveness in course clinical N(%) 79 (66.9) 95 (80.5) 78 (57.1) 53 (44.9) 47 (39.8) research methods yet unknown Integrating online materials in order actively Present, N(%) 79 95 (66.1) 68 (57.1) 53 47 Table 2. Present, Grades by presence/absence of (66.1) the 11868 -engage Integrating online materials in order to to actively - Flipped classroom: basic knowledge moved to pre-class, Present, N(%) 79 (66.9) (66.9) 95 (80.5) (80.5) 78 78 (66.1) 68students (57.1) 53 (44.9) (44.9) 47 (39.8) (39.8) students and enhance learning is a challenge engage students and enhance learning is a challenge research methods yet unknown engage students and enhance learning is a challenge Table 2.Presence Gradesorby presence/absence of the 118 students -- Effects on learning effectiveness in course clinical Cohort <5.5 >=5.5 Totaal application and syntheses in-class Effects on learning effectiveness in course clinical -research Effectsmethods on learning in course clinical Table 1. absence Presence of all 118 students during allpassed) 6 tutorials (T) yet effectiveness unknown (% failed) (% aantal Table 2. Grades by presence/absence of the 118<5.5 students research methods unknown Totaal Cohort Presence or >=5.5 - Integrating online materials in order to actively engage Table research methods yet yet unknown Objective Table 2. 2. Grades Grades by by presence/absence presence/absence of of the the 118 118 students students students studenten Cohort Presence or <5.5 >=5.5 Totaal absence aantal (% failed) (% passed) students and enhance learning is a challenge Cohort Presence or Totaal <5.5 >=5.5 Objective or <5.5 >=5.5 Totaal (% failed) failed) (%56.8% passed) absence aantal 2015-2016 Cohort >=5Presence 44aantal 43.2% to- assess the effects of flipped classroom absence (% (% passed) students studenten Objective (% failed) (% passed) absence aantal Effects on learning effectiveness in course clinical research Objective students 74 studenten 71.6% Objective <5 students 28.4%* into teaching Research Methods 2015-2016 >=5 studenten 43.2% 56.8% students studenten 44 assessClinical the ofthe flipped classroom 2015-2016 >=5 44 43.2% 56.8% methods yeteffects unknown to assess effects of flipped classroom 2015-2016 >=5 44 43.2% 56.8% 118 61.0% 39.0% nd to assess the effects of flipped classroom 2015-2016 >=5<5 44 43.2% 71.6% 56.8% year students Technical Medicine, toin2 teaching to assess the effects of flipped classroom 74 28.4%* <5 74 71.6% 28.4%* Clinical Research Methods in teaching Clinical Research Methods <5 74 71.6% 28.4%* Being present leads to in teaching Clinical Methods <5 74 71.6% 28.4%* in teaching Clinical Research Research Methods ontolearning effectiveness (engagement, 118 11861.0% 61.0% 61.0% 39.0% nd year 39.0% nd students Technical Medicine, to 2 118 39.0% nd 2 year students Technical Medicine, year students Technical Medicine, to 2 higher % passing exam. 118 61.0% 39.0% nd Being present leads to topresent 2014-2015 102 48.0% 52.0%* year students Technical Medicine, to 2learning OBJECTIVE Being present leads Being le and grades) on effectiveness (engagement, Being present leads to on learning effectiveness (engagement, on learning effectiveness (engagement, higher % passing exam. No large differences on learning effectiveness (engagement, 2014-2015 102 48.0% 52.0%* higher % passing exam. 2014-2015 -102 48.0% 52.0%* to assess the effects of flipped classroom and grades) grades) higher % passing exam. higher % passin 2014-2015 102 48.0% 52.0%* and 2014-2015 102 48.0% 52.0%* No large large differences over the years. and grades) and grades) No differences 2013-2014 116 60.3% 39.7% Noover large differences in teaching Clinical Research Methods No large differe the years. 2013-2014 116 60.3% 39.7% over the years. * chi-square2013-2014 p-value <0.01 -116 60.3% 39.7% over the years. 2013-2014 116 60.3% 39.7% ** chi-square Methods to 2nd year students Technical Medicine, over the yea chi-square p-value p-value <0.01 <0.012013-2014 116 60.3% 39.7% * chi-square p-value <0.01 Methods Methods Methods(engagement, * chi-square p-valueby<0.01 on learning effectiveness Table 2. Grades presence/absence of the 118 students Methods Table 3. Table Grades by presence frequency ofofthe Cohort of 118 2nd Cohort yr TM of students 2015-2016 3. Grades by presence frequency the118 118 students students 118 2nd TM students 2015-2016 nd yr Table and grades) Cohort of 118 yr 2015-2016 Table 3. 3. Grades Grades by by presence presence frequency frequency of of the the 118 118 students students Cohort ofCourse 118 2 2ndEpidemiology yr TM TM students students 2015-2016 10 week Feb 2017 10 week Course Epidemiology Feb 2017 Presence N Grade Min-Max 10 week Course Epidemiology Feb 2017 Presence N Grade Min-Max Being more oftenoften Presence N Grade Min-Max 10 week Course Epidemiology Feb 2017 Being more Being more often nd Presence N Grade Min-Max Two teachers Being more often Table 3. Grades by presence frequency of the 118 students Cohort of 118 2Two yrteachers TM students 2015-2016 Mean (SD) Two teachers present leads to to Mean (SD) Mean (SD) Two teachers present leads to Mean (SD) present leads METHODS present leads to Not at all 8 5.0 (1.0) 3.5 –– 6.8 10 week Course Epidemiology Feb 2017 higher grades. grades. Not at all 8 5.0 (1.0) 3.5 6.8 Not at all 8 5.0 (1.0) 3.5 – 6.8 higher Presence N Grade Min-Max Not at all 8 5.0 (1.0) 3.5 – 6.8 Being more oft Learning effectiveness measured by: higher grades. higher grades. 1 11 4.7 (0.9) 3.7 –– 6.4 Learning effectiveness measured by: 10 week Cohort of 118 2nd yr measured TM students 2015-2016 1 11 4.7 (0.9) 3.7 6.4 Learning effectiveness measured by: Two teachers Learning effectiveness by: 1 11 4.7 (0.9) 3.7 – 6.4 1 11 16 4.7 (0.9) 6.4 Mean (SD) 3.7 -- Teachers impression 2 4.9 (1.0) 3.2 – 6.8 present leads 2 16 4.9 3.2 -- Teachers Teachers impression Epidemiology Feb impression 2017 Two teachers 2 16 4.9 (1.0) (1.0) 3.2 –– 6.8 6.8 - Course Teachers impression results comparing grades 2 Not at 3.2 6.8 3.5 – 6.8 3 all 16 22 8 4.9 (1.0) 5.0 (0.9) (0.9) 3.2 – – 7.3 7.3 5.0 (1.0) -- Examination Examination results comparing grades 3 22 5.0 3.2 – higher grade Examination results comparing grades 3 22 5.0 (0.9) 3.2 – 7.3 a) students present absent 5.3 (1.3) 3.2 - Learning Examination results comparing grades 3 22 17 5.0 (0.9) –– 7.8 7.3 3.7 – 6.4 effectiveness measured by: vs a) students present vs 17 5.3 3.2 1444 4.7 (0.9) 3.2 a) students present vs absent absent 17 11 5.3 (1.3) (1.3) 3.2 –– 7.8 7.8 5 21 5.7 (1.3) 3.5 – 7.8 b) previous cohorts 5 21 5.7 (1.3) 3.5 – 7.8 b) previous cohorts a) students present vs absent 4 17 21 16 5.3 (1.3) 7.8 3.2 – 6.8 Learning effectiveness measured by: - Teachers impression 5.7 (1.3) 3.5 – 7.8 b) previous cohorts 256 4.9 (1.0) 3.2 23 5.7 (1.1) 3.5 –– 7.8 -- Evaluation reports comparing cohort 6 5.7 3.5 Evaluation reports comparing cohort 5 * ANOVA 210.06 23 3.5 7.8 3.2 – 7.3 b) previous cohorts 6 P-value 23 22 5.7 (1.3) 5.7 (1.1) (1.1) 3.5 – – 7.8 7.8 -2015-2016 Evaluation reports comparing cohort - - Teachers impression Examination results comparing grades 3 5.0 (0.9) vs previous cohorts on engagement, * ANOVA P-value 0.06 do not not like this this kind kind of of 2015-2016 vs cohorts on engagement, ‘‘‘III do 6 * ANOVA P-value 230.06 5.7 (1.1) 3.5 – 7.8 2015-2016 vs previous previous cohorts on of engagement, - - Evaluation comparing cohort do not like like this kind of motivation and comments on type teaching teaching, II really prefer a)reports students present vs absent Examination results comparing grade Table 43. Grades by presence frequency 17 5.3 (1.3)of the 118 students 3.2 – 7.8 motivation and on teaching, motivation and comments comments on type type of of teaching teaching teaching, I really really prefer prefer * ANOVA P-value 0.06 lectures’. 2015-2016 vsb)previous cohorts on engagement, lectures’. ‘ I do not like this kind of 5 21 5.7 (1.3) 3.5 – 7.8 previous cohorts lectures’. a) students present vs absent motivation and comments on type of teaching Evaluation reports teaching, I really prefer Evaluation reports Flipped classroom: 6 23 5.7 (1.1) 3.5 – 7.8 Evaluation reports comparing cohort Evaluation reports b) previous cohorts EVALUATION REPORTS Flipped classroom: lectures’. Flipped classroom: Students: own efforts and interest * ANOVA P-value 0.06 Students: own efforts and interest 2015-2016 previous cohorts on engagement, Previous years: Students: own efforts and interest - Evaluationvsreports comparing cohort 2015-2016 vs Previous years: Previous years: Results good, did did not not like like no no lectures lectures were were‘I do not like this k Evaluation reports Previous years: Results good, Results classroom: Students: appreciated variety good, did not liketutorials, no lectures wereteaching, I really p motivation comments on type ofmotivation teaching and Students: appreciated variety previous and cohorts on engagement, Students: appreciated varietyFlipped lectures and lectures given, motivation and attendance Students: appreciated variety ‘‘Are we really not getting given, motivation and attendance we really not lectures and tutorials, lectures given, motivation and attendance Students: own efforts and interest lectures’. ‘Are Are we reallyfrom not getting getting lectures and tutorials, lectures low, tutorials (quiz) useful, book any lectures you?’ Teachers impression of lectures and tutorials, lectures low, tutorials (quiz) useful, book comments onTeachers type ofimpression teaching of were clear, tutorials were useful, took time, book not read as any Previous years: were clear, tutorials were useful, low, tutorials (quiz) useful, book any lectures lectures from from you?’ you?’ Teachers impression of were clear, tutorials were useful, Results good, did not like no lectures were not nice, teachers did not give students: clear,book tutorials wereas useful, not teachers did not give Evaluation reports students: took time, not read a not nice, nice, teachers did not used give to awere standard, teachers enthusiastic, topic uninteresting. Students: appreciated variety students: Flipped classroom: took time, book not read as a clear answers. Teachers: given, motivation and attendance took time, book not read as a clear answers. Teachers: used to ‘Are we really not getting teachers enthusiastic, clear answers. Teachers: used to lecturesstandard, and tutorials, lectures standard, teachers enthusiastic, more active students, expected RESULTS Students: efforts -- were very surprised, standard, teachers enthusiastic, more active students, expected low, tutorials (quiz) own useful, bookand interest any lectures from you?’ were very surprised, Teachers impression of topic uninteresting. more active students, expected -- did were very surprised, topic uninteresting. Previous years: more self-reflection and an were clear, tutorials were useful, not want to believe this, topic uninteresting. more self-reflection and an Results Flipped classroom: Teachers impression of students: good, did not like no lectures were did not want to believe this, not nice, teachers did not give more self-reflection and an students: did not want to believe this, academical attitude. Students: appreciated --- were somewhat confused, took time, book not read as a variety academical attitude. somewhat confused, academical attitude. given, motivation and attendance Students: own efforts and interest good, did not like no clear answers. Teachers: used to -- were were somewhat confused, ‘ Are we really not getting doing pre-class work was necessary lectures and enthusiastic, tutorials, lecturesmore active students, expected standard, teachers -- realized realized doing pre-class work was necessary realized doing pre-class work was necessary low, tutorials (quiz) useful, book any lectures from you?’ lectures were given, motivation and attendance low, tutorials were very surprised, - -were very surprised, Discussion Teachers impression of to make in-class meeting useful, Discussion to in-class meeting useful, topic uninteresting. were clear, tutorials were useful, Discussion to make make in-class meetingmeetings useful, were really more self-reflection and an who came to the in-class not nice, teachers did not give - -did notnot want to believe this,to (quiz) useful, book approach not nice,does teachers didincrease not give clear did want believe Flipping the classroom not notably learning students: --to who came the in-class meetings were Flipping the classroom approach not increase learning who came and tothis, the in-classwith meetings were really really took time, not read as does a academical Flipping thebook classroom approach does not notably notably increase learning motivated engaged the material. attitude. effectiveness of all students Technical Medicine. It is not easily accepted motivated and clear answers. Teachers: used to - -were somewhat confused, effectiveness of all students Technical Medicine. It is not easily accepted answers. Teachers: used to more active students, expected were somewhat confused, motivated and engaged engaged with with the the material. material. effectiveness of all students Technical Medicine. It is not easily accepted standard, teachers enthusiastic, and student do not like the change. For some students the traditional and student do not like the change. For some students the traditional - -realized doing pre-class work was necessary more active students, expected - were very surprised, and student do not like the change. For some students theexam traditional more self-reflection and an might academical attitude. realized doingOver pre-class work was necessary make time less students showed up (tableto1) and more approach of lectures and tutorials have led to better results. Over time less students showed up topic uninteresting. approach of lectures and tutorials might have led to better exam results. Over timeneeded less students showed up (table (table 1) 1) and and more moreDiscussion make in-class meeting useful, approach of lectures and tutorials might have led to better exam results. more self-reflection and an time was for in-class work. -todid not want to believe this, It would be a challenge to found out which students prosper by what in-class meeting time useful, was for work. It would be to out students prosper what was needed needed for in-class in-class It would be a a challenge challenge to found founddoes out which which students prosper by by what - who came to thetime in-class meetings were work. really academical attitude. type of (mixed) approach. Flipping the classroom approach not notably increase learning were somewhat confused, type of (mixed) approach. - who came to the in-class meetings were really motivated DISCUSSION type of (mixed) approach. and engaged withwork the material. effectiveness of all students Technical Medicine. It is not easily accepted -motivated realized doing pre-class was necessary and engaged with the material. Flipping the classroom approach does not notably increase and student do not like the change. For some students the traditional Carine Doggen, Associate Prof. Discussion to make in-class meeting useful, Carine Doggen, Associate Prof. learning effectiveness of all students Technical Medicine. Carine Doggen, Associate Prof. Over time less students showed upEmail: (table 1) and more c.j.m.doggen@utwente.nl approach of lectures and tutorials might have led to better exam results. Email: c.j.m.doggen@utwente.nl - whotime came tostudents the in-class meetings were Email: c.j.m.doggen@utwente.nl Health Technology and Services Research Flipping the classroom approach doesdo notnot notably increase learning Health Technology and Research Over less showed up (table 1)really and more timeIt would Itbe is not easily accepted student like the time was needed for in-class work. Health Technology and Services Services Research a challenge to found and out which students prosper bychange. what motivated and engaged with the material. effectiveness of all students Technical Medicine. It is not easily accepted was needed for in-class work. some students the traditional approach of lectures and type of For (mixed) approach. and student do not like the change. For some students the traditional tutorials might have led to better exam results. It would be Over time less students showed up (table 1) and more approach of lectures might have led tobybetter a challenge to foundand out tutorials which students prosper what exam results time was needed Carine for in-class Doggen, work. Associate Prof. Ittype would be a challenge to found out which students prosper by what of (mixed) approach. Email: c.j.m.doggen@utwente.nl Health Technology and Services Research type of (mixed) approach.