ICSEI 2017 Assessment as integral understanding

Collaborative Partnerships for System-Wide Educational Improvement

Closing the gap from within – reframing assessment through integral understanding Niall MacKinnon Avernish Perspective, Highland, Scotland, UK. International Conference for School Effectiveness and Improvement Ottawa, Canada January 2017

Abstract Through a major curriculum review commencing in 2004 Scotland replaced a matrix of elaborated attainment targets with experiences and outcomes to foster 'Four Capacities' of children's potential. These are: confidence, responsibility, contribution and learning.

The focus of this paper is a single piece of imaginative writing. It was written by a higher attaining pupil in upper stages in a small rural primary school in Scotland. It was apparent that the story was markedly inferior to this child's norm. It could be put down to a one-off lapse and be discarded. The new curriculum focus on personalisation and capacities of children's potential caused the teacher to 'see' this story differently. The teacher came to realise that through the literary device of the story the pupil was telling the reader something of their own personal development and of their own needs. There was nothing sinister, implying anything untoward, but it became apparent that the simplistic language and plot were part of what that child was conveying – the meta-story. The teacher came to see the piece as sophisticated, despite the simplicity of surface structure. The shift of meaning came from an understanding of assessment which broke through the narrower confines of standardised approaches. This child was 'closing the gap', and so was the school, through its changing approach to pedagogy, but in an entirely different conceptual arena to the conventional performative implications of that term.

Introduction Through a major curriculum review commencing in 2004 Scotland replaced a matrix of elaborated attainment targets with experiences and outcomes to foster 'Four Capacities' of children's potential. These are: confidence, responsibility, contribution and learning. The focus of this paper is a single piece of imaginative writing1. It was written by Mark2, a higher attaining pupil in upper stages in a small rural primary school in Scotland. How might one assess this piece of writing?

‘Conventional’ assessment It was readily apparent to the teacher that the story was markedly inferior to this child's norm. It could be put down to a one-off lapse and be discarded. But Mark is a proficient writer as his 1

The piece of writing is not included for reasons of data protection, but this paper highlights the key features of the story, from which the reflection arose.

2 Not his real name.

1

portfolio evidences and his teacher well knows. The story is not all that long yet he has produced many pieces of exceptional length and quality for a primary school age child. This was a fairly short writing task. So that is not really a relevant factor here. At the time he wrote the story Scottish primary school education was in the midst of the 5-14 / Curriculum for Excellence switchover. So one may have assessed this piece in terms of the former 5-14 curriculum criteria and started to look at the attainment targets for, say, levels C (age 9) to D (age 11) to identify matching characteristics. After the technical aspects one might then look at the attainment targets for content and style. Or one might examine the piece of writing in terms of the new curriculum framework, which is more broad in terms of levels but more convoluted in terms of ‘outcome’ descriptors.

Was this lesson set in an ‘attainment target’ conceptual framework (of the teacher) or an ‘Es and Os3’ / ‘Four capacities’ framework (again, of the teacher)? Either way, on technical criteria Mark is clearly an accomplished writer, with pointers to improvement – such as “would of” instead of “would have”, but clearly a writer who has gone beyond technical barriers to be able to express himself fluidly in written form.

So let’s move to style and content. Here the story, in terms of narrow assessment criteria by either curriculum framework, would seem to be weak. The story is set as a football match with a list of players and before anything has really happened in the story a zombie has appeared. The zombie then acts in an extreme, violent way, and the rest of the plot is no better. Children can often introduce a disruptive event into a story as if this is exciting when in fact it is disrupting and the lack of plausibility kills the story. Other children in the class do this but actually Mark is a very accomplished writer and not prone to this tendency. Many of his pieces display a considerable depth of emotional insight and use literary devices, content and plot in a skilful way to link style, content and emotion. That does not seem to be happening here.

Or does it?

The context of the child It is essential to place this piece of writing in the context of the child, by which I mean understanding who this child is, where he is coming from and what his world ‘is’ – cognition in other words. Mark displayed particularly disturbed and disruptive behaviour in the school the subject of this study. This necessitated the involvement of psychological and behaviour support services, including linkage with other agencies in terms of home support. Mark came to the school midway through the primary years. He settled in well but then a series of tantrums began and these became progressively more severe. Mark was intellectually and emotionally sophisticated, but also fragile, vulnerable and insecure. This is extensively written up elsewhere.

My point here is to seek to ‘understand’ this piece of writing in the context of the child, and to use it as part of the assemblage of assessment evidence to inform our involvement in his developmental progression, in all respects, from, say, literacy to health and well-being.

3

Experiences and outcomes, being the terms which have replaced attainment targets, reflecting the altered conceptualisation of the incoming (now outgoing) curriculum approach.

2

Mark is separated from his father who does not live nearby, yet is emotionally close to him, though seeing him rarely. He is also very close to his brother. This is significant in terms of the separation with his father. Shortly after enrolling at the school Mark met his father after a long gap and his mother warned the teacher that this might trigger an emotional reaction. It did.

Similarly some time later his mother warned the teacher that his brother, who had joined the army, was to do a tour of duty in a combat zone, and that this might have a major reaction. It did.

Mark is very socially well aware, despite very disruptive behaviour on occasion. He is highly literate and adept in the use of new media for his age. Use of new media integrated into pedagogic contexts is a major characteristic of the teaching style in the class, particularly using new media through information communications technology for real purposes, not just for ‘teaching and learning’. The school’s and the teacher’s own practice has been innovative, for instance collaborative virtual learning environment working between schools, programming in various forms and animated stories. The explicit pedagogic approach of the class was to seek to build in progression of skills into real world tasks. The teacher uses ICT 4 in conjunction with other curricular areas, such that ICT comes to be considered and utilised not so much as another curriculum area, but, as the world is developing, as a form of media in its own right, and a form of literacy in its own right for expressing oneself. Despite the need for literacy and numeracy as more conventionally understood in school education, literacy is itself developing apace in new forms. Digital literacy is a form of literacy in its own right, one of central relevance to all our lives. Mark took a major part in these projects and initiatives. Mark was digitally literate, and in a world, which in this context is transforming before our eyes, Mark took part in and responded well to what were novel forms of pedagogical-ICT utilisation in the class and in the school. This was particularly so in Mark’s years in the school, which were for two and a half years in upper stages. Mark works very well independently, once motivated, which he is, by and large. But when he is disturbed his social world caves in, and so does that of everyone around him in consequence.

The wider context of the story It became apparent that his brother serving in the combat zone was a major worry for him. His brother also came home on occasion. The point which the teacher came to grasp is – this story is not fantastical at all. It is a description of reality, as groups of comrades go out, and then, out of nowhere, have their head blown off. It is significant that Mark has listed in detail the names of his classmates using their real names. At this time Mark wrote a very sophisticated piece on the online forum of Oracle Thinkquest5 concerning the war where the British Army were operating.

He put this together with a piece about his school and its village and the contrast to the two locales. It was multimedia and linked personal information and images with that sourced online relevant to the topic, and with his own narrative and observations, but particularly personal and reflective. In terms of Mark, in considering this story, this was his world, a profound reality, one happening to the workmates of his brother, and which was a major risk to the life of his brother – to be ‘taken out’ in just such a way as for the players in this football match, who were Mark’s ‘workmates’. The army features in the story here, and despite horror the story has a happy ending. The army deals with the ‘zombies’.

4 5

Information Communications Technology Oracle Education Foundation (2009)

3

Beyond school ‘work’ This story is simplistic, and perhaps not intended to be taken too seriously, either by him, or as intended by him for the reader, but it points to another reality. Mark’s Thinkquest online publication on war in the combat zone and relating it to his own family was very sophisticated indeed, both in terms of content and presentation – digital, multimedia and online (behind the Oracle Thinkquest firewall). But it is very unlike traditional school ‘work’. Similarly the context, process and function through which this activity occurred was nothing like a ‘lesson’ and most certainly bore not even passing resemblance to the three-part lesson, of ‘learning intentions’ ‘enaction-teaching’ ‘assessment-next steps’, as have so dominated the ‘quality assurance’ procedures by which Scottish schools are evaluated. But the story was imbued with purpose and high order skills in so many forms. It went through a truly formative process of engagement of task, with purpose and iterative exchange-reflection with other people including his teacher. The wider curriculum change significance is that this is where a huge pedagogic conflict is being worked out, as one mindset, its methods and its untrammelled power extinguishes other pedagogic visions and their means of enablement. In this case a very different integral understanding was necessary, for Mark, and his teacher. He benefited in that from his teacher, but which was scarcely considered by the audit management system, as the teacher reported. But it was though under intense therapeutic attention by the psychological, behavioural support and family liaison services. However the headteacher reported that these have scant connection with the ‘quality assurance’ audit management system applying to the school of this case study.

The pedagogic context The school’s peaceful, attractive, rural locale may seem the complete contrast to a military combat zone. Certainly in terms of overt violence and civil insurrection that may be so. But events in the school and deriving from and linking to the community during the time this pupil was in the school, and matters pertaining to ‘support’ and ‘improvement’, have shown a severe side, not recognised, and certainly not by wider society. I contend that our job as educators is to enable the development of the whole child, not fill them full of attainment targets, or more latterly ‘Es & Os’6. We want them to develop in terms of their capacities as whole people, yes including literacy and numeracy, but in an integral way as these are embedded in contexts, purposes and the new literacies of the twenty-first century. As I am indicating also here, the notion of ‘literacy’ is itself being transformed. Thus I contend that the notion of assessment as integral understanding that I am urging here, exemplified by this piece of writing and the context of the pupil, relates to different pedagogic models by which we may understand schools, and thereby by which we ‘do’ things to them, particularly in terms of forms of support, or judgement.

This piece of writing has to be seen as part of who Mark is, what he was trying to express and why. This was not a ‘serious’ piece, but underlying it was something very serious indeed. The meta-story is that Mark was worried that his brother would go out with his workmates one day in the course of their work, in normal fashion, and then, out of the blue have his head blown off. He would not say that directly. Nor would he or did he write this directly. But it became very apparent that it was the real story he was telling here. Actually a clue to the deeper meaning and 6

Experiences and outcomes.

4

use of language deployed here was given by the use of the expletive, “F*@*” in dialogue: “What the F*@* was that”.

Mark knew that that would shock, but as a device he is using it exactly right. To have then criticised him for the lack of a question mark would have been profoundly wrong – an irrelevance. And if he had sought to shock the teacher and cause a reaction by the use of an expletive that was not going to happen either. This use of language, here, in this context, and utilised by a standard literary device, indicating Mark’s social literacy, was very appropriate.

The whole child and a system thought The conventional targets, pace, impact, outcomes, tracking, challenge, deliverology, ‘increased expectations’, ‘raising the bar’ mindset of Scottish education is so deeply rooted that it has no means to understand integral learning based on experiential and contextual relevance, capacities, potentials and capabilities, or constructing meaning at the interface between pupil and educator at the individual level. Thereby it has narrow, attenuated means to understand children or their teachers. For the assessment conclusion that needed to be drawn from this piece of writing was not that it was structurally simple. On standard literacy assessment it would be deemed lower than this child’s norm. But that is not what this teacher applied. The assessment had to be structured and flexible enough to serve to inform the teacher what the pupil was saying, and what he was saying obliquely was the central message. Thereby it was also too the central assessment criterion.

Where does this aspect fit with objectives to ‘raise attainment’ now a central priority of Scottish education. It doesn’t. This assessment could only make sense to a teacher who knew the pupil well, who could then use it, with Mark, and others, as the basis to better meet his needs, to assist him in developing the awareness of his own development, and participate actively in the processes which meet his needs. I would call this deep formative assessment. It is about revealing learning, not concealing it.7 Such deep holistic and linked approaches to pedagogy and assessment was where the impetus for the Scottish 2004 Curriculum Review came from, but this understanding was so muted that it allowed the conceptual framework to be subsumed into the conventional content-delivery, impact-outcome, performance-measurement, judgmentalspecifications mindset of the outgoing curriculum era. The Scottish schools’ inspectorate HMIE made this explicit by their approach to evaluating the new curriculum approach. They ignored it.

Please note that inspection teams are not evaluating progress with implementation of Curriculum for Excellence as such but evaluating the current work of the school using the five QIs8, QI 1.1, 2.1, 5.1, 5.3 and 5.9, including its impact on learners.9 Why weren’t they evaluating in accordance with the new schema “as such”? This was in 2011, a year after the full implementation of the new curriculum in August 2010, following its introduction in 2004 and the ‘Building the Curriculum’ process of staged intervention 20047

See Bower (2013).

Quality Indicators of the How Good Is Our School? measuring template used in Scottish school inspections (HM Inspectorate of Education. 2007).

9 HMIE (2011). HMIE Inspection Advice Note, Livingston: HM Inspectorate of Education.

8

5

2010. This was not engaged with, explicitly, in school regulatory evaluation. What did that “as such” mean other than to state that the inspectorate would not participate with schools in this ‘building’ process of collegiate construction between the levels and agencies of the system during this expressly major curriculum reform, described at ministerial level as “different in scale, scope and approach to any kind of educational development we have undertaken before”10? Would “evaluating the current work of the school using the five QIs 11, QI 1.1, 2.1, 5.1, 5.3 and 5.9, including its impact on learners” assist in understanding Mark, and relate to the deep formative assessment of the writing context? I do not contend that it would.

The ‘Es and Os’12 also typify this failure to grasp the conceptual pedagogic underlay of the new curriculum approach, as their bloated content and jargon-ridden content 13 now substitutes for the curriculum itself, in my view destroying a curriculum, and, more to the point, rethinking of the notion of curriculum14. If one starts with “using the Experiences and Outcomes to improve learners’ achievements and develop high quality learning and teaching;” or “improving standards of literacy and numeracy by involving all staff and tracking each learner’s progress”; or “developing ways of showing evidence of areas where learners’ experiences and outcomes have improved as a result”15, one will never get to understand a child like Mark. If ‘evaluated’ in the sense by which HMIE understand the term as grade levels of fixed specifications, by inspectors and Local Authority Quality Improvement Officers likewise, it will be misinterpreted by wrong criteria and criticised (the pupil and the teacher) by criteria which miss the point of the pupil’s real learning and the real development pathway in terms of its processes and context. Such constructionist understanding does not occur in Scottish school inspections and their Local Authority ‘quality assurance’ equivalents.

What are the ‘next steps’ here? To learn the difference between “would of” and “would have”, to ‘learn’ not to write pieces which are structurally weak in terms of plot, not to swear in an essay in a primary school, or to be explicit about ‘learning intentions, planned learning outcomes, teaching and learning sequence, assessment and next steps’?16. That makes my point. All those are wrong. The need is to penetrate the deeper meaning, from that grasp purpose, and from that, with the pupil, formatively distil developmental paths in terms of actions, searching out ways forward as new meanings emerge. Mark will do it for himself, in his engagement with others.

One could never understand the meaning of a piece of writing like this within the current ‘quality 10

Hyslop (2009) Quality Indicators of the How Good Is Our School? measuring template used in Scottish school inspections (HMIE 2007).

12 The prescribed ‘Experiences and Outcomes’ of the Curriculum for Excellence 13 E.g. for primary children “I am developing confidence when engaging with others within and beyond my place of learning. I can communicate in a clear, expressive way and I am learning to select and organise resources independently. LIT 2-10a / LIT 3-10a and, for nursery children, “I understand the importance of mental wellbeing and that this can be fostered and strengthened through personal coping skills and positive relationships. I know that it is not always possible to enjoy good mental health and that if this happens there is support available.” HWB 0-06a and “As I explore the rights to which I and others are entitled, I am able to exercise these rights appropriately and accept the responsibilities that go with them. I show respect for the rights of others.” HWB 0-09a. This is the mindset which is extinguishing cognition.

14 Bloomer, Keir (2008); MacKinnon, N., (2015). Sahlberg. P., (2005) 15 HMIE Briefing, Summer 2010 16 Derived from a ‘support’ visit and ‘plan’ of the quality assurance system for the school of the Local Authority in this study.

11

6

assurance’ framework of Scottish school education because the pedagogic framework is not set up to penetrate meaning and thereby the pupil’s social and developmental reality. It is a behaviourist stimulus-response mode of thinking pertaining to pedagogy and personal development, matched to a similar mindset in regard of professional and of institutional learning – that of scientific management through numeric measurement – which is why there is the obsession with ‘performance’, ‘tracking’, ‘outcomes’, ‘impact’, ‘indicators’ and the dismal deconstruction of the laudable intent to ‘declutter’ the curriculum by turning it into a labyrinthine compendium of 317 pages of densely worded semantically contorted ‘Es and Os’. With 688 of them for the primary stages, and a further 649 17 in high schools18, just what happened to the central ‘de-cluttering’ vision of the 2004 ‘A Curriculum for Excellence’ report? I have analysed this in depth in my paper A systems-thinking approach for school enhancement which I gave at International Conference of School Effectiveness and Improvement, Santiago, Chile, January 201319. My reason for doing this was out of frustration at a national system of education which has exalted performative, reductionist, behaviourist, business-management orientated notions of pedagogy and institutional development over all other learning criteria, to the extent of now rendering them unthinkable through mechanisms of power and control, which have drilled down into thinking itself. Beyond Scotland (and England) there is some very different thinking about education going on indeed. In Scotland this is now literally unthinkable in the very literal sense that the mental constructs by which learning may be conceptualised to deal and integrate the likes of Mark and his needs have been rendered as beyond thought, by a control system operating via mandated universalist audit idealisations20, engendering fear. It even imposed “The seven characteristics required for successful implementation of Curriculum for Excellence” 21 without even any ironic awareness that such imposition, with no negotiation or discussion, negates the whole basis of ‘building the curriculum’ by which the conceptual framework of the incoming curriculum approach was to be brought to practical and conceptual reality in Scottish schools, through constructing meaning in the context of our own circumstances.

I said in my 2013 paper in Santiago, Chile (Ibid), “the explicit and integral framework for Curriculum for Excellence does not require a prescriptive audit overlay. Indeed this destroys the whole endeavour. We do not need a regulatory priesthood interpreting the core texts ‘for’ us, imposing a ‘true faith’ in a simplified catechism – “the right direction” – then subjecting us to periodic inquisitions for seven-point ‘required characteristics’, and claiming to have ‘shared’ these with us.” (p6). Which brings me back to my own learning journey, and that of the school of this study, enabling the optimum progression of the capacities of children’s development – children like Mark and all the pupils there. The route to their developmental progression has to derive from within them. It requires psychological connection. The same is true for their teachers. The CfE Four Capacities cannot succeed and are not, since they are prevented from meeting the needs of Mark, in this sense.

I consider that the teacher got through to Mark but not in a way that the conventional performativity-deliverology method and mindset would ever penetrate. The “Es & Os” cannot be 17

18

Approximately, since my mind went foggy counting them Learning and Teaching Scotland (2009) Curriculum for Excellence: Experiences and Outcomes

19

MacKinnon, Niall (2013) through mechanisms further analysed and discussed in MacKinnon (2011) 21 HMIE (2011) 20

7

a starting point. They proclaim to enable and contain higher order skills, but the necessity is to start with the child and build up the progression of developmental understanding, in the wider capacities’ sense, and then integrate meaning and understanding in such a way that appropriate response is formatively derived and flexibly applied. The actual experiences and outcomes have to be situationally derived and contextually constructed. Tasks may then be selected to enable those, also with pupils having high ownership of these. That is what the teacher considered was done here, in regard of a pupil with profound emotional need, who was also operating at very high order levels in terms of awareness. He was both aware and vulnerable. He had high order need, but needed high order skills-based approaches to meet his needs. That requires a highly personalised approach within a class and a school, even if in a broader sense the activities of the class are not all individualised in the sense of separated. Crucially, their manner of application and tailoring may be, indeed should be, highly personalised.

Integral understanding as assessment This story revealed an aspect of the child's awareness and perception, when the means was there to 'see' it beyond a surface reading. It then 'fitted' with other learning outputs of this child – in various media. The process of integral understanding constitutes one understanding of assessment. The teacher came to perceive the need to see this piece of writing in terms of the whole child, but also for it to be seen in terms of the whole teacher, the whole school, the whole education authority, the whole national education system and indeed encompassing global education change. The teacher came to adopt similar specific manifestations of children’s development as the essential evidence comprising assessment. More pertinently the teacher and the school were redefining assessment. More conventional performance criteria can be fitted in, but are a very narrow subset of an altered pedagogic mode of a higher order interpretation of Scotland’s ‘Four Capacities’.

There is a profound need to absorb variety of potential, capability and circumstance to place a central focus on purpose in brokering education change. This requires understanding and negotiation, not prescription and delivery. Scotland's 'Four Capacities' enabled schools to realign with the wider perspectives of children's capability. Potential is there to be realised – the meaning of capacity. The shift of emphasis was of the latent variety of children's potentials. Those require to be revealed, having profound implications for assessment, being the means by which educators may understand and record development and progression. But it may not be linear or numeric.

The relationship between assessment and pedagogy is integrative not mechanistic. It is an interpretative process, informing learning, personal development and enactment between students/pupils and educators, and the wider systems. The most important collaborative partnerships to be opened up and deepened are between educators and pupils/students. The wider systems need to support those. They can only do so if they engage with local meaning and purpose, and in so doing ‘absorb variety’ 22. The variety to be absorbed is the immense diversity of the characteristics of the students/pupils as set in their varied local contexts.

22

This forms a component part of the Vanguard Method, a process to understand work as a system, developed by John Seddon for public services https://vanguard-method.net/library/systems-principles/absorbing-variety/

8

Conclusion The focus of this paper has been a single piece of imaginative writing. The new Scottish curriculum focus on personalisation and capacities of children's potential caused the teacher to 'see' this story differently. The teacher came to realise that through the literary device of the story the pupil was telling the reader something of their own personal development and of their own needs. There was nothing sinister, implying anything untoward, but it became apparent that the simplistic language and plot were part of what that child was conveying – the meta-story. The teacher came to see the piece as sophisticated, despite the simplicity of surface structure. The shift of meaning came from an understanding of assessment which broke through the narrower confines of standardised approaches. This child was 'closing the gap', and so was the school through its changing approach to pedagogy, but in an entirely different conceptual arena to the conventional performative implications of that term23.

References Bloomer, K., (2008). The dangerous word is curriculum, Times Educational Supplement for Scotland, 15 February https://www.tes.com/news/tes-archive/tes-publication/dangerous-word-curriculum Bower, J., (2013). ‘Reduced to numbers: from concealing to revealing learning’ in: Bower, J., and Thomas, P.L. (eds), (2013) De-testing and de-grading schools, Peter Lang: New York HMIE, (2007). How Good is Our School?. Livingston: HM Inspectorate of Education.

HMIE, (2010). HMIE Briefing Summer 2010, Livingston: HM Inspectorate of Education HMIE, (2011). HMIE Inspection Advice Note, Livingston: HM Inspectorate of Education.

Hyslop, F., (2009). Curriculum review experiences and outcomes, Letter to all school staff, The Scottish Government. May Learning and Teaching Scotland, (2009). Curriculum for Excellence: Experiences and Outcomes MacKinnon, N., (2011). ‘The urgent need for new approaches in school evaluation to enable Scotland’s Curriculum for Excellence’, Educational Assessment, Evaluation and Accountability, Volume 23, Number 1. February http://link.springer.com/article/10.1007/s11092-011-9116-4 MacKinnon, N., (2013). A systems thinking approach for school enhancement, 26th International Congress of School effectiveness and Improvement, Santiago, Chile MacKinnon, N., (2015). Inter-school collaboration: a practitioner perspective, ICSEI Express and Digest, Volume 6 Issue 2, The International Congress of School Effectiveness and Improvement. http://www.icsei.net/index.php?id=1774 (p6) Oracle Education Foundation, (2009). The Power of Project Learning with Thinkquest, prepared by SRI International http://www.ciosummits.com/media/pdf/2009_11/Oracle_Power_Project_Learning.pdf Sahlberg. P., (2005). ‘Curriculum change as learning - in search of better implementation' in: Curriculum Reform and Implementation in the 21st Century – Policies, Perspectives and Implementation. International Conference on Curriculum Reform and Implementation in the 21 st Century: Policies, Perspectives and Implementation.’ Istanbul, Turkey. http://www.sac.smm.lt/bmt/wp-content/uploads/2009/09/1Passi-Sahlberg-Curriculum-change-as-learning.pdf Vanguard Consulting (website). The Vanguard Method https://vanguard-method.net/ 23

For a an exposition of assessment as revealing learning from the same conceptual realm watch the presentation by Joe Bower, which I attended, ‘Do schools conceal or reveal learning’, given at the conference ‘Revisiting the fundamentals of traditional curricula, R/Evolution: what “R” would mean for education’ Barcelona, Spain, 1-2 December 2014. Note in particular the video excerpt of a young girl’s skiing lesson, and the assessment conclusions which Joe drew: http://www.uoc.edu/portal/en/symposia/unesco-chair-seminar-2014/Videos/index.html and Bower (2013).

9