Second order learning in developmental evaluation new methods for complex conditions andrew mitchell

Page 1


Second order Learning in Developmental Evaluation New Methods for Complex Conditions

Visit to download the full and correct content document: https://textbookfull.com/product/second-order-learning-in-developmental-evaluation-n ew-methods-for-complex-conditions-andrew-mitchell/

More products digital (pdf, epub, mobi) instant download maybe you interests ...

Comprehensive Practice of Exploration and Evaluation Techniques in Complex Reservoirs Xiaodong Cheng

https://textbookfull.com/product/comprehensive-practice-ofexploration-and-evaluation-techniques-in-complex-reservoirsxiaodong-cheng/

Spinal Conditions in the Athlete A Clinical Guide to Evaluation Management and Controversies Wellington K. Hsu

https://textbookfull.com/product/spinal-conditions-in-theathlete-a-clinical-guide-to-evaluation-management-andcontroversies-wellington-k-hsu/

Conversation Analytic Research on Learning in Action The Complex Ecology of Second Language Interaction in the wild John Hellermann

https://textbookfull.com/product/conversation-analytic-researchon-learning-in-action-the-complex-ecology-of-second-languageinteraction-in-the-wild-john-hellermann/

Computer Vision Technology for Food Quality Evaluation, Second Edition Sun

https://textbookfull.com/product/computer-vision-technology-forfood-quality-evaluation-second-edition-sun/

Acquisition of Complex Arithmetic Skills and Higher Order Mathematics Concepts Volume 3 Mathematical Cognition and Learning 1st Edition David C. Geary

https://textbookfull.com/product/acquisition-of-complexarithmetic-skills-and-higher-order-mathematics-conceptsvolume-3-mathematical-cognition-and-learning-1st-edition-david-cgeary/

Italy in the New International Order, 1917–1922 Antonio Varsori

https://textbookfull.com/product/italy-in-the-new-internationalorder-1917-1922-antonio-varsori/

Structural Methods in the Study of Complex Systems

Elena Zattoni

https://textbookfull.com/product/structural-methods-in-the-studyof-complex-systems-elena-zattoni/

Methods for the economic evaluation of health care programmes Fourth Edition Drummond

https://textbookfull.com/product/methods-for-the-economicevaluation-of-health-care-programmes-fourth-edition-drummond/

Modeling Behavior in Complex Public Health Systems

Simulation and Games for Action and Evaluation 1st Edition Christopher Keane Mph Scd

https://textbookfull.com/product/modeling-behavior-in-complexpublic-health-systems-simulation-and-games-for-action-andevaluation-1st-edition-christopher-keane-mph-scd/

Second-order Learning in Developmental Evaluation New Methods for Complex Conditions

Second-order Learning in Developmental Evaluation

Andrew

Second-order Learning in Developmental Evaluation

New Methods for Complex Conditions

ISBN 978-3-319-99370-6 ISBN 978-3-319-99371-3 (eBook)

https://doi.org/10.1007/978-3-319-99371-3

Library of Congress Control Number: 2018951582

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifcally the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microflms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specifc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affliations.

Cover illustration: © Melisa Hasan

This Palgrave Pivot imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Acknowledgements

If it takes a village to raise a child, then writing a book requires at the least a coherent community of support. This book is no different, and I have been blessed with some considerable and high calibre support, which I would like to briefy acknowledge. Thank you to Gabriel Everington from Palgrave Macmillan for his support on behalf of the Pivot series of books, and to Madeleine Holder before him. Also, thanks to Klaus W. Mueller from Carl-Auer Verlag GmbH for their generosity in allowing me to use images from one of their published interviews with Humberto Maturana. My deep appreciation to Humberto Maturana, and the late Francisco Varela, whose work has been the primary source of inspiration for this book, and to Professor Ray Ison for reviewing the initial proposal and endorsing it to Palgrave. Also, thanks are due to the team of practitioners from the case study community-based sustainability initiative for their patience and for indulging me and my never-ending questions.

I would also like to extend my appreciation to Professor Mark Lemon for many stimulating hours of bouncing ideas backwards and forwards, and for his gentle challenges to ask the ‘So what?’ question to keep these heady ideas grounded. I thank my late parents for their love, support, and for believing in the value of a good education which, in many ways, set me along the trajectory I have pursued. Of course, none of this would have been possible without the love and support generously offered by my dearest friend and life partner, Sharon Ann. It is to you that this work is dedicated, with much love.

This book is for all of those who confront the practical challenges of responding to wicked problems, a process described as akin to being “passengers on an aircraft we must not only fy but redesign in fight” (Sterman, J. D. 2000. Business dynamics: Systems thinking and modeling for a complex world (p. 4). Boston: Irwin McGraw Hill).

Finally, despite the support of the aforementioned, any errors of omission or commission in the following text are my own responsibility.

list of figures

Fig. 3.1 Structural coupling (Source Maturana and Poerksen 2004: 86) © Carl-Auer Verlag

65

Fig. 3.2 Structural coupling between two unities and a medium, giving rise to a linguistic domain (Source Maturana and Poerksen 2004: 87) © Carl-Auer Verlag 66

list of tAbles

Table 3.1 Comparison of three generations of cognitive science

Table 4.1 Code categories for the thematic analysis

Table 5.1 Guidance for recruiting practitioners into orthogonal interactions

59

107

133

CHAPTER 1

Introduction

Abstract Mitchell reviews a number of the challenges encountered in evaluating community-based sustainability and international developmental aid initiatives using traditional impact and process evaluation methods. Given the pressing urgency of the Anthropocene, Mitchell argues that project-based learning is an asset, and identifes developmental evaluation as an emerging approach that facilitates project actors to identify, acquire and use their experiential learning to innovate responses to wicked problems.

Keywords Anthropocene · Wicked problems

Monitoring and evaluation frameworks · Impacts · Processes Post-normal and complexity science

The UK, as part of its policy framework response to the ever-mounting evidence of anthropogenic climate change (Hansen et al. 2015, 2016), has passed ambitious legislation committing the government to an 80% reduction in greenhouse gas emissions over the 1990 baseline by 2050 (H.M. Government 2008). This is an ambitious commitment to an overwhelming challenge, as briefy reviewed below.

Since the Industrial Revolution, and rapidly picking up pace following the end of the Second World War, humanity has presided over a dramatic increase in atmospheric greenhouse gases triggering climatic change and global warming (Hansen et al. 2005; EEA 2012). Such increases are

© The Author(s) 2019

A. Mitchell, Second-order Learning in Developmental Evaluation, https://doi.org/10.1007/978-3-319-99371-3_1

associated with the release of vast quantities of primarily carbon dioxide (CO2). In turn, these increases trigger positive (amplifying) feedback in the form of warming melting the permafrost and releasing unknown volumes of methane1 (CH4) (Hansen et al. 2007; Intergovernmental Panel on Climate Change [IPCC] 2014).

Such changes in atmospheric chemistry are amplifed by a diminishing planetary albedo, caused by melting polar ice caps and glaciers. As the planet gradually blackens, it absorbs more heat than it refects. Taken together, the systemic properties of the greenhouse effect, leading to global warming and climate change impacts, give rise to a positive feedback loop as a continued escalation of those very properties of the system that bring about amplifed change. The pace at which these changes are accelerating suggests that previous assessments of an accepted upper global temperature tolerance level set at “2 °C now represents the threshold of extremely dangerous climate change”, and yet remains a level society may have “little to no chance of maintaining” (Anderson and Bows 2011: 41).

In addition to clear existential risks from climate change, evidence is mounting of a perfect storm of converging threats which has been designated the Anthropocene epoch (the age of the human) (Crutzen and Stoermer 2000; Zalasiewicz et al. 2010; Verburg et al. 2015). While its precise point of origin is currently under debate a number of candidate starting points have been mooted. For example, one candidate is the dawn of agriculture when humans frst sought to control and bend nature to our own ends (Lyons et al. 2015), while another is the Industrial Revolution and the magnifcation in scale of human power afforded by burning fossil fuels (Waters et al. 2016).

However, consensus has recently converged around the detonation of the frst atomic devices in July 1945 during the Trinity nuclear tests as the most likely point at which the Anthropocene started, because globewide changes in the stratigraphic record can be identifed in the sudden appearance of plutonium, caesium, and strontium at this time (Waters et al. 2015). The signifcance of this new epoch is that it signals the end of the Holocene, a period of some 11,500 years characterised by moderate

1 Methane is generally taken as a more potent contributor to the greenhouse effect by several orders of magnitude relative to CO2, albeit with a considerably shorter half-life. A prolonged release of previously trapped CH4 into the atmosphere will greatly accelerate global warming and climate change effects.

and stable temperature ranges conducive to human development and fourishing. In contrast, the newly emerging epoch is characterised by profound uncertainty and risk for the future of human civilisation.

There are a number of markers associated with the dawn of the Anthropocene, in addition to the sudden marked appearance of radioactive isotopes. Evidence for the new epoch includes climate change, the human precipitation of a sixth biological mass extinction event (Novacek and Cleland 2001; Ceballos et al. 2015), patterns of transgressing planetary processes beyond a safe operating space (Röckstrom et al. 2009), and signifcant changes to land cover biomes (Haberl et al. 2007; FischerKowalski et al. 2014) and hydrological systems (Nilsson et al. 2005).

In addition to these key markers, more recent estimates are that intensive agriculture has reduced arable soil to the extent that there are approximately 60 years of harvests left in the available soil resources (Food and Agriculture Organization of the United Nations [FAO] and Intergovernmental Technical Panel on Soils [ITPS] 2015), while the regeneration of three centimetres of top soil is a process that takes up to 1000 years. When these anthropogenic impacts are viewed collectively, some have been prompted to reconsider the utility of concepts such as sustainability in favour of a renewed emphasis on adaptation. Such emphasis necessitates radical changes to economic infrastructures and our modus vivendi (Dumanoski 2009; Craig and Benson 2013; Benson and Craig 2014; Foster 2015). Still others have even begun contemplating the likelihood of an end to civilisation as we know it (Kingsnorth and Hine 2009; Diamond 2011; Scranton 2015). Clearly, threats to human sustainability exemplify, par excellence, a wicked problem (Rittel and Webber 1973), characterised as a set of challenges that have no defnitive formulation, that tend not to resolve in a true or false outcome and which have no specifc stopping point.

Against these alarming and apocalyptic trends, the UK government introduced a national community energy strategy, in which are outlined the scope of the anticipated contribution community-based sustainability (CBS) projects can make to helping the government attain its 80% reductions, as per the Climate Change Act 2008. Building on the devolution of powers enshrined in the UK Localism Act (H.M. Government 2011), the national community energy strategy calls upon CBS projects to contribute to a rigorous and robust “evidence base”, in order to demonstrate “their effectiveness, fnancial sustainability and wider social benefts to secure investment” (Department of Energy and Climate

Change [DECC] 2014: 45). As per the Localism Act 2011, CBS projects are becoming increasingly expected by policy-makers to contribute to the establishment and control of localised community-scale sources of energy supply and demand management (Bradley 2014; Aiken 2015).

As the policy and performance pressures on CBS practitioners mount, it is apparent that funded CBS projects are coming under increasing scrutiny to not only deliver on community sustainability outcomes, but to be able to evidence the extent of this delivery. This mounting pressure ups the ante, in turn, for CBS practitioners to ensure that their monitoring and evaluation (M&E) practices are appropriately framed in order to make a strong case for demonstrating the added value such projects bring to local, regional, and national climate change mitigation and adaptation strategies. As arms-length delivery vehicles for the UK national energy strategy to help realise the ambitions enshrined in the Climate Change Act 2008, CBS projects have become—at least to some extent—accountable to national UK policy frameworks, even if such commitment is mitigated via the channels of donor funding organisations.

Before continuing, it is necessary to say a few words about CBS projects. To be clear, there is no standard CBS initiative model. Instead, this descriptor refers to a heterogeneous group of actors who have in common the intent to “serve the environmental and social sustainability needs and interests of (mostly) place-based communities [and] may operate for proft or not” (TESS 2016: 1). Because there is no standardised model, initiatives range from a full complement of paid staff to volunteer-only projects, some being formally constituted as charities, while others are more ‘organic’ and emerge from local interest to undertake a specifc activity (e.g., a local garden scheme). While the CBS initiative as a class of community activities may have its roots in the environmental pressure groups emerging in the late 1960s and 1970s, examples of CBS initiatives include, but are not limited to, the transition town network and smaller neighbourhood interest groups, donor funded and formal projects such as the Good Life Initiative (funded by Joseph Rowntree) or the Communities Living Sustainably (CLS) cluster of projects (funded by Big Lottery).

The focus here is on the latter type of project—those that are formally constituted, funded, staffed, and accountable to external bodies for performance relative to agreed outcomes. The outcomes to which such projects work are generally supported by monitoring and evaluation (M&E) frameworks, of varying sophistication, and are also broadly aligned with key policy concerns, such as fuel poverty in the case of the

CLS projects. Consequently, CBS initiatives confgured in such ways most closely parallel initiatives designed, funded and deployed for international developmental purposes, and are therefore also most appropriate to be considered as arms-length policy delivery vehicles.

This shift in the UK government’s emphasis on local not-for-proft CBS (community-based sustainability) projects as quasi delivery vehicles for the UK national energy commitments is not without its problems. One of the sources of these problems concerns the M&E practices of CBS projects. As observed over a decade ago, there “is a lack of defnitive evidence on the impact, costs and benefts of community initiatives designed to secure individual behaviour change [with respect to] ‘low carbon lifestyles’” (Letcher et al. 2007: 4).

While it is diffcult to ascertain the degree to which such concerns about the lack of impacts still hold some ten years after Letcher’s et al. (2007) meta-review was published, it is clear that CBS projects refect a considerable range and diversity in how they are organised, their objectives and ambitions, their achievements, and even the length of time for which they have been active (Seyfang et al. 2013).

What this means is that the impact and effective contribution of such projects warrants on-going evaluation (Hamilton 2013; Seyfang et al. 2014), especially as, according to DECC,2 there are in excess of 500 such community-based projects in the UK at present (DECC 2014) with nearly 500 chapters of the transition town model across the world who have ‘signed up’ to the transition initiative protocol.3 It may be anticipated then that the quality of M&E practices across this heterogeneous sector will be highly variable.

While “community initiatives have proven diffcult to evaluate because they do not lend themselves to traditional experimental methods” (Milligan et al. 1998: 45), community initiatives also pose their own problems for evaluation even when taken on their own terms. These challenges extend across the gamut of the three evaluation types identifed in the UK Government’s ‘Magenta Book’ (HM Treasury 2011)— process (how was the endeavour delivered), impact (what difference did the endeavour make), and economic (do any benefts accrued from the endeavour justify the costs). The present focus restricts itself to process

2 In early 2016, the Department for Energy and Climate Change (DECC) was rebranded as BEIS (the Department for Business, Energy & Industrial Strategy).

3 https://transitionnetwork.org/initiatives/by-number. Accessed 11 March 2016.

and impact evaluations only, and these certainly appear to be the more prevalent approaches used in the evaluation of community sustainability and development project efforts. Economic evaluation tends to be of the form of post hoc value for money summative reviews.

A 2007 meta-review of community-based energy initiatives concluded that “[e]valuation of the impact of community initiatives on individual behaviour is generally low quality and terms such as ‘behavioural change’ and ‘behavioural measures’ were not meaningful to community groups” (Letcher et al. 2007: 5). Indeed, for some time now such projects have “been struggling to fnd evaluation strategies and methodologies that correspond well to the goals and designs of the initiatives themselves” (Connell and Kubisch 1998: 15). In a controversial critique of the impact of the American environmental movement, for example, it was noted that, considering “the long string of global warming defeats [in America], it is hard not to conclude that the environmental movement’s approach to problems and policies hasn’t worked particularly well” (Shellenberger and Nordhaus 2004: 7). Indeed, the literature on effectiveness research of community-based campaigns and initiatives to address climate change and the anthropogenic causes of greenhouse gas emissions do not return particularly favourable results (Kern and Smith 2008; Stephenson et al. 2010). This claim seems substantiated by reports that climate change continues, with CO2 levels now higher than at any point during the previous 23 million years (Hamilton 2016). Although emissions have ‘plateaued’ for the three years till 2017 due to land use changes, estimates suggest that from 2017 these will begin to rise again (Hausfather 2017).

Even as we know more about the reality of climate change, it seems that our concern about climate change declines (Shi et al. 2016). There is compelling evidence that scare-tactic and single issue campaigns peak in their impact quickly but have short-lived and limited scale effects (Crompton and Kasser 2009). Research suggests that in the face of reminders of one’s own death, there’s an increase in the tendency to deny one’s connectivity with non-human life, attitudinal swings towards increased religiosity and a right wing politics, and lowered tolerance for change and uncertainty (Rosenblatt et al. 1989; Greenberg et al. 1990; Goldenberg et al. 2001). In turn, these responses tend to be positively correlated with consumption and the generation of increased levels of greenhouse gas emissions and a corresponding decline in concern about anthropogenic global warming (Dryzek 2013).

Given the variable outcomes afforded by this sector to date, “the emphasis placed on evidence within wider knowledge, research, and learning systems of NGOs has changed from negligible prominence to a more central notion within NGO discourse, policy, and practice” (Hayman 2016: 129). NGOs and funded community-based initiatives must demonstrate the delivery of specifc objectives, measured against key performance indicators (KPIs) and other metrics, and increasingly operate as policy vehicles and knowledge management systems evidencing change attributable to the service. However, what counts as evidence of impact varies signifcantly, both due to sector and regional contexts, but also because clearly specifed defnitions are not widely shared (Gooding 2016).

There is a clear need for M&E research to make a contribution to CBS initiatives, so that “M&E can have an overtly strategic function […] using certain representations of impact to gain entry into spheres of infuence that may enable groups’ ethos and aims to gain further public traction with the public or other stakeholders” (Hobson et al. 2016: 14). Such ‘representations of impact’ must support stakeholders in estimating “these initiatives’ effects on interim and longer-term outcomes and the need for information on how the interventions produce those outcomes” (Connell and Kubisch 1998: 15, added emphasis). To gain traction into such ‘spheres of infuence’, capturing project actor experience and learning must become a priority for M&E practice.

There is little available research into how CBS projects use monitoring and evaluation (M&E) in their practice. Consequently, the EVALOC4 research project was designed to understand how twenty UK-wide community-based low carbon groups might best identify, evidence, and communicate their impacts and contributions to mitigating greenhouse gas emissions and facilitating community adaptations to climate change. One of the main conclusions from EVALOC was that there is “a defnite shortfall in groups’ current abilities to capture lessons learnt from project and activities, in part not only due to skills and ‘know how’, but also due to a lack of time and direct incentive to do so” (Hobson et al. 2016: 9). As a result, practitioner learning is not being captured and fed forward into future project design, even though it is a key resource and asset. Arguably, this defcit is exacerbated by CBS practitioners themselves,

4 EVAluation of LOw Carbon communities (EVALOC). http://www.evaloc.org.uk

Accessed 2 September 2015.

for whom the “[m]onitoring and evaluation of targets is often seen as a box-ticking exercise, or an onerous activity that communities do not have the resources to undertake” (Merritt and Stubbs 2012: 101), and is of a low priority among practitioners relative to their other commitments (Hobson et al. 2014, 2016).

The challenges confronting CBS practitioners in undertaking M&E also extend beyond issues of time and resource constraints and skillsbased capacities. A signifcant barrier practitioners face in evidencing impacts of their efforts is due to the nature of what it is they are attempting to evidence. Natural-science oriented research into sustainability attempts to fnd an apolitical empirical basis with which to establish what constitutes sustainability, and any changes in the domain of interest are established in relation to those metrics, for monitoring climatic and Anthropocene impacts. Natural sciences cannot account for the social value basis that underpins and conditions what is deemed important enough to sustain (Blühdorn 2007, 2011). In practice, the challenge to develop sustainability indicators (SIs) that are “globally comprehensible, [but] locally derived” is a “highly politicised process” (Elgert and Krueger 2012: 565), fraught with competing value systems and multiple perspectives (Bell and Morse 2008).

It is worth noting that the paradigm within which SIs exist as meaningful markers of change, and the epistemological claims that afford merit to the idea of ‘globally comprehensible’ metrics, is itself predicated on the so-called “normal scientifc” paradigm (Funtowicz and Ravetz 1993). This paradigm is characterised by causal relations among components that correspond to the three Newtonian laws of motion, and which presuppose a neutral or objective observer who describes a series of events from what is assumed to be a value-free vantage point, as if a view from nowhere or a God’s eye perspective (Maturana 1988). As a result of the focus on SIs, practitioners of formally constituted and funded CBS initiatives may be attempting to apply “normal scientifc” methods to monitor and evaluate activities within a domain of operation which is better accounted for by a more complex paradigm (Chambers 2014).

The mode-2 knowledge production perspective lends credence to this view that problems of community sustainability refect multiple perspectives and vested interests, thereby endorsing the recognition that such initiatives operate within inherently complex domains and require cross-sector collaborations (Regeer et al. 2009). This is in contrast with mode-1 knowledge production which tends to be academically-oriented

and constrained to discipline-specifc forms of investigator led research (Gibbons et al. 1994; Nowotny et al. 2001; Hessels and van Lente 2008). Mode-2 production elicits an emergent design protocol invoking experimentation and engaging participant actors in a process of ‘double loop’ learning. While single loop learning is characterised by reconsidering decisions made in light of new information, double loop learning involves a re-evaluation of the assumptions themselves that informed a given decision (Argyris and Schön 1978). In Bateson’s (1972) studies on cybernetic recursion, he also referred to triple ‘loop’ (deutero or second order) learning to describe the recursive practice of learning about the process of learning itself. Second order learning incorporates the context under which particular learning holds as valid as part of the learning itself (Bateson 1972, 1979).

Monitoring and evaluating impacts attributable to CBS projects is hence far from straight forward. Indeed, evaluating how sustainability initiatives are delivered (that is, “process evaluation”) raises a higher degree of complexity than does impact evaluation. Unlike the latter, process evaluations will be determined by the focus of the initiative and not by any “simple, generic characterisation of questions such as those that tend to be applicable […] for impact evaluation” (HM Treasury 2011: 18). Monitoring impacts within complex systems is therefore, itself, a complex matter, and the applicability and validity afforded by linear descriptions of causality to account for change give way to narratives of emergence, refexivity, and cross-scalar infuences (Cilliers et al. 2013; Byrne and Callaghan 2014; Wise et al. 2014).

This epistemological shift exposes the short-fall in standardised evaluation strategies to tackle the complexities of community sustainability and the “plurality of values and perspectives, permanent uncertainties, and pervasive interconnectedness between ecological, social, institutional, political, and economical system[s]” (Regeer et al. 2009: 521). Attempts to identify and link interventions and impacts in terms of a model of linear causality are consequently regarded as inadequate. Because evaluation involves making sense of data and exercising judgement as to the degree of alignment between how the problem is defned, and the outcomes by which the impacts on this are known, evaluations are subject to the framework of distinctions drawn by an observer (Kaufmann 2011; Nielsen 2016), even though this is rarely explicitly acknowledged in M&E practice.

Consequently, CBS initiatives may be constrained in responding meaningfully as policy vehicles to provide robust and rigorous evidence

of effectuating changes in the energy-related behaviour of communities, as per the national energy strategy (DECC 2014) to help deliver against the UK Climate Change Act. The matter is further complicated by how CBS projects’ experience and attend to critical M&E processes, the utility of the M&E frameworks with respect to the epistemology that informs project design, and even the level of expertise and the resource capacity of CBS practitioners to engage in rigorous M&E practices (Hobson et al. 2014, 2016).

While the EVALOC research only tapped into a small percentage (~ 4%) of DECC’s estimated 500 CBS initiatives and did not claim to represent the sector, when this and other reviews of CBS initiatives (e.g., Elgert and Krueger 2012) are read alongside available research from international development agency M&E work (Powell 2006; Ravallion 2008; Ramalingam 2013; Chambers 2014; Burns and Worsley 2015), a similar pattern emerges. Even though this sector may be argued to be populated with “knowledge-intensive organizations” (Lettieri et al. 2004: 17), on the whole, it also tends to be defcient in the formal processes of capturing, sharing, and diffusing knowledge (Hume and Hume 2008).

Most case study research of CBS projects tend to focus on how the project engaged with members of the host community, the deployment of particular interventions and activities and report on their relative effcacy and the strategic decisions made by the project leads (e.g., Cinderby et al. 2014). These case studies are critical in contributing to the accumulation of both academic and practitioner understanding of deploying CBS projects. What also is necessary is for research efforts to track how front-line CBS practitioners make sense of the complexities of the arena within which they operate. This means that project evaluation is to straddle the division between processes and impacts. In many cases, practitioners are not responsible for designing the project, but are hired post hoc to deliver on a set of outcomes already agreed to and funded. Certainly this is the instance in the twelve CLS projects funded by BIG Lottery, one of which is the extended case study project explored in this research. Practitioners are therefore constrained by the set of outcomes to which the funding proposal has committed itself and seek to deliver as best they can against those outcomes in a way that is consistent with their operating theory of change.

Consequently, evaluation research should shift away from a sole focus on the Magenta Book type distinction between what is a process or an impact evaluation and instead seeks to explore a more cross-cutting

cognitive, or sense-making, terrain with practitioners. In this book, I take as my starting point the phenomenological perspective of front-line CBS practitioners who, upon agreeing to act as practitioners, fnd themselves confronted with and attempting to reconcile a set of agreed outcomes for which they are held accountable for delivering against. This research approach uses the theory of change articulated by practitioners and their knowledge claims about causality and how change is induced, elicited, and brought about within the constraints of complex and dynamic social, ecological, and economic systems. The research recognises explicitly that such systems are the expression of multiple perspectives, each with vested interests that may or may not coincide with the objectives of the project. This is consistent with what Patton terms a ‘developmental approach’ to evaluation (Patton 1996, 2011), commensurate with the complex contexts within which CBS initiatives operate.

Traditional approaches to evaluation broadly make judgements about how the evaluand has performed relative to the outcomes of the project. What is seldom taken into account however are any differences between how practitioners frame the causes (diagnostic framing) and how funders and designers frame the solutions to the problem that motivates the project’s initiation. Both perspectives involve making sense of complex data, drawing on signifcant assumptions about how the world is thought to work. What is often overlooked in traditional evaluations is the learning that practitioners acquire through the process of reconciling the outcomes framework with the operational constraints of delivery, unless this is explicitly included as an objective.

The focus of this book explores and explicates how front-line CBS practitioners conceptualise change, measure its scale and plot its direction of travel, correlate interventions and effects, evaluate impacts, and draw on their accumulating wealth of knowledge to foster and diffuse learning within their own project. The research reported in this book intends to make a contribution to this increasingly important domain of knowledge which has been hitherto under-researched and concerns how practitioners themselves evaluate and implement their learning in the course of delivering on the CBS initiative they are responsible for. The approach refects the philosophy of Utilisation-Focused Evaluation5

5 UFE is concerned, as a methodology, with designing evaluations that are useful and relevant for decision making by priority users and stakeholders to obtain and apply the fndings to improve project outcomes (Patton 1997).

(UFE) approach (Patton 1997; Stuffebeam and Shinkfeld 2007) by considering how practitioners themselves construe and deploy evaluative techniques and how they then utilise fndings contemporaneously to the on-going process of delivery.

While the preceding discussion has concerned sustainability, the point has been to fesh out the nature of one of the most pressing exemplars of a wicked problem in modern times. However, the focus of this book is not about sustainability per se, nor about effective interventions as such. Rather this is a consideration of how actors in an intervention project learn to learn to be relevant with respect to the operational conditions characterised by the wicked and complex problem of sustainability. That is, how a CBS initiative becomes a second order learning system.

The argument of this book concurs with Moore’s (2011: 27) claim that “[h]ow wicked problems are resolved is determined by the means and methods used to make sense of them”, and in recognition of this offers a detailed account of the role the observer plays in making sense of the complexity inherent to CBS initiative work. In brief, this foregrounds the observer as determining the description and interpretation of the system being observed. While frst-order cybernetics (or broadly, systems theory) concerns itself with observed systems, the study of the observing system is the concern of second-order cybernetics, and is the key to understanding second-order learning systems. The concepts of second-order cybernetics and second order learning bring an epistemological coherence to the theory and practice of developmental evaluation.

The following chapter introduces and reviews the practice of evaluation as it pertains to initiatives that respond to complex and wicked problems, such as CBS projects. The shortcomings inherent to traditional modes of M&E when it comes to such initiatives are reviewed, and the chapter then introduces the developmental evaluation (DE) approach which was designed to ameliorate these practice-based needs. As will be subsequently discussed, while the emerging DE approach flls a clear practitioner-based need for evaluation that is oriented to innovative projects working under complex and uncertain conditions, as a result of it still being relatively recent in its own development, it currently lacks an epistemological framework which helps to cohere it. It is at this point that the second-order cybernetic paradigm can make a meaningful contribution to the approach.

This then is the focus of Chapter 3 which introduces the logic of recursion, drawing from the neurobiological research of Maturana and

Varela, the calculus of distinctions, and an understanding of human systems as recursive linguistic systems. Chapter 4 operationalises the theory of second-order cybernetics through an extended case study of a fve year fully funded English CBS initiative. The book concludes with a recap of the main points raised and discussed, identifes limitations of the current study and notes potential areas for future research.

references

Aiken, G. T. (2015). (Local-) community for global challenges: Carbon conversations, transition towns and governmental elisions. Local Environment, 20(17), 764–781. https://doi.org/10.1080/13549839.2013.870142

Anderson, K., & Bows, A. (2011). Beyond “dangerous” climate change: Emission scenarios for a new world. Philosophical Transactions of the Royal Society A, 369, 20–44. https://doi.org/10.1098/rsta.2010.0290.

Argyris, C., & Schön, D. (1978). Organizational learning: A theory of action perspective. Cambridge, MA: Addison-Wesley.

Bateson, G. (1972). Steps to an ecology of mind. New York: Ballantine Books. Bateson, G. (1979). Mind and nature: A necessary unity. New York: Bantam. Bell, S., & Morse, S. (2008). Sustainability indicators: Measuring the immeasurable? (Revised). London: Earthscan.

Benson, M. H., & Craig, R. K. (2014). The end of sustainability. Society & Natural Resources, 27(7), 777–782. https://doi.org/10.1080/08941920.201 4.901467.

Blühdorn, I. (2007). Sustaining the unsustainable: Symbolic politics and the politics of simulation. Environmental Politics, 16(2), 251–275. https://doi. org/10.1080/09644010701211759.

Blühdorn, I. (2011). The politics of unsustainability: COP15, post-ecologism, and the ecological paradox. Organization & Environment, 24(1), 34–53. https://doi.org/10.1177/1086026611402008.

Bradley, Q. (2014). Bringing democracy back home: Community localism and the domestication of political space. Environment & Planning D: Society & Space, 32(4), 642–657. https://doi.org/10.1068/d17312

Burns, D., & Worsley, S. (2015). Navigating complexity in international development: Facilitating sustainable change at scale. Rugby: Practical Action Publishing Ltd.

Byrne, D., & Callaghan, G. (2014). Complexity theory and the social sciences: The state of the art. Abingdon, Oxon: Routledge. Ceballos, G., Ehrlich, P. R., Barnosky, A. D., García, A., Pringle, R. M., & Palmer, T. M. (2015). Accelerated modern human–induced species losses: Entering the sixth mass extinction. Science Advances, 5, 9–13. https://doi. org/10.1126/sciadv.1400253.

Chambers, R. (2014). Knowing in development: A radical agenda for the twenty-frst century. Forum for Development Studies, 41(3), 525–537. https://doi. org/10.1080/08039410.2014.959376.

Cilliers, P., Biggs, H. C., Blignaut, S., Choles, A. G., Hofmeyr, J. S., Jewitt, G. P. W., et al. (2013). Complexity, modeling, and natural resource management. Ecology and Society, 18(3), 1. https://doi.org/10.5751/ES-05382-180301

Cinderby, S., Haq, G., Cambridge, H., & Lock, K. (2014). Practical action to build community resilience: The Good Life initiative in New Earswick. Joseph Rowntree Foundation. Available at: http://www.jrf.org.uk/sites/fles/jrf/ community-sustainability-environment-full.pdf.

Connell, J. P., & Kubisch, A. C. (1998). Applying a theory of change approach to the evaluation of comprehensive community initiatives: Progress, prospects, and problems. In K. Fulbright-Anderson, A. C. Kubisch, & J. P. Connell (Eds.), New approaches to evaluating community initiatives. Volume 2: Theory, measurement, and analysis (pp. 15–44). Washington, DC: Aspen Institute.

Craig, R. K., & Benson, M. H. (2013). Replacing sustainability. Akron Law Review, 46(4), 841–880.

Crompton, T., & Kasser, T. (2009). Meeting environmental challenges: The role of human identity. Godalming: WWF-UK.

Crutzen, P. J., & Stoermer, E. F. (2000). The Anthropocene. Global Change Newsletter, 41, 17–18.

Department of Energy and Climate Change [DECC]. (2014). Community energy strategy: Full report. London: Department of Energy and Climate Change. Diamond, J. (2011). Collapse: How societies choose to fail or survive. London: Penguin Books.

Dryzek, J. S. (2013). The politics of the earth: Environmental discourses (3rd ed.). Oxford: Oxford University Press.

Dumanoski, D. (2009). The end of the long summer: Why we must remake our civilization to survive on a volatile Earth. New York: Three Rivers Press.

EEA. (2012). Climate change, impacts and vulnerability in Europe 2012: An indicator-based report. Copenhagen. https://doi.org/10.2800/66071.

Elgert, L., & Krueger, R. (2012). Modernising sustainable development? Standardisation, evidence and experts in local indicators. Local Environment, 17(5), 561–571. https://doi.org/10.1080/13549839.2012.681466.

Fischer-Kowalski, M., Krausmann, F., & Pallua, I. (2014). A sociometabolic reading of the Anthropocene: Modes of subsistence, population size and human impact on earth. The Anthropocene Review, 1(1), 8–33. https://doi. org/10.1177/2053019613518033

Food and Agriculture Organization of the United Nations [FAO], & Intergovernmental Technical Panel on Soils [ITPS]. (2015). Status of the World’s Soil Resources (SWSR)—Main Report. Rome, Italy. Available at: https://hal.archives-ouvertes.fr/hal-01241064/

Foster, J. (2015). After sustainability: Denial, hope, retrieval. Oxford: Earthscan. Funtowicz, S., & Ravetz, J. (1993). Post-normal science. Futures, 25(7), 739–755. https://doi.org/10.1007/978-3-8350-9053-8_13.

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.

Goldenberg, J. L., Pyszczynski, T., Greenberg, J., Solomon, S., Kluck, B., & Cornwell, R. (2001). I am not an animal: Mortality salience, disgust, and the denial of human creatureliness. Journal of Experimental Psychology: General, 130(3), 427–435.

Gooding, K. (2016). What do we mean by evidence-based advocacy? Ideas from NGOs in Malawi. In R. Hayman, S. King, T. Kontinen, & L. Narayanaswamy (Eds.), Negotiating knowledge: Evidence and experience in development NGOs (pp. 15–28). Rugby: Practical Action Publishing Ltd.

Greenberg, J., Pyszczynski, T., Solomon, S., Rosenblatt, A., Veeder, M., & Kirkland, S. (1990). Evidence for terror management theory II: The effects of mortality salience on reactions to those who threaten or bolster the cultural worldview. Journal of Personality and Social Psychology, 58(2), 308–318.

Haberl, H., Erb, K. H., Krausmann, F., Gaube, V., Bondeau, A., Plutzar, C., et al. (2007). Quantifying and mapping the human appropriation of net primary production in earth’s terrestrial ecosystems. Proceedings of the National Academy of Sciences, 104(31), 12942–12947. https://doi.org/10.1073/pnas.0704243104.

Hamilton, C. (2016). How scared or hopeful should we be in a warming world? The Guardian. Available at: https://www.theguardian.com/environment/2016/may/12/how-scared-or-hopeful-should-we-be-in-a-warmingworld. Accessed 28 May 2016.

Hamilton, J. (2013). Community monitoring and evaluation methods survey: Report and analysis. Evaluating Low Carbon Communities: Working Papers and Reports. Available at: http://media.wix.com/ugd/caf2de_ fd61a89150c88692722b9b0ea329ec98.pdf. Accessed 12 Dec 2014.

Hansen, J., Nazarenko, L., Ruedy, R., Sato, M., Willis, J., Genio, A. D., et al. (2005). Earth’s energy imbalance: Confrmation and implications. Science, 308, 1431–1435. https://doi.org/10.1126/science.1110252.

Hansen, J., Sato, M., Hearty, P., Ruedy, R., Kelley, M., Masson-Delmotte, V., et al. (2015). Ice melt, sea level rise and superstorms: Evidence from paleoclimate data, climate modeling, and modern observations that 2°C global warming is highly dangerous. Atmospheric Chemistry and Physics Discussion, 15, 20059–20179. https://doi.org/10.5194/acpd-15-20059-2015

Hansen, J., Sato, M., Kharecha, P., Russell, G., Lea, D. W., & Siddall, M. (2007). Climate change and trace gases. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 365(May), 1925–1954. https://doi.org/10.1098/rsta.2007.2052

Hansen, J., Sato, M., Ruedy, R., Schmidt, G., & Lo, K. (2016). Global temperature in 2015, climate science, awareness and solutions. Available at: http:// csas.ei.columbia.edu/2016/01/19/global-temperature-in-2015/. Accessed 7 June 2016.

Hausfather, Z. (2017). Analysis: Global CO2 emissions set to rise 2% in 2017 after three-year ‘plateau’. Carbon Brief. Available at: https://www.carbonbrief.org/analysis-global-co2-emissions-set-to-rise-2-percent-in-2017-following-three-year-plateau. Accessed 17 Jan 2018.

Hayman, R. (2016). NGOs and the evidence-based policy agenda. In R. Hayman, S. King, T. Kontinen, & L. Narayanaswamy (Eds.), Negotiating knowledge: Evidence and experience in development NGOs (pp. 129–145). Rugby: Practical Action Publishing Ltd.

Hessels, L. K., & van Lente, H. (2008). Re-thinking new knowledge production: A literature review and a research agenda. Research Policy, 37(4), 740–760. https://doi.org/10.1016/j.respol.2008.01.008

H.M. Government. (2008). Climate Change Act. London: Crown Publishers. Available at: http://www.legislation.gov.uk/ukpga/2008/27/pdfs/ukpga_ 20080027_en.pdf.

H.M. Government. (2011). Localism Act 2011 (c.20). UK.

H.M. Treasury. (2011). The Magenta Book: Guidance for evaluation. London.

Hobson, K., Hamilton, J., & Mayne, R. (2014). Monitoring and evaluation in UK low-carbon community groups: Benefts, barriers and the politics of the local. Local Environment, 21(1), 124–136. https://doi.org/10.1080/13549 839.2014.928814.

Hobson, K., Mayne, R., & Hamilton, J. (2016). Monitoring and evaluating eco-localisation: Lessons from UK low carbon community groups. Environment and Planning A, 48(7), 1393–1410. https://doi.org/10.1177 /0308518X16640531.

Hume, G., & Hume, M. (2008). The strategic role of knowledge management in nonproft organisations. International Journal of Nonproft and Voluntary Sector Marketing, 13, 129–140. https://doi.org/10.1002/nvsm.316.

Intergovernmental Panel on Climate Change [IPCC]. (2014). Fifth assessment synthesis report: Climate change 2014 synthesis report. Kaufmann, B. A. (2011). Second-order cybernetics as a tool to understand why pastoralists do what they do. Agricultural Systems, 104(9), 655–665. https:// doi.org/10.1016/j.agsy.2011.07.006

Kern, F., & Smith, A. (2008). Restructuring energy systems for sustainability? Energy transition policy in the Netherlands. Energy Policy, 36(11), 4093–4103. https://doi.org/10.1016/j.enpol.2008.06.018.

Kingsnorth, P., & Hine, D. (2009). Uncivilisation: The dark mountain manifesto. The Dark Mountain Project. Available at: http://dark-mountain.net/ about/manifesto/. Accessed 12 Dec 2015.

Letcher, R., Roberts, S., & Redgrove, Z. (2007). Mobilising individual behavioural change through community initiatives: Lessons for tackling climate change The Energy Review Study [on-line]. https://www.cse.org.uk/downloads/ reports-and-publications/community-energy/behaviour-change/mobilising-individual-behavioural-change-through-community-summary.pdf. Bristol.

Lettieri, E., Borga, F., & Savoldelli, A. (2004). Knowledge management in non-proft organizations. Journal of Knowledge Management, 8(6), 16–30. https://doi.org/10.1108/13673270410567602

Lyons, S. K., Amatangelo, K. L., Behrensmeyer, A. K., Bercovici, A., Blois, J. L., Davis, M., et al. (2015). Holocene shifts in the assembly of plant and animal communities implicate human impacts. Nature. https://doi.org/10.1038/ nature16447.

Maturana, H. R. (1988). Reality: The search for objectivity or the quest for a compelling argument. The Irish Journal of Psychology, 9(1), 25–82.

Merritt, A., & Stubbs, T. (2012). Incentives to promote green citizenship in UK transition towns. Development, 55(1), 96–103. https://doi.org/10.1057/ dev.2011.113

Milligan, S., Coulton, C., York, P., & Register, R. (1998). Implementing a theory of change evaluation in the Cleveland community-building initiative: A case study. In K. Fulbright-Anderson, A. C. Kubisch, & J. P. Connell (Eds.), New approaches to evaluating community initiatives. Volume 2: Theory, measurement, and analysis (pp. 45–85). Washington, DC: Aspen Institute.

Moore, D. T. (2011). Sensemaking: A structure for an intelligence revolution Washington, DC: National Defense Intelligence College.

Nielsen, S. N. (2016). Second order cybernetics and semiotics in ecological systems—Where complexity really begins. Ecological Modelling, 319(10), 119–129. https://doi.org/10.1016/j.ecolmodel.2015.08.006

Nilsson, C., Reidy, C. A., Dynesius, M., & Revenga, C. (2005). Fragmentation and fow regulation of the world’s large river systems. Science, 308(2005), 405–409. https://doi.org/10.1126/science.1107887.

Novacek, M. J., & Cleland, E. E. (2001). The current biodiversity extinction event: Scenarios for mitigation and recovery. Proceedings of the National Academy of Sciences of the United States of America, 98(10), 5466–5470. https://doi.org/10.1073/pnas.091093698.

Nowotny, H., Scott, P., & Gibbons, M. (2001). Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge: Polity Press.

Patton, M. Q. (1996). A world larger than formative and summative. Evaluation Practice, 17(2), 131–144. https://doi.org/10.1177/109821409601700205

Patton, M. Q. (1997). Utilization-focused evaluation (3rd ed.). Thousand Oaks, CA: Sage.

Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: The Guildford Press.

Powell, M. (2006). Which knowledge? Whose reality? An overview of knowledge used in the development sector. Development in Practice, 16(6), 518–532. https://doi.org/10.1080/09614520600957951.

Ramalingam, B. (2013). Aid on the edge of chaos: Rethinking international development in a complex world. Oxford: Oxford University Press.

Ravallion, M. (2008). Evaluating anti-poverty programs. In T. P. Schultz & J. Strauss (Eds.), Handbook of development economics (pp. 3787–3846). Oxford: Elsevier B.V.

Regeer, B. J., Hoes, A.-C., van Amstel-van Saane, M., Caron-Flinterman, F. F., & Bunders, J. F. G. (2009). Six guiding principles for evaluating mode-2 strategies for sustainable development. American Journal of Evaluation, 30(4), 515–537. https://doi.org/10.1177/1098214009344618.

Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169.

Röckstrom, J., Steffen, W., Noone, K., Persson, A., Chapin, F. S., Lambin, E. F., et al. (2009). A safe operating space for humanity. Nature, 461, 472–475. https://doi.org/10.1038/461472a

Rosenblatt, A., Greenberg, J., Solomon, S., Pyszczynski, T., & Lyon, D. (1989). Evidence for terror management theory: I. The effects of mortality salience on reactions to those who violate or uphold cultural values. Journal of Personality and Social Psychology, 57(4), 681–690. Available at: http://www. ncbi.nlm.nih.gov/pubmed/2795438.

Scranton, R. (2015). Learning to die in the Anthropocene: Refections on the end of a civilization. San Francisco: City Lights Books.

Seyfang, G., Hielscher, S., Hargreaves, T., Martiskainen, M., & Smith, A. (2014). Environmental innovation and societal transitions a grassroots sustainable energy niche? Refections on community energy in the UK. Environmental Innovation and Societal Transitions. https://doi. org/10.1016/j.eist.2014.04.004

Seyfang, G., Park, J. J., & Smith, A. (2013). A thousand fowers blooming? An examination of community energy in the UK. Energy Policy, 61, 977–989. https://doi.org/10.1016/j.enpol.2013.06.030.

Shellenberger, M., & Nordhaus, T. (2004). The death of environmentalism: Global warming politics in a post-environmental world. Available at: http:// www.thebreakthrough.org/images/Death_of_Environmentalism.pdf . Accessed 1 May 2008.

Shi, J., Visschers, V. H. M., Siegrist, M., & Arvai, J. (2016, April). Knowledge as a driver of public perceptions about climate change reassessed. Nature Climate Change, AOP. https://doi.org/10.1038/nclimate2997.

Stephenson, J., Barton, B., Carrington, G., Gnoth, D., Lawson, R., & Thorsnes, P. (2010). Energy cultures: A framework for understanding energy behaviours. Energy Policy, 38, 6120–6129. https://doi.org/10.1016/j. enpol.2010.05.069.

Stuffebeam, D. L., & Shinkfeld, A. J. (2007). Evaluation theory, models, and applications. San Francisco, CA: Jossey-Bass. TESS. (2016). Success factors of community-based sustainability initiatives— Results from TESS. Potsdam. Available at: http://www.tess-transition.eu/ wp-content/uploads/2016/10/Tess_draft_Policy-Brief_success_1-1.pdf.

Verburg, P. H., Dearing, J. A., Dyke, J. G., van der Leeuw, S., Seitzinger, S., Steffen, W., et al. (2015). Methods and approaches to modelling the Anthropocene. Global Environmental Change https://doi.org/10.1016/j. gloenvcha.2015.08.007.

Waters, C. N., Syvitski, J. P. M., Ga uszka, A., Hancock, G. J., Zalasiewicz, J., Cearreta, A., et al. (2015). Can nuclear weapons fallout mark the beginning of the Anthropocene Epoch? Bulletin of the Atomic Scientists, 71(3), 46–57. https://doi.org/10.1177/0096340215581357.

Waters, C. N., Zalasiewicz, J., Summerhayes, C., Barnosky, A. D., Poirier, C., Gałuszka, A., et al. (2016). The Anthropocene is functionally and stratigraphically distinct from the Holocene. Science, 351(6269). https://doi. org/10.1126/science.aad2622

Wise, R. M., Fazey, I., Stafford Smith, M., Park, S. E., Eakin, H. C., Archer Van Garderen, E. R. M., et al. (2014). Reconceptualising adaptation to climate change as part of pathways of change and response. Global Environmental Change, 28, 325–336. https://doi.org/10.1016/j.gloenvcha.2013.12.002

Zalasiewicz, J. A. N., Williams, M., Steffen, W., & Crutzen, P. (2010). The new world of the Anthropocene. Environmental Science and Technology, 44(7), 2228–2231. https://doi.org/10.1021/es903118j.

CHAPTER 2

Evaluating Complexity

Abstract The fndings from evaluations of community-based sustainability and international developmental aid initiatives are generally disappointing. Mitchell reviews the literature and concludes that the fndings converge around an instrumental critique, which holds that the monitoring and evaluation frameworks and indicators are inappropriate or inadequate and that project practitioners are ill-equipped to collect, analyse and manage evaluation data, and a design error critique, which suggests that projects deployed to address complex and wicked problems are predicated on a linear design model. Mitchell introduces and discusses the developmental evaluation approach that supports practitioners to draw from their projectbased experience to innovate and respond adaptively to complex operating conditions.

Keywords Wicked problems · Community-based sustainability International developmental aid · Monitoring and evaluation frameworks · Indicators · Developmental evaluation

2.1 introduction

Chapter 1 introduced some of the challenges and constraints encountered by front-line community-based sustainability (CBS) practitioners. As policymakers have come to see the potential role grassroots and community-based sustainability initiatives offer to further governmental policies on climate

© The Author(s) 2019

A. Mitchell, Second-order Learning in Developmental Evaluation, https://doi.org/10.1007/978-3-319-99371-3_2

Another random document with no related content on Scribd:

“It’s on the dark side of the house, night and day the same,” explained Mrs. Weaver. “It’s around on the dark side, where no one would find it in a month of Sundays, just about the length of my foot above the ground. Such a small thing it is, and the light it lets in is that little! Oh lauk, I’m feeling worse to be thinking upon it!”

“Then you mustn’t talk about it any more,” Garde assured her, sympathetically. “And I must be going home. I do hope the simples will make you better, and I’m so glad I came. I must say good night, for I suppose you will all be going to bed very soon.”

“I shall be there directly,” Mrs. Weaver informed her, “but dear me, Blessedness won’t be touching a pillow for an hour, and then he’ll sleep with his stockings on. He always does the first night with new rogues in the house. Good night, dearie, and God bless you for a sweet child.”

Garde went out and walked slowly toward Grandther Donner’s. She had an hour to wear away, for she would not dare to be searching about the jail before the jailer at least retired to his couch.

The time was one of dread and chills. Her teeth chattered, not from any suggestion of cold in the night air, but from the nervous strain of this time of suspense. She had never been so frightened of any action in her life, as she was when at length she crept back to the prison, through the dark, deserted streets, and began to search about to find the tiny window of which Mrs. Weaver had spoken.

There were two dark sides to the building. One was constantly in the shadow of a tavern, which almost abutted against it, while the other was on the northern face of the building, in a narrow street. Garde went first to the northern exposure, for in order to get at the other shaded side, she would have been obliged to climb a low, brick wall.

Scarcely had she more than come to her destination, and begun her feverish search, before she heard the sound of distant footsteps, which rapidly approached. She crouched in a black little niche, in fear, with a violent commotion in her breast which threatened to drop her down in a swoon. Almost stepping on her toes, some pedestrian

passed, leaving the girl so horribly weak that she shut her eyes and leaned against the wall, laboring to get her breath.

Nerved again by the things Mrs. Weaver had told her, she came out of her hiding-place, after several minutes, and feeling the cold rock-wall she passed eagerly along, shaking with her chill and fearing to breathe too loud, in the silence.

She was doomed here to bitter disappointment. The window was not to be found. She searched again and again, unwilling to give it up, but it was not there. She realized that she must climb the brick barrier, and try on the other side of the building.

She found the wall not difficult to surmount, but when she jumped down, on the further side, she struck on a heap of broken crockery, thrown out from the tavern.

She crouched down instantly, for the noise she had made attracted the notice of some one in the public house. A door at the rear of the hostelry was thrown open and a man looked out. He appeared to be looking straight at her and listening.

“Must have been a cat,” he said, to somebody back in the house, and he disappeared and closed the door.

Garde could not have been any more wrought upon than the whole affair had made her already. She could not become calm. She could merely wait for moments of partial relief from overwhelming emotions.

Thus in time she was creeping along again, feeling the dark stone as before and peering vainly and desperately into the shadows which lay so densely upon the whole enclosure. Hastily she traversed the whole length of the wall. She arrived at the far end, ready to sink down and cry in anguish. She had not discovered the window.

Back again she went, choking back hysterical sobs and bruising her delicate hands on the rough rocks, as she played with her fingers along that grim, dark pile. She failed again.

Sitting where she was, in the grass, which was growing rank in the place, she clasped her hands in despair. She would have to give it up. There was some mistake. There was no window.

Yet once more she would try. She could not give it up. The dungeon’s horrors and the terrible character of Edward Randolph made her fear that if the morning came before Adam was free, he would no longer have need for freedom, nor light.

Slowly, this time, and digging at the base of the stone-wall that rose above her, she felt down to the very roots of the grass, for the aperture which represented a window To her unspeakable joy, her fingers suddenly ran into an absolute hole in the solid rock, in a matted growth of roots and grass, which had grown up about it!

She sank down, momentarily overcome with this discovery. It was too much to believe. She felt she was almost dying, so insupportable was the agitation of her heart. But she presently clutched at the grass and tore it away in a mad fever of haste. She dug, with her fingers and her finger nails. She could smell the odor of the bruised grass, and then the wholesome fragrance of earth. She had soon uncovered a small square opening, no larger, as the jailer’s wife had said, than a good-sized hand.

On her knees as she was, she bent her head down to a level with the hole and put her lips close to the opening. She tried to speak, but such a faintness came upon her that she could not utter a sound. She had worked with a tremendous resolution toward this end, and now the flood of thoughts of everything said and done that evening, came upon her and rendered her dumb, with emotion and dread.

Making a great effort she essayed to speak again. Once more she failed. But she waited doggedly, for the power she knew would not desert her in the end. Thus for the third time she mustered all her strength and leaned down to the window

“Adam,” she said, faintly, and then she waited, breathlessly.

There was no response. There was not a sound from that tomb, the dankness of which she now began to detect in her nostrils.

“Adam!” she repeated, this time more strongly.

Some subterranean rustling then came to her ears.

“Adam! Oh, Adam!” she said, in a voice that trembled uncontrollably

“Who’s that? Who’s speaking? Is it you, John Rosella?” came in a rumble from the dungeon.

She failed to recognize his voice, so altered did the passage from his place of imprisonment make it.

“Oh, is that you, Adam—Mr. Rust?” she asked, trembling violently.

“Garde!” he said, joyously. “Garde! Oh, my darling! Yes, it’s I. Where are you? What have you done?”

Garde felt her strength leave her treacherously. Thus to hear the endearing names leap upward to her from that terrible place was too much to bear, after all she had learned.

“Here—here are the keys,” she whispered down to him, haltingly. “And your friends—your two companions—they are also in the prison. I hope—I hope you can find your way out. I am dropping them down—the keys. Here they come.” She tossed the bunch, which she had taken from her pocket with nerveless fingers, and now she heard the metallic clink, as they struck the floor, come faintly up through the aperture.

Adam was starting to say something. She dared not wait to listen. Now that her task was done, she knew she would absolutely collapse, if she did not at once bestir herself to flee.

“I mustn’t stop!” she said to him, a little wildly. “Be careful. Goodby,” and without even waiting to hear him answer, she arose, thrust a bunch of grass back into place over the opening, and hastened away.

CHAPTER XXXVII.

RATS IN THE ARMORY.

A’ disappointment, when he got no more responses to the eager questions and blessings he breathed upward to his unseen sweetheart, was keener than all the anguish he had felt at being so foully imprisoned. He had caught up the keys, quickly enough, but when he failed to catch any more of her trembling words he felt more deserted and surrounded by the blackness than he had been in all this new experience. However, his heart was soon tripping with gladness.

At least it was Garde who had come to save him. Love was his guardian angel. He could face the world full of foes, after this. He grew impatient, abruptly, to get out of the dungeon at once and go to Garde—his brave, darling Garde!

Then he thought of the beef-eaters. He had fancied he heard their voices, as Randolph’s men had been taking him into the prison corridor. It had seemed impossible that they had already arrived and been apprehended till he remembered how many days it had been since last he had seen them.

Having been asleep when Garde first called down to him, through the tiny air-passage, the rover was a little refreshed. But he was still nearly famished for something to eat, having been provided only with a dry chunk of bread, as large as his fist, and a jug of water. He was

also quite lame, for he had not been able to do anything for his wounded foot.

Nevertheless he was alert, now, for his slumber of an hour had been profoundly deep and his constitution was one of great elasticity, rapidly responding to the most inconsiderable restorative influence. He hobbled about in his small den, finding the door without difficulty, after which he tried the lock with key after key, on the bunch, until he thought he had rejected all, when his high hopes came swiftly tumbling down.

The key to the dungeon had not been found among the lot on the ring!

In his weakened condition this apparent discovery was prostrating in its dire effect. He suffered more than he would have done had there been no attempt made to free him at all. He felt cold beads of perspiration break out on his brow. Hope for himself and the beefeaters, snatched away almost as soon as given, unnerved him. Nevertheless he pulled himself together, to try every key in the bunch again.

The first one he handled entered the lock and threw back the bolt.

Cautiously swinging the door open, he suddenly started, at the sound of some one approaching in the corridor. In a second he was back in the dark hole and had locked the door again upon himself. Weaver, the jailer, making an unusual round of the premises, came down the dungeon-steps and tried the door. Satisfied that all was well, he proceeded onward to his bed.

Adam lost little time in again starting forth. This time he locked the dungeon and took his bunch of keys with him. He climbed the nine steps, which the jailer’s wife had so frequently counted, and found himself in the corridor, which was lighted by a single lamp, which was small and odorous. Noting his bearings, he limped along toward the cell where he thought he had heard the beef-eaters talking.

There was no sound to give him guidance now, and there were several doors confronting him, behind any one of which his retinue might be locked. It was a matter presenting necessities for nicety in

judgment. If he were to open the door on some wrong prisoner, the ensuing disturbance would be most unfortunate. Moreover, he did not know but what there might be guards galore in some of the jailapartments. It would not do to call, or to whisper, for the sake of attracting the beef-eaters’ attention, for obvious reasons.

There was nothing for it but to open door after door till he found the faithful pair. Luckily the doors were numbered, and he found there were corresponding numbers on the keys. There being no choice, he unlocked the first door he saw. Shifting the bolt cautiously, he was presently able to listen for anything like a sound inside the cell.

He could hear nothing. The room was empty. To the next door he went, and repeated his simple experiment. This apartment proved to be, not a cell, but a place in which all manner of rubbish had been thrown. It also contained swords, pistols, some blunderbusses and other arms. The room, indeed, was the prison armory. Adam nodded at this discovery as being good, but it left him as far as before from his friends. Leaving this door unlocked, he went back in the other direction and tried again.

Listening now, as before, upon opening a second cell, he heard snoring. Better than this, it was snoring that he knew He went in and nudged the retinue with his foot.

“What, ho! Who knocks?” said Halberd, in a sleepy growl.

“Be quiet,” said Adam. “Get up, the two of you, quickly. We are about to seek more commodious apartments.”

“The Sachem!” said Pike.

“Who else,” answered Halberd. “Sire, I have been expecting this kindness these three hours.”

“You may expect to be hanged, in the morning, if you do not shut your mouth and come with me instantly,” said Rust.

“I was dreaming of my wedding with a fair princess,” said Pike. “These are no days of chivalry, when a man will leave so sweet a damsel in so vile a place.”

“What have you done with your swords and side arms?” the Sachem demanded, in a whisper. “Did they take them from you?”

“They did. Else we had slain the whole score of rascals that took us,” said Halberd.

“Make haste, then, till we arm anew,” instructed the rover.

He locked the door behind them and led the way to the armory at once. They had gone half the distance to the place when there came a clanking of opening doors, a rattle of scabbards, a rumble of muffled voices and the tramp of many feet, around in the angle of the corridor, leading to the outside world.

“Quick! Quick!” commanded Adam, and darting forward, lame foot and all, to the armory-door, he opened it, thrust in the beef-eaters, with a word of admonition to beware of making a noise, and closed the barrier, only as Randolph and six of his creatures came tiptoeing down the passage and stopped fairly opposite where Adam was standing.

The rover reached out in the dark of the room they were in, as he braced silently against the door, and felt his hand come in contact with a sword, which he had noted when first he peered into the room. He could hear the men outside, whispering.

Weaver was with them, pale and frightened at what he knew these midnight visitors contemplated doing. He dared not make the slightest protest; his master stood before him.

“Here, is this the room above the dungeon?” said Randolph. He laid his hand on the knob, the inside mate of which Adam was holding.

“No, sir, this is the room, here upon the other side,” said Weaver. “It’s a few steps further along.”

The private executioners, with their chief, were moving away, when one of the beef-eaters stepped upon something on the floor of the armory, making a sound that seemed terrific.

“What was that?” demanded Randolph, quickly.

“We have rats in the property chamber,” said Weaver, honestly

“It sounded too big for rats,” said the voice of Psalms Higgler, whom Adam readily identified.

“We may look there if you like,” said the jailer.

“Never mind the rats at present,” dictated Randolph. “Show us the room above the cellar.”

The other door could then be heard to open and to close behind the visitors. Adam snatched up swords for three on the instant.

“Here, take it—and not a word,” he breathed, thrusting a weapon upon each of his trembling companions. “If they come for us—fight!”

Silently and slowly he reopened the door, having buckled a sword upon him. There came a light patter of footsteps on the corridor floor. Just as the rover was stepping forth, Psalms Higgler, who had not been satisfied with the theory of the rats, came gliding to the spot. He and Adam suddenly faced one another, a foot apart. The startled little monster stared wildly for the briefest part of a second and then would have fallen back, yelling like a demon to raise the alarm.

Pouncing upon him, without a sound, yet with the terrible strength and nimbleness of a tiger, Adam clutched him fiercely by the neck, with both his powerful hands, and choking back the yell already starting to the creature’s lips, lifted him bodily off the floor, to prevent him from kicking upon it, to raise a disturbance, and carried him, squirming and writhing, to the door by which the visitors had so recently entered.

“Open the door! Open the door and get out!” ordered Rust of his followers, sternly, never for a moment relaxing his grip or his lift on Higgler. “Lift the bar! Lift it! There!”

The door swung open. The beef-eaters sprang outside, trying both to go at once. The commotion they made rang through the building. Adam was after them swiftly, forgetting to limp, as he felt the outside air in his face.

Higgler by this was becoming absolutely limp. Adam dropped him on the ground, where he lay, barely left alive and unable to move or

to speak.

Adam had the keys in his pocket, the largest one uppermost. This was the one to this outside door He could hear the men inside running toward the spot and already shouting the alarm. He dared to lock the door, deliberately, and to pull out the key and put it again in his pocket. Then he calmly drew the borrowed sword from its scabbard, rammed its end smartly home, in the key-hole and snapped it off short, spiking the aperture completely.

Already the beef-eaters were running up the street. Psalms Higgler was drawing his breath in awful gasps, where he lay

“Good friend, farewell,” said Rust to him, cheerfully. “I shall be pleased to report you an excellent rat-catcher, at the earliest opportunity afforded.”

He disappeared from Higgler’s ken in a twinkling and soon overtook his retinue, making good time for the country.

CHAPTER XXXVIII.

LOVE’S LONG GOOD-BY.

A that his ruse in locking the jail upon his jailers would hold them only till they could think of taking off the lock and knocking out the sword-end, Adam was nevertheless determined upon going to David Donner’s residence, for the purpose of seeing Mistress Garde.

With this purpose in view, and expecting his pursuers to be soon on a keen race for the open flats, which he had been known to cross before, in his successful escape to the woods, he led his retinue straight off at right angles from such a course, and brought them in fifteen minutes to the silent ship-yard of William Phipps.

Here, with small ado, they climbed the fence and struck across the enclosure, past the gaunt skeleton of a ship, growing on the ways, and so came to a quiet bit of water, at the private landing, where three small boats were moored in safety

The trio were soon aboard the lightest skiff and rowing her westward, with silent, effectual strokes. Guided by the rover, the beef-eaters steered for the shore, and after a ten-minute pull Adam landed near the spot where he had sat upon a rock, waiting for night, on the occasion of his last meeting with his sweetheart.

“Wait for me here,” he said. “I shall not be long.”

He was soon at the gate and then in the garden. There was not a sound to be heard. The house was dark. He raised a little whistle, as

he slowly walked about the place, watching the windows intently

Garde heard him. She was up. She had not had a moment of peace or freedom from dreadful suspense since arriving at the house, while waiting, listening, starting at all those uncanny sounds of stretching, in which a building will indulge itself at night. Greater unhappiness or despair she had never known, nor greater worry, fearing that Adam would come, and then fearing more that he would not.

When she heard him whistle, her heart seemed suddenly dislodged in her bosom. Her breath came laboredly She opened the window in the kitchen, this room being furthest from her grandfather’s apartment, and saw Adam limp eagerly toward her.

“Garde!—Sweetheart!” he said.

“Oh—oh, you—you got away,” she faltered, faintly “Here, I have— tied you up—a luncheon. Take it, please, and—and you had better go—at once.”

“God bless you!” said Adam, stuffing the parcel she gave him inside his coat. “I have brought you back the keys. My Garde! My own blessed sweetheart. Oh, Garde, dearest, come out to me, just for a moment—just for one little good-by.”

“I—I cannot,” Garde said, fighting heroically against the greatest temptation she had ever known. “We must say——good-by, now, and I must——”

“Yes, I know, dear,” he broke in impetuously, “but just for a moment, just——”

He was at the window. He tried to take her hands, to draw her toward him. She shrank away with an action so strange that his sentence died on his lips. “Why, Garde,” he said, “can’t I even touch your hands?”

She shook her head. He could barely see her, in the pale light which the stars diffused.

“I—I must never see—never see you—again,” she stammered, painfully, “we must say—say good-by.”

“You must never——Garde—why—we must say—But, Garde, dear,—I don’t understand you. What does all this mean?”

“Oh, please go—now,” she said. “That is all—all I can say It must be good-by.”

Adam was made dumb for a moment. He stared at her unbelievingly. He passed his hand across his brow, as if he feared his fasting and long-endured labors had weakened his mind.

“What in heaven’s name has happened?” he said, as if partially to himself. “Am I Adam Rust? Are you Garde? Say good-by?—— Dearest, has anything happened?”

She nodded to him, forcing back the sob that arose in her throat. “Something—something has happened,” she repeated. For maidenly shame she could not broach the subject of the Indian child.

He was silent for a moment before replying.

“But you came to-night and gave me the keys, an hour or so ago,” he said, in wonderment and confusion. “You did that?”

“I—couldn’t—do less,” she answered, mastering her love and anguish by a mighty resolution.

“Do you mean—you would have done the same for anybody?” he asked. And seeing her nod an affirmative he gave a little laugh. “I am crazy now, or I have been crazy before,” he told himself. “Something has happened. Something—Of course—it couldn’t help happening, in time. Some one has told you——I might have known it would happen.... And yet—you once said you could wait for me fifty years. And I believed it.... Well, I thank you. I have been amused.”

His broken sentences seemed to Garde to fill in the possible gaps of the story—to make his confession complete. But Adam had, in reality, stopped himself on the verge of accusing her of listening to the love-making of some one other than himself, in his absence.

She made no reply to what he had said. She felt there was absolutely nothing she could say. Her heart would have cried out to him wildly. When he spoke so lightly of the fifty years which she could have waited, she swayed where she stood, ready to drop.

Almost one atom more of impulse and she would have thrown herself in his arms, crying out her love passionately, in defiance of the story of his perfidy. But her honor, her maidenly resolution, steeled her in the nick of time. Though her heart should break, she could not accept the gilded offer of such a love.

“Oh, Garde—sweetheart, forgive me,” said Adam, after a moment of terrible silence. “I have wronged you. Forgive me and tell me it is all some nightmare—some dreadful——”

The night stillness was broken by the sound of men running swiftly up the street. Randolph had thought of the possibility of Adam’s visit to Mistress Merrill.

Garde heard and comprehended. Rust heard and was careless.

“Oh, go, Ad—Mr. Rust, please go at once,” pleaded the girl already closing down the window

“Garde! Garde!—not forever?” cried the man in a last despair.

“Forever,” she answered, so faintly that he barely heard, and then the window came down to its place.

Limping back into the shadow, at the rear of the garden, Adam lay out full length on the ground, as two tiptoeing figures entered the gate and came sneaking silently about the somber house. He saw them make a circuit of the garden. One of them walked to within a rod of where he lay—therefore within a rod of death,—and then turned uncertainly away and retired from the place with his fellowhound.

The rover heard them go on up the street, hurriedly making toward the woods. He came back to the place by the window, at last, and whistled softly once again, unable to believe that what he had heard could be so. There must be some explanation, if only he could get it.

There was no response, partially for the reason that Garde had sunk down upon the floor, on the other side of the window, in a dead faint.

His lameness fully upon him again, Adam hobbled a few steps away, halted to look back, yearningly, and then once more dragged

himself off, to join the faithful beef-eaters, waiting in patience with the boat.

CHAPTER XXXIX. MUTATIONS.

W Garde, heart-broken, pale and ill, was restoring her uncle’s keys to their accustomed hook, in the morning, Adam and his retinue were taking a much-needed sleep in the woods.

Having recovered his own good sword and his leather jerkin, from the place where he had concealed them, on the evening of his capture, he had led the beef-eaters into a maze of trees where no one in Boston could have found them, and here he was doing his best to prove himself a cheerful and worthy companion, to share their natural distresses.

Refusing at first to eat of the luncheon provided by Garde, the rover finally yielded to the importunities of his companions, and thereby got much-needed refreshment. By noon they were far on their way toward New Amsterdam, their only safe destination. They kept close to the edge of the woods, as they went, remaining thereby in touch with the farms, on which they depended, in their penniless condition, for something to eat.

By sheer perversity, Adam wore away his lameness. He bathed his foot often and he also wrapped it in leaves, the beneficent qualities of which he had learned from the Indians, years before, and this did as much, or more, than his doggedness to make repairs in the injured tendons.

They were many days on this wearisome march which contrasted, for Adam, so harshly with that other stroll, to Boston, from Plymouth. On many occasions they went hungry for a day and a night together. But what with cheer and good water, they lost nothing of their health.

With boots beginning to gape at the toes, and with raiment dusty and faded, they arrived, at last, at the modest house, at the corner of Cedar and William streets, in New Amsterdam, where Captain William Kidd resided with his wife. Here they were made welcome. On behalf of himself and his comrades, Adam presently secured a working passage to Hispaniola, where he meant to rejoin William Phipps, in the search for the sunken treasure. He could think of nothing else to do, and he had no longer the slightest desire to remain on American soil.

Prior to sailing, however, he wrote a long, detailed account of his finding the man and his Indian child, with all the incidents related thereto, which he forwarded straight to Henry Wainsworth. This concluded his duties. He only regretted, he said in his letter to Henry, that he could not apprise him of what disposition had been made of the body of the little man, Henry’s nephew, when the minions of Randolph took it in their charge.

This letter came duly into Henry Wainsworth’s possession. Having been aware, as no other man in Massachusetts was, that his refugee brother was living his isolated life in the woods, Henry was much overcome by this sad intelligence. He made what cautious inquiries he dared, with the purpose of ascertaining what had become of the little body. He then made a pilgrimage into the woods, stood above the grave which Adam had made, and then, taking a few worthless trinkets, as mementoes, from the deserted cabin, he came sadly away.

But not Henry’s sadness, nor yet that of Garde, served to do more than to signalize the sense of affliction which the citizens of the colony felt had come upon them. They had been a joyless people, with their minds and their bodies dressed in the somber hues suggested by a morbid condition of religious meditation, but at least they had enjoyed the freedom for which they had come so far and

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.