82
5. Closing remarks The welfarist methodology provides a useful and coherent framework for evaluating culture. It is the dominant framework used by OECD governments and has benefitted from a rich history of debate and research. We have only touched on a handful of topics here – there are ongoing debates on themes as varied as aggregation of individual welfare to discount rates that have helped develop CBA and the welfarist thinking behind it. There is therefore a wide range of methodologies that is available to those wishing to evaluate culture in a robust way.
Section three – Conclusion In short, we need a different purpose, framework and forum for our evaluation techniques to make us fit for the future. This is going to demand significant cultural change across funders, commissioners and the cultural sector, so that an emphasis on co-production, causality, and societal welfare allows access to a richer account of practice and value. For example, in an accountability environment dominated by the need for return on investment, funders and commissioners of the cultural sector must be wary of promoting ‘compliance monitoring’ – which is high on measurement and weak on causality, value and learning. As a review of third sector evaluation notes: Compliance reporting focuses on and values success while information for learning requires a more open enquiry, looks for information about failure as well as success, and tries to understand what has caused or prevented change. Informants recognised that the need to demonstrate best use of public money also acted as a disincentive to researching difficulties, failure or negative effects.130 At the same time, cultural organisations must rise to the challenge, committing to learning new skills and ways of working. The sector is rightly determined that collecting evidence must not get in the way of the creative work it does, but involving practitioners and the public in empowered planning, delivery and review will make evaluation a natural and creative part of the work, integral to what we do. We’ve made the case that we need to build robust shared approaches to better assess the wider impacts of the cultural sector; improve the effectiveness of interventions, and in turn increase social productivity and societal welfare. Our ROCI framework is one possible route forward, but there will be others. What matters most is that we all take more seriously the need to build a shared framework. In a difficult funding and investment climate, it is hard to make the case for core costs to be devoted to evaluation, let alone to build this network and the new skills required. A centrally supported learning forum is long overdue. Both Arts Council England and DCMS need to more actively support the cultural sector’s efforts in these respects, to quickly raise the capacity of the sector – in terms of technical skills in data collection and analysis, and to share approaches to evaluation and learning. In the language of ‘grand partnership’, Arts Council England can be a broker; bringing together the best of what we do, and maybe the best of what we don’t do – from the private sector and elsewhere – to help democratise access to relevant knowledge and skills.
130 Ellis, J (2009) ‘Monitoring and evaluation in the third sector: meeting accountability and learning needs’ Charities Evaluation Services