Assessing policymakers’ use of academic health research Steve Hanney Health Economics Research Group (HERG) Brunel University stephen.hanney@brunel.ac.uk
Presentation to LSE conference: Research to Policy: Academic Impacts on Government
Institute for Government: 12 March 2012
Long history of research use in policymaking Alternative opening of Jane Austen’s Pride and Prejudice: ‘It is a truth universally acknowledged that a single researcher in possession of a good finding must be in want of a policymaker’ (Hanney, 2005)
Rothschild experiment in government departments in 1970s: formative evaluation in DHSS by Kogan & Henkel (’83;’06) - multimodality: policymakers in different parts of even the same department have different patterns of knowledge use etc; - collaborative approach developed & analysed: importance but difficulty of getting policymakers & researchers to work together on agendas etc; - knowledge brokerage role in getting research used by policy receptors; - importance of evaluative criteria recognising policy relevance.
Ideas informed approaches to assessing research impact on policy that recognise the many dimensions of policymaking (Hanney et al., 2003)
HERG’s Payback Framework • Payback Framework for assessing health research: 2 main elements - categorisation & model (Buxton & Hanney, 1996) • Multidimensional categorisation of benefits or paybacks: - knowledge production - targeting future research, capacity building, & absorption - informing policies & product development (often key to later impacts) - health and health sector benefits, eg better health, health equity etc - broader economic benefits (overlaps and changing definitions) • Logic model of how to assess the benefits: - 7 stages & 2 interfaces (informed by work of Kogan & Henkel) Applied in many countries; in UK also used to analyse how receptor bodies can lead to ‘impartial’ policy impact of Health Technology Assessment programme (Hanney et al, 2007)
The payback framework: model to assess health research impacts Click to edit Master text styles Second level ● Third level ● Fourth level ● Fifth level
Adapted from: Hanney S, Gonzalez-Block M, Buxton M and Kogan M, The Utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Research Policy Systems 2003, 1:2
Impact of research on heart transplants
Impact case study organised using Payback Framework
Interface: policymakers liaised with researchers to create a single economic evaluation based on researchers’ proposals Receptor policy body (DH) organised to know likely findings of final report through interaction/absorption of interim reports Findings relevant for key policy decision; supportive but also provided strong evidence base for specific policy Initial policy to increase programme funding announced on day final report received, subsequently confirmed in decision gradually to expand the concentrated transplant programme: could go on to estimate other impacts
Impact of project on careers of women doctors Impact case study conducted through documentary analysis & interviews with PI (Isobel Allen), officials etc. Considerable impact identified from the project: - knowledge production: major, well reviewed book publications - helped target further research by PI & others - had a substantial impact on policies from official committees set up to consider the findings by DH (known as the ‘Isobel Allen Committee’), BMA etc, including on changes to Retainer Scheme, careers in surgery etc - various changes implemented but many influences on this including the general direction policy was going: working through the various stages of the Payback Framework helps explore complex attribution Framework helped identify factors linked to impact: - important liaison with receptor body (ie DH) in both interfaces - role of a key broker in DH who promoted findings more widely - quality & timing of the research helped ministers use findings
Impact of research on Heartstart Scotland Random selection; framework informed case study documentary analysis & interviews with PI, expert etc. Described using the various stages: eg Topic identification: - manual defibrillators (to give electric shock to restart heart) carried on a few ambulances but required extensive training, little used in UK - Automated external defibrillators (AEDs) required much less training. Heartstart Scotland scheme to introduce them throughout Scotland (first national coverage), PI chaired evaluation committee & in 1990 secured funding to collect and analyse data about results: continued many years. Impacts in all 5 Paybacks categories, including policy: - informed policies of Scottish ambulance service, continued use & AED renewal (example of impact through PI collaborating with users) - informed guidelines and training documents on AEDs at many levels: professional and national bodies; European; international (traditional academic dissemination & an expert as product champion) - almost certainly contributed to decision of some ambulance services in UK and elsewhere to adopt AEDs (an expert as product champion)
References Buxton M, Hanney S (1996) How can payback from health services research be assessed? J Health Serv Res Policy, 1: 35-43. Hanney S, Gonzalez-Block M, Buxton M, Kogan M (2003) The utilisation of health research in policy-making: Concepts, examples and methods of assessment. Health Res Policy Syst, 1:2 [http://www.health-policy-systems.com/content/1/1/2] Hanney S (2005) Personal interaction with researchers or detached synthesis of the evidence: modelling the health policy paradox. Evaluation and Research in Education, 18: 72-82. Hanney S, Buxton M, Green C, Coulson D, Raftery J (2007) An assessment of the impact of the NHS Health Technology Assessment Programme. Health Technol Assess, 11(53). Kogan M, Henkel M (1983) Government and Research: The Rothschild Experiment in a Government Department. London: Heinemann. Kogan M, Henkel M, Hanney S (2006) Government and Research; Thirty Years of Evolution. (Second edition). Dordrecht: Springer.