Chapter Nine: Our contribution, our conclusions and main reservations, and a research agenda
Our conclusions on rates of return and our main reservations All our work emphasises to us that our ‘bottom-line’ estimates of the rates of return need to be treated with extreme caution. Nevertheless, our ‘bottom-line’ best estimate suggests that for CVD the IRR from the value of net health gains alone (ignoring GDP impacts) is around 9%. Most oneway sensitivity analyses place the IRR within the range of 6–14%, but in the ‘pessimistic scenario’ the value of the net benefit was less than the cost of the investment whilst in the ‘optimistic scenario’ the IRR was over 25%. For mental health, our best estimate of the IRR from net health gains was around 7%. Most one-way sensitivity analyses place the IRR within the range from a situation where the investment exceeded the net benefits to a positive rate of return of just over 11%. In the ‘pessimistic scenario’ again the research investment exceeded the net benefits, but in the ‘optimistic scenario’ we estimated a return of around 16%. These rates of return are very sensitive to the lags between the research and the health gain. The relationship in these two particular cases is made more complex by the data limitations and artefacts of the differing time periods we were able to consider with different lag structures. However it is clear that in reality, other things being equal, the sooner the benefits can be realised the greater their value relative to the research investment. From a policy point of view this is crucial: a ‘guaranteed’ way to improve the return is to speed up each or any step in the various stages from basic research to the health gains that it can lead to, and so to reduce the time lag. It is inevitable that these rates will be compared with the ‘exceptional returns’ that have featured in the international literature. As previously stated, the ‘exceptional returns’ for the USA were largely implicit rather than formally estimated as rates of return on a specific past investment, so direct comparisons are not possible (Funding First, 2000; Murphy and Topel, 2003). However, the two analyses from Australia are much more explicit (Access Economics, 2003, 2008). However, not only are the Access Economics methods of estimation different in the two studies and different again from our own, but crucially their method of measuring the return is peculiar, in that they express it as a ratio of the value of health benefits to cost (B/C ratio) or as the ratio of health benefits minus research costs to research costs (so called return on investment). We principally estimate the more meaningful internal rate of return: see Chapter 7 for a fuller explanation of this measure. Using IRR as our standard measure enables us to combine the value of the health gain and GDP returns and it can also be compared, if desired, with returns to other public and private investments.
52
Therefore, the first point to make is that our estimate of an IRR of 9% for CVD cannot in any way be compared with the estimate of an ‘ROI’ of 788% for CVD in the 2003 Australian study or of 117% for health R&D overall in the 2008 Australian study. If we take our own (discounted) base-case estimate of CVD benefits and of CVD research investment, the ROI, broadly as measured in the Australian study, would be 484%. But we do not accept that that is a meaningful figure. So what are the key substantive differences in our estimates? The key differences between our methods and those of the Australian studies are: • The lag structure: the 2003 Australian study compares expenditure and health gain in the same year; the 2008 study uses a 40-year lag. Our CVD best estimate uses 17 years; our mental health best estimate uses 12 years. • Neither Australian study appears to net off the health service costs involved in achieving the health gain from improved services. • Both Australian studies use high values for a life year: the first, based on an ‘eclectic’ calculation underpinned by the US estimates of individuals’ ‘willingness to pay’ for a statistical life used AUS$150,000 per life year gained, and the second, based on a meta-analysis of studies, used AUS$267,000. We valued a QALY at £25,000 (approximately one-fifth of the most recent Australian value), a value that reflects the opportunity cost of the spending on R&D if it were instead to be spent directly on health care. • Both Australian studies use a ‘top-down’ approach, starting with overall or disease-specific estimates of national changes in mortality and morbidity over time, and attributed 50% of these changes to health research. Moreover, to cope with the 40-year lag, these were projections of what morbidity/mortality might be in 2033 and later. Our study is ‘bottom-up’, aggregating health gains from observed changes in the use of specific evidence-based technologies. We believe that our approach is both more logical and more firmly based in the reality of UK clinical practice and health policy. Nevertheless, we stress that the results should be treated cautiously. Our reservations and cautions relate both to what we have done and to what we have not done: 1 In terms of our analysis, our estimates of the investment and of the health gain both involve a series of assumptions and estimates, about which we have been explicit. Further research and data collation, more systematic reviewing of the evidence, and greater input from clinical specialists could no doubt improve some of these assumptions, and refine these estimates. But it is not self-evident from the lack of
Medical Research: What’s it worth? Estimating the economic benefits from medical research in the UK