Advancing knowledge Gerard R.M. Molleman, Machteld A. Ploeg, Clemens M.H. Hosman and Louk H. M. Peters
Preffi 2.0- a quality assessment tool Abstract: The findings of many metastudies into the effects of health promotion programmes indicate that there is still much room for improvement in the quality of these programmes. Insights gained from research are rarely applied in practice. Practitioners and policymakers often find it hard to assess the value of the many and sometimes contradictory research findings, partly because the necessary contextual information is usually lacking. Practical considerations force them to respond to specific problems at short notice in the form of programmes that are as effective as possible. Hence, effective health promotion requires not only the dissemination of effective programmes but also insights into principles of effectiveness and the way professionals use these insights. It is against this background that the Netherlands Institute for Health
Promotion and Disease Prevention (NIGZ) has developed and implemented the Preffi instrument. Preffi consists of a set of guidelines with items relevant to the effectiveness of health promotion and prevention projects, reflecting scientific and practical knowledge about effect predictors. This article describes the systematic, seven-step development process of the second version of the instrument, Preffi 2.0, a process in which scientists and practitioners were closely involved throughout. The article also describes the Preffi model and its scoring method. The draft version of Preffi 2.0 was tested for usefulness among 35 experienced practitioners from a range of health promotion institutes. They were asked to use the draft version to assess two project descriptions and to comment on their experiences using Preffi 2.0. They gave the instrument an average overall
mark of 7.7 on a scale of 10, and the large majority of them evaluated the instrument as valuable, complete, clear, well-organised and innovative. The findings of this trial implementation were used to construct the definitive version of Preffi 2.0. To an experienced user, applying Preffi to assess a project takes less than an hour. Preffi is used as a diagnostic quality assurance instrument at various stages of a project, either to critically evaluate one’s own project or to comment on projects proposed by others. Assessing other people’s projects may be difficult if the necessary information is lacking or unclear. A supplementary discussion with the project manager is always required. Users have commented that applying Preffi to a project yields a balanced and useful assessment, as well as a clear overview of points in the project that could be improved.
This manuscript was submitted on June 10, 2004. It received blind peer review and was accepted for publication on September 26, 2005.
What makes health promotion programmes effective? How can projects be designed and implemented so as to maximise the chances of it being effective?
studies into the effects of health promotion programmes have yielded many new insights, their findings are rarely applied in practical health promotion. Practitioners find it hard to assess the value of the many and sometimes contradictory research findings. In addition, researchers often fail to supply details about the contextual conditions (in terms of available time, funding and support) and circumstances (local context, social, cultural and economic influences, timing) in which the reported effects were achieved, whereas such details are required to decide whether and how the intervention needs to be adjusted to allow its successful repetition or implementation in a different situation. Researchers and practitioners often work to different time-scales. Attempts to improve effectiveness through controlled studies require long-term investments and involve long periods between programme development and the provision of feedback on effects. This is an important but extremely slow process. Policy-makers and practitioners, however, often need faster feedback as well, since they are asked to respond to specific problems at short notice, in the form of preventive programmes and interventions that yield the maximum
Keywords • quality • effectiveness • health promotion
Gerard R.M. Molleman, PhD NIGZ Centre for Knowledge and Quality P.O.Box 500 NL-3440 AM Woerden The Netherlands Phone: +31 348 437621 E-mail: gmolleman@nigz.nl Machteld A. Ploeg, MA NIGZ Finance department Woerden, The Netherlands Clemens M.H. Hosman PhD Prevention Research Centre, Dept. of Health Education and Promotion University of Maastricht, and Dept. of Clinical Psychology University of Nijmegen The Netherlands Louk H. M. Peters, MA NIGZ Woerden, The Netherlands
Over the past 20 years, many researchers have tried to answer these questions in studies focusing on meta-analysis of the effectiveness of programmes, and preferably assessing effectiveness by means of randomised controlled trials (RCTs). These studies have found great variation in the extent to which health promotion programmes are evidencebased (Durlak and Welsh, 1997; Kok et al., 1997; Boddy, 1999) . Although significant effects of many programmes have been proven in various countries, the average size of such effects has so far been moderate. And whereas many programmes were found to be effective or moderately effective, many others were poorly effective or even ineffective. In addition, many of the effective programmes have proved to be effective only for part of the target group, only for a limited number of objectives or only in the short or medium term, or to lose some or all of their effectiveness in a different setting. This means that there is much room for improvement in the quality and effectiveness of programmes. Although the various studies and meta-
IUHPE – PROMOTION & EDUCATION VOL. XIII, NO. 1 2006
9
