About the ACHS Clinical Indicator Program

The Australian Council on Healthcare Standards (ACHS) provides the world’s largest dedicated Clinical Indicator (CI) data collection and reporting service. The Clinical Indicator Program (CIP) examines data sourced from a broad range of clinical speciality areas. It includes CIs developed by specialist clinicians and as relevant to inpatient, outpatient, and community health facilities. The program is highly valued by participating healthcare organisations (HCOs) and is developed by Australian and New Zealand clinicians. Clinical Indicators and Healthcare Organisations CIs are designed to indicate potential problems that may need addressing, rather than to provide definitive answers for HCOs. This is achieved by identifying variations within the data results. CIs are used to assess, compare and determine the potential to improve care within an organisation. They are, therefore, a tool to assist in assessing whether or not a standard of patient care is being met and can provide evidence for accreditation. HCOs select those CIs that are relevant to their organisation.

History The ACHS CIP was established in 1989 through the initiative of Dr Brian Collopy, a surgeon and then Chairman of the ACHS Board, who still remains involved in the program today.

The rationale for introducing the program was to provide measures to support the clinical component of the ACHS accreditation standards and to increase the involvement of medical practitioners in quality improvement initiatives within HCOs. At the time of its introduction, medical staff were familiar with the use of measures to assess a patient’s health status; however, there were almost no tools to assess the performance of an HCO when delivering clinical care.

The first set of CIs, the Hospital-Wide Medical CIs, was introduced in 1993 and the program has continued to evolve since its inception nearly three decades ago. The program has expanded by working in collaboration with specialist colleges, societies, and associations, to include a wide range of speciality areas, totalling 21 CI sets by 2020. The CIP is now strongly supported by all clinician groups who make significant contributions to the development and application of the CIs. Clinical Indicators and Accreditation Accreditation with ACHS has always had a focus on quality improvement. The CIP continues to be free for all HCOs that are accredited by ACHS. The program is one of a number of tools that facilitate the review and improvement of HCO performance. While the data are not a focus for accreditation, assessors are able to monitor the HCO’s response to an outlier measure or a deteriorating trend. HCOs and assessors are able to question what was investigated, what was learnt, what action had been, or would be, taken, and finally what was the outcome of those actions.

Supporting Clinical Indicator Program Customers The Performance and Outcomes Service (POS) at ACHS provides email, telephone, webinar and workshop support to its members, including user access, CI collection assistance clarification on the User Manuals and generation of customised reports.

ABOUT THE ACHS CLINICAL INDICATOR PROGRAM

Developed by Clinicians for Clinicians Decisions are made on each CI set by a Working Party selected to provide broad representation. The ACHS Performance and Outcomes Service facilitate the process by providing secretariat support. When developing CIs, ACHS relies on practising clinicians from specialist areas in public and private HCOs. Members of CI Working Parties encompass relevant professions and include personnel from non-metropolitan centres and from a number of different states and territories. The Working Party Chair is selected by the lead college, society or association, which will also oversee and endorse the revised CI User Manual.

Assisting with data analysis and offering support and advice to the Working Parties is the HSRG at the University of Newcastle. Prof Robert Gibberd, who has consulted on the ACHS program for more than 17 years, is supported by Mr Stephen Hancock and a team that has made healthcare data its focus.

Comparisons of Performance The focus when collecting CI data should always be to identify opportunities for improvement. All participating HCOs receive benchmarking reports that compare their performance to that of all other HCOs submitting data for the CI, and to HCOs from their peer group. Peer groupings are determined by the Working Party and the HCO is then able to select the most appropriate stratification for their organisation. Reports are prepared every six months following data submission. In addition, trend reports are developed annually for HCOs submitting regularly, which enable the HCOs to compare their own trended performance against that of the group overall.

By definition, 20% of all contributors of CI data must be in the poorer performing centile. If an HCO has rates in the poorest 20% of rates it is not necessarily an indicator of poor performance, especially when variation between HCO rates is relatively small. In the latter case centile gains will be relatively small. However, being in the poorer performing centile may indicate a greater opportunity for improvement.

As participation in the ACHS program is voluntary, the number of HCOs submitting data for any single CI may be small; therefore the sample may not represent the overall population. Furthermore, participating HCOs are not identified during statistical analysis, which limits comparisons between HCOs. The program’s statisticians believe that, in most specialities, with greater numbers comes greater confidence that the data are representative. For this reason, ACHS reports also include outlier data which notify an HCO that their rate is more than three standard deviations from the mean. In conjunction with the centile data, outlier status provides HCOs with a realistic ‘snapshot’ of their performance against all other reports submitted for a specific CI.

Research in the area of organisational response to CI outcomes has identified the phenomenon of ‘data denial’, where HCOs are sometimes reluctant to accept the implications of CI data and reject the findings rather than investigate their implications, or seek explanations that are not associated with their own performance. Acceptance of the data as both correct and relevant is the first step towards positive action and change.1

It is necessary that clinicians and healthcare executives recognise that a CI result is a marker of change over time, rather than the equivalent of an ‘exam result’ with its designated pass/fail outcome. Although the ACHS CI reports provide data from multiple HCOs, CI data outcomes should not be considered as ‘league tables’.

CIs are so named because they do not provide answers; they ‘indicate’. This means an HCO’s rate can raise questions for further evaluation. A considered analysis of potential reasons for trends over time and/or variation between HCOs can then be used to highlight quality issues or monitor the progress of quality improvement initiatives.

Clinical Indicator User Manuals The ACHS CI User Manuals contain greater information about the CIs. Members can access the User Manuals from the ACHS website. The User Manuals include information such as: the rationale for CI development suggested sources for data collection (including ICD10-AM codes where applicable) desired rates (i.e. whether the organisation should be aiming for a high or low rate) stratification variables data cleaning rules definition of terms numerator and denominator details including inclusion and exclusion criteria evidence-based information about the CI area Accompanying resources to the User Manuals are blank templates to assist HCOs to collect their data and retain details of their collection.

REFERENCE 1. Berwick DM, James B, Coye MJ. Connections between Quality Measurement and Improvement. Medical care. 2003 Jan 1; 41 (Suppl 1): 30-38.

122 AUSTRALASIAN CLINICAL INDICATOR REPORT 2012 - 2019

123