Hong Kong Journal of Emergency Medicine
Application of APACHE II in the assessment, classification of severity and predictive ability of Chinese patients presenting to an emergency department resuscitation room CK Lee and TH Rainer
Objectives: To evaluate the use of APACHE II (Acute Physiology and Chronic Health Status Evaluation) in Chinese patients managed in the resuscitation room of an Accident & Emergency Department of one of the teaching hospitals in Hong Kong. Design: Prospective study on Chinese patients. Setting: Resuscitation Room in an Accident & Emergency Department of a university hospital in Hong Kong. Patients and measurements: Consecutive patients managed in the resuscitation room between 14th August, 2000 and 20th August 2000 (excluding patients younger than 18 years old and patients who were not admitted to hospital after initial assessment and treatment). For each patient, demographic data, diagnosis, the chronic health points, and the worst physiological parameters in the A&E resuscitation room were recorded. The total APACHE II scores and the probability of death were calculated. The accuracy of APACHE II for predicting group mortality was assessed by receiver operating characteristic curve analysis and linear regression analysis. Results: Of the 88 patients included in the study, 13 (15%) died and 75 (85%) survived. Significant factors associated with mortality included age, mean arterial pressure, heart rate, respiratory rate, arterial pH, serum sodium, Glasgow coma score, and chronic health points. For the three scoring subdivisions of APACHE II â€“ total APS score, age points and chronic health points â€“ higher mean values were found in those patients who died compared with patients who survived. Using logistic regression analysis, the APACHE II score determined in the emergency resuscitation room is a strong predictor of mortality (r 2=0.712). At a cut off score of >28 the sensitivity is 100.0% (95% CI 100.0-100.0), specificity is 68.0% (95% CI 56.278.3), positive likelihood ratio is 3.13, positive predictive value is 35.1 and negative likelihood ratio is 100.0. Analysis of the ROC curve reveals an area under the curve of 0.910 (95% CI 0.829-0.960). In patients not admitted to ICU, there was a positive correlation between APACHE score and length of hospital stay in patient who survived (r=0.320, P=0.0075) and a negative correlation between APACHE score and length of hospital stay in patients who died (r=-0.760, P=0.0225). Conclusions: The APACHE II scoring system may be usefully applied in emergency departments for predicting mortality, for classifying and assessing severity of disease, for evaluating performance and for planning departmental resource allocation. (Hong Kong j.emerg.med. 2002;9:188-194) Keywords: APACHE, critical care, mortality, severity of illness index, survival
Correspondence to: Lee Ching Kong, FRCS Prince of Wales Hospital, Accident and Emergency Department, 30-32 Ngan Shing Street, Shatin, N.T., Hong Kong Email: email@example.com Prince of Wales Hospital, Accident and Emergency Medicine Academic Unit, The Chinese University of Hong Kong, 30-32 Ngan Shing Street, Shatin, N.T., Hong Kong Timothy H Rainer, MRCP, MD
Introduction In the last quarter of the twentieth century, advances in medical expertise and technologies in critical care medicine have outpaced the development of good judgement for decision-making and prediction of patient outcome for critically ill patients. The pursuit for accurate and practical scoring systems that might
Lee et al./APACHE II and critically ill emergency patients
be used to standardize severity of illness and accurately predict survival has been one focus of critical illness researchers in the 1990s, particularly from the perspective of intensivists.1 Patient outcome prediction was useful in prognosis, decision making for treatment withdrawal, 2,3 costbenefit analysis, comparison between different centres,4,5 monitoring and assessment of new therapies, and population sample comparison in studies in the past decades. Severity scoring systems are needed to assess quality of care, treatment efficacy, and may facilitate auditing and optimization of departmental resource utilization, with the aim of reducing health care cost, providing better care, and improving outcomes. Various prognostic scoring systems have been proposed,6 including for example the Apgar score, 7 which is a simple scoring system and one of the first objective measures of illness severity developed in the early 1950's by Virginia Apgar to evaluate neonates. It has long been known that the outcome of critically ill patients rely to a great extent on their physiological response to the stress or disease process. With regards to the field of emergency medicine, the development of various scoring systems for the prediction of prognosis in patients with multiple trauma before the millennium has brought about great improvement in the acute trauma care. Nevertheless, these combined anatomical and physiological scoring systems may not be applicable to critically ill non-trauma patients, which are currently using up most of the resuscitative efforts of emergency physicians. It seems that one of the tasks for emergency physicians in the new millennium will be to develop similar scoring systems for more accurate outcome prediction and evaluation of critically ill emergency patients. The Acute Physiology and Chronic Health Status Evaluation (APACHE), devised by Knaus et al, was developed at George Washington University in 1981.8 It is a method of indexing the severity of disease and predicting mortality that has been widely used by intensive care units for measuring illness severity in groups of critically ill patients.
The purpose of this study was to determine whether data collected for APACHE II scoring system from patients in an emergency department might be used to predict mortality. If so, then such data analysis may be useful for classifying severity of disease, monitoring hospital performance and planning for resource allocation.
Materials and methods This study was conducted in the Accident & Emergency Department of Prince of Wales Hospital, the teaching hospital of The Chinese University of Hong Kong. This 1400-bed hospital serves a catchment area with a population of about 1.2 million and currently has a daily attendance of 550 new patients. The required data were collected prospectively by a research nurse on all patients managed in the resuscitation room from 14th August 2000 to 20th August 2000. (Appendix 1) For each patient, demographic data, diagnosis, the worst APACHE II score in A&E resuscitation room, and survival at hospital discharge were recorded. Patients younger than 18 years old and patients n o t a d m i t t e d a f t e r i n i t i a l m a n a g e m e n t we r e excluded. The APACHE II 9 classification system is a revised, simplified and improved version of the prototype APACHE I system. The database was developed from multi-centre study with a total patient number of 5,815. It uses a point score based upon initial values of 12 readily available routine physiological variables, (Table 1) age, and pre-admission health status to provide a measure of severity of disease. Variable selection and weighting was based on 'expert' physician determination. The number of physiological variables was reduced from 34 to 12 and a higher score was assigned to acute renal failure and coma. It has been validated in several countries and has proved to be highly reproducible.10-17 Increased scores correlated with hospital mortality (specificity >98% and sensitivity <30%). The overall correct prediction was about 80%.
Hong Kong j. emerg. med.
Table 1. Number of patients by diagnostic category and observed deaths (N=88).* Patients survived (N=75) Medical (N=74) Respiratory insufficiency from: Asthma 1 (1) Chronic obstructive airway disease 4 (5) Pulmonary embolus 1 (1) Infection 5 (7) Pneumothorax 1 (1) Cardiovascular insufficiency from: Rhythm disturbance 8 (11) Heart failure 8 (11) Coronary heart disease 10 (13) Post cardiac arrest 0 Sepsis 2 (3) Neurological: Seizure disorder 1 (1) CVA 1 (1) ICH/SDH/SAH 3 (4) Metabolic/Renal: 5 (7) Rheumatologic 1 (1) Others: Oncological 3 (4) Drug overdose 4 (5) Drug withdrawal 1 (1) Gastrointestinal bleeding 2 (3) Anaphylaxis 2 (3) Surgical (N=14) Trauma Multiple trauma 2 (3) Head trauma 2 (3) Orthopedic 3 (4) Chest trauma 1 (1) Burn 2 (3) Gastrointestinal Obstruction/perforation 0 Sepsis 1 (1) Vascular 0 Obstetric & gynaecologic 1 (1)
Vol. 9(4) Â„ Oct 2002
Patients died (N=13)
4 (31) 1 (7)
1 (7) 3 (21)
1 (100) 1 (100)
The accuracy of outcome prediction by APACHE II was assessed by the following methodologies: a) Logistic regression; and b) Receiver operating curve. A receiver operating characteristic cur ve was constructed from the observed and predicted outcomes of the patients. The true positive rate was plotted against the false positive rate and the area under the curve was measured. The area under the curve is a measure of the overall discriminatory power of the prognostic variable (A value of 0.5 equals random prediction and a value of 1.0 indicates perfect discrimination).
Results During the study period, a total of 99 patients (2.37% of the total weekly attendance of 3,720 patients) were managed in the resuscitation room. Eleven patients were excluded including seven children, three that took discharge against medical advice, and one who was transferred to another hospital for further management. As a result, a total of 88 Chinese patients were enrolled in the study. Patient characteristics and univariate analysis for predicting mortality are shown in Table 2. Thirteen (15%) patients died and significant factors associated
Lee et al./APACHE II and critically ill emergency patients
Table 2. Patient characteristics and univariate analysis for predicting mortality.* Characteristic Age (years) Mean Range Male sex (no.) (%) Total APS score Temperature – °C Mean arterial pressure – mmHg Heart rate – per minute Respiratory rate – per minute PaO 2 Arterial pH Serum sodium – mmol/L Serum potassium – mmol/L Serum creatinine Haematocrit WBC GCS Serum HCO 3 Age points Chronic health points
61±18 18-90 42 (56) 14.8±7.43 0.29±0.86 1.69±1.4 1.59±1.2 0.73±1.08 0.45±1.09 0.23±0.75 0.08±0.4 0.47±0.94 0.37±1.05 0.44±0.81 0.21±0.53 1.23±2.92 0.32±0.81 3.5±2.3 3.2±2.4
73±15 40-89 7 (54) 28.77±8.77 0.77±1.48 2.85±1.73 2.54±1.13 1.77±1.59 0.15±0.56 0.85±1.35 0.39±0.77 0.54±0.66 0.46±0.88 0.31±0.63 0.23±0.6 7.46±5.29 0.62±1.26 4.8±2.2 5.0±0.0
0.03 0.89 <0.001 0.1 0.01 0.01 0.004 0.34 0.02 0.03 0.79 0.78 0.58 0.91 <0.0001 0.27 0.05 0.0099
*Plus-minus values are means±S.D. Percentages do not always sum to 100 because of rounding. APS, acute physiology score WBC, whole blood count GCS, Glasgow Coma Score
with mortality included age, mean arterial pressure, heart rate, respiratory rate, arterial pH, serum sodium, Glasgow coma score, and chronic health points. The three scoring subdivisions of APACHE II – total APS score, age points and chronic health points – revealed higher mean values in those patients who died.
Analysis of the receiver operating characteristic (ROC) curve for predicting mortality using the APACHE score reveals an area under the curve of 0.910 (95% CI 0.829-0.960; Figure 2). In patients not admitted to ICU, there was a positive correlation between APACHE score and length of hospital stay in patient who survived (r=0.320, P=0.0075) and a negative correlation between APACHE score and length of hospital stay in patients who died (r=-0.760, P=0.0225).
There was a clear predominance of medical (n=74, 84%) over surgical cases (n=14, 16%; Table 1). The system category of cardiovascular insufficiency constituted half of the cases (n=43, 50%), with the most frequent diagnostic categories being coronary heart disease (n=10, 11%), heart failure (n=8, 9%), and rhythm disturbance (n=8, 9%).
Using logistic regression analysis, the APACHE II score determined in the emergency resuscitation room is a strong predictor of mortality (r2=0.712). At a cut off point of >28 for predicting mortality (Figure 1), the sensitivity is 100.0% (95% CI 100.0-100.0), specificity is 68.0% (95% CI 56.2-78.3), positive likelihood ratio is 3.13, positive predictive value is 35.1 and negative likelihood ratio is 100.0.
This study shows that data collected at the time that patients are admitted to the resuscitation room of an emergency depar tment may used to derive an APACHE II score, and that this scoring system may be used to predict mortality. The study also revealed useful information on the proportion, classification and severity of conditions of patients admitted into an emergency department in Hong Kong.
Hong Kong j. emerg. med.
Vol. 9(4) Â„ Oct 2002
>28.0 Sens: 100.0 Spec: 68.0
0 Total_Points Mortality=1
Figure 1. Dot plot showing the relationship between APACHE II score and mortality.
Figure 2. ROC curve analysis showing the relationship between different APACHE II score cut off points, sensitivity, specificity and mortality.
Lee et al./APACHE II and critically ill emergency patients
Fifty percent of patients triaged to the resuscitation room suffer from cardiovascular causes, including coronary heart disease, cardiac failure, and rhythm disturbance (most commonly fast atrial fibrillation). Consequently, emergency medicine training should focus on developing in our trainees the knowledge, skills and attitudes necessary for managing of acute cardiac events. Current training programmes emphasise these aspects and should continue to do so. The compulsory attendance of ACLS courses and regular refreshment schedules should be mandatory. Being widely used as a measure of illness severity in critically ill patients, the APACHE II model has been demonstrated by multiple studies to have good discrimination and calibration for predicting or assessing group outcome. The findings of our study closely agree with similar studies 11 conducted within the intensive care unit of the same hospital. The present study is the first to evaluate the potential use of APACHE II in emergency departments as a measure of illness and a predictor of mortality, and also the first series of APACHE II validation performed on Chinese patients outside the setting of an intensive care unit. The results suggest that the APACHE II scoring system, which is frequently used in ICUs, may also be usefully applied in the resuscitati o n room c on t e x t of a n e m e rg e n c y department to assess, classify, audit and possibly predict outcome of critically ill patients. Despite its imperfections, we have demonstrated that APACHE II is remarkably robust when applied to the emergency context as evidenced by the ROC curve analysis. Preliminary results of the evaluation and validation revealed a promising predictive and discriminative power of the model. The ROC curve values in this study are very similar to those found from the study11 performed in the same hospital but using data collected on Chinese patients after they were admitted to ICU. The study makes use of easily and routinely available objective data, which could be utilized in a wide variety of hospital settings, including Accident and
Emergency Departments. It may be a potentially better method of evaluating the quality of care than waiting times or reattendance rates of patients in EDs. Moreover, as the worst scores were recorded in the resuscitation room before initial resuscitation actually takes place, it helps to eliminate the potential underestimation of the mortality if the worst scores were taken in the ICU after a period of aggressive resuscitation in A&E resuscitation room. Limitations of the study include its small sample size, involvement of only a single centre, and limited analysis. In addition, there are pitfalls in applying APACHE II in the study. First, derivation of APACHE II risk of death (a probability) is based on the difficult subjective choice of a single diagnostic category in what is usually a multi-system problem. Second, it is a static model and may not reflect the rapidly changing nature of the pathological processes affecting patients with critical illnesses. With the recruitment of multiple centres, expansion of the sample size and resultant better assessment of the medical and surgical sub-groups, a larger scale study promise a solution to overcome the present limitations and an extrapolation of the model for use in the future. APACHE III 18,19 was introduced in 1991 to expand and improve the prognostic estimates provided by APACHE II. This system, which is only commercially available, comprises an APACHE III score and a series of predictive equations linked to diagnosis and the APACHE III database. It was not chosen for evaluation because the risk-of-death predictive equations are not available for use. In conclusion, we have shown that APACHE II may be usefully applied in the assessment of disease severity and prediction of mortality in emergency medicine in China. Although insufficiently accurate for individual patient assessment, it may be usefully applied to populations to evaluate the overall quality of care. Further studies to be conducted in a larger scale would be required to confirm such a value.
Hong Kong j. emerg. med.
References 1. Cullen DJ, Chernow B. Predicting outcome in critically ill patients. Crit Care Med 1994;22(9):1345-8. 2. Ruark JE, Raffin TA. Initiating and withdrawing life support. N Engl J Med 1988;318(1):25-30. 3. Lee DK, Swinburne AJ, Fedullo AJ, et al. Withdrawing care. Experience in a medical intensive care unit. JAMA 1994;271(17):1358-61. 4. Rapoport J, Teres D, Lemeshow S, et al. A method for assessing the clinical performance and cost-effectiveness of intensive care units: a multicenter inception cohort study. Crit Care Med 1994;22(9):1385-91. 5. Becker RB, Zimmerman JE. ICU scoring systems allow prediction of patient outcomes and comparisons of ICU performance. Crit Care Clin 1996;12(3):503-14. 6. Sirio CA, Knaus WA. Application of prognostic scoring i n a d u l t i n t e n s i v e c a re . Cu r r e n t Op i n i o n i n Anaesthesiology 1990;3:241-4. 7. Apgar V. A proposal for a new method of evaluation of the newborn infant. Anesth Analg 1953;32:260-7. 8. Knaus WA, Zimmerman JE, Wagner DP, et al. APACHE – acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981;9(8):591-7. 9. Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a severity of disease classification system. Crit Care Med 1985;13(10):818-29. 10. Knaus WA, Le Gall JR, Wagner DP, et al. A comparison of intensive care in the USA and France. Lancet 1982; 2(8299): 642-6.
APACHE II Severity of Disease Classification System 2. Triage Category 1 / 2 / 3 / 4 / 5 (Please ring one)
3. APACHE II Classification (choose worst result in ED setting) 4. Emergency Diagnosis ____________ICD-9 CODE ____________ 5. Primary Specialty ICU / Medical / Surgical / O&T / Paediatric / Other 6. Hospital Discharge
Date of 1. discharge or 2. death (Please ring one)
Vol. 9(4) Oct 2002
11. Oh TE, Hutchinson R, Short S, et al. Verification of the Acute Physiology and Chronic Health Evaluation scoring system in a Hong Kong intensive care unit. Crit Care Med 1993;21(5):698-705. 12. Rowan KM, Kerr JH, Major E, et al. Intensive Care Society's APACHE II study in Britain and Ireland – II: Outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. BMJ 1993;307(6910):977-981. 13. Zimmerman JE, Knaus WA, Judson JA, et al. Patient selection for intensive care: A comparison of New Zealand and United States hospitals. Crit Care Med 1998;16:318-26. 14. Sirio CA, Tajimi K, Tase C, et al. An initial comparison of intensive care in Japan and the United States. Crit Care Med 1992;20(9):1207-15. 15. Chen FG, Koh KF, Goh MH. Validation of APACHE II score in a surgical intensive care unit. Singapore Med J 1993;34(4):322-4. 16. Giangiuliani G, Mancini A, Gui D. Validation of a severity of illness score (APACHE) in a surgical intensive unit. Intensive Care Med 1989;15(8):519-22. 17. Wong DT, Crofts SL, Gomes M, et al. Evaluation of predictive ability of APACHE II system and hospital outcome in Canadian intensive care unit patients. Crit Care Med 1995;23(7):1177-83. 18. Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100(6):1619-36. 19. Bastos PG, Knaus WA. APACHE III study: a summary. Intensive Care World 1991;8(1):35-8.
Appendix 1. Data Collection Form
1. Patient Details
__ __ / __ __ / __ __
7. Discharge Diagnosis ________________ ICD-9 CODE ____________ ________________ ICD-9 CODE ____________ ________________ ICD-9 CODE ____________
Published on Sep 19, 2011
CK Lee and TH Rainer, Hong Kong Journal of Emergency Medicine, 2002