ComparingPOSSUMscoring.pdf

ORIGINAL ARTICLE COMPARISON OF THE POSSUM, P-POSSUM AND CR-POSSUM SCORING SYSTEMS AS PREDICTORS OF POSTOPERATIVE MORTALITY IN PATIENTS UNDERGOING MAJOR COLORECTAL SURGERY RYASH VATHER,KAMRAN ZARGAR-SHOSHTARI,SAMUEL ADEGBOLA AND ANDREW G. HILL Department of Surgery, South Auckland Clinical School, University of Auckland, Auckland, New Zealand Background: Physiologic and operative severity score for the enumeration of mortality and morbidity (POSSUM), ?Portsmouth?- physiologic and operative severity score for the enumeration of mortality and morbidity (P-POSSUM) and ?Colorectal?-physiologic and operative severity score for the enumeration of mortality and morbidity (Cr-POSSUM) are three related scoring systems, which uses individual patient parameters to predict postoperative mortality. POSSUM overpredicts mortality in low-risk patients and underpredicts mortality in elderly and emergency patients. P-POSSUM was developed to compensate for these weaknesses. Cr- POSSUM was developed specifically for colorectal surgery. We aim to establish which of these scoring systems would be most useful in an Australasian context. Methods: Data were collected for 308 patients and predicted mortality risk values were generated using each of the three systems. The Mann?Whitney U-test was then carried out on the scores for each system. Receiver?operator characteristic curves were designed to determine the relative accuracy of each approach at discriminating between death and survival. Results: All three POSSUM scoring systems showed a statistically significant ability to predict postoperative mortality. Addition- ally, ineach system therewas asignificant difference inthe rawphysiologic andoperativeseverityscores between survivorsandthose whodied.Arisk-stratificationmodelwas appliedto eachset ofdata, showingacorrelationbetween anincreasein riskandanincrease inmortalityrate. Finally, thereceiver?operatorcharacteristic curvesgenerated showed thatinthisstudygroupPOSSUM, P-POSSUM and Cr-POSSUM were all satisfactory predictive tools although the latter tended to be relatively less accurate. Conclusion: Physiologic and operative severity score for the enumeration of mortality and morbidity, P-POSSUM and Cr-POS- SUM are all reliable predictors of postoperative mortality in the Australasian context; although there was a trend towards POSSUM and P-POSSUM being better predictors than Cr-POSSUM. However, Cr-POSSUM requires fewer individual patient parameters to be calculated and is thus easier to generate. An ideal preoperative scoring system remains to be developed for predicting mortality in patients undergoing colorectal surgery. Key words: colorectal, Cr-POSSUM, mortality, POSSUM, P-POSSUM. Abbreviations: CI, confidence interval; Cr-POSSUM, colorectal-physiologic and operative severity score for the enumer- ation of mortality and morbidity; POSSUM, physiologic and operative severity score for the enumeration of mortality and morbidity; P-POSSUM, Portsmouth-physiologic and operative severity score for the enumeration of mortality and morbidity; ROC, receiver?operator characteristic curve. INTRODUCTION Physiologic and operative severity score for the enumeration of mortality and morbidity (POSSUM) is a predictive equation com- monly used to predict outcomes of surgery. 1 It uses several indi- vidual parameters to form a ?physiologic score? and an ?operative severity score?, which is then used to generate a predicted value for mortality and morbidity. It has been shown that POSSUM tends to overestimate the probability of death in low-risk patients and underestimate mortality in elderly and emergency patients. 2?4 Thus, ?Portsmouth? physiologic and operative severity score for the enumeration of mortality and morbidity (P-POSSUM) was developed. 5 P-POSSUM uses the same scoring parameters as POSSUM and has largely replaced POSSUM as a risk predictor. A later study evaluated the use of P-POSSUM as a predictive tool in colorectal surgery, and it was shown that a newer, simpler and dedicated scoring system, ?Colorectal? physiologic and operative severity score for the enumeration of mortality and morbidity (Cr- POSSUM), was more accurate at predicting postoperative mor- tality. 4 It uses fewer variables and has a different scoring system. AIM Given the uncertainty of the performance of these scoring sys- tems, we wished to establish which one would be most useful in Australasia. This study is a retrospective analysis of patient R. Vather BHB; K. Zargar-Shoshtari MBChB; S. Adegbola; A. G. Hill MD, FRACS. Correspondence: Associate Professor Andrew G. Hill, Department of Sur- gery, South Auckland Clinical School, University of Auckland, Middlemore Hospital, PO Box 93311, Otahuhu, Auckland, 2024, New Zealand. Email: ahill@middlemore.co.nz Accepted for publication 7 May 2006. ANZ J. Surg. 2006; 76: 812?816 doi: 10.1111/j.1445-2197.2006.03875.x C211 2006 Royal Australasian College of Surgeons outcomes after major colorectal surgery over a 46-month period. It considers a variety of individual patient parameters. Data collected were used to calculate a POSSUM, P-POSSUM and Cr-POSSUM score for each patient, with a comparison of the functionality of each system as a predictive tool. PATIENTS AND METHODS A database of 536 consecutive patients undergoing a major colo- rectal operation between January 2002 and October 2005 at Mid- dlemore Hospital or Manukau Surgical Centre was created. Complete data (necessary for generation of the predictive scores) were available only for 308 patients. The primary reason for missing data was because of the lack of a standardized method of documenting perioperative information, leading to variability in the consistency and completeness of different operation-record sheets (determined mainly by the overseeing clinician). Also, a small number of files were inaccessible due to patients being deceased. The risk profiles of these discarded patients are unknown but can only be assumed to have had a similar distribu- tion to the analysed patients, as they were not excluded for any reason other than incomplete or unavailable perioperative data. Physiologic and operative severity scores were calculated for each of the 308 patients as follows: (1) Each variable has several predefined brackets, within which raw data may fall (listed in Table 1). (2) Thesebrackets are inturneach given a particular weighting with respect to their relative influence in the development of the score. For example, for the variable ?Glasgow Coma Score?, a score of 15 was given 1, 12?14 given 2, 9?11 given 4 and £8 given 8. It is important to note that Cr-POSSUM varies from the other two systems in that it uses different weightings for similar groupings, hence requiring separate analysis. (3) The weightings of all relevant variables that comprise the physiologic or operative severity score are then added to give two independent scores. 1,4,5 Values for the parameter ?operative blood loss? were only pres- ent in approximately a quarter of the patient files. Recent work has shown that missing data may be treated as normal values (in all three POSSUM scoring systems, the expected value of a nor- mal person, and indeed that which conferred the lowest risk, was given the lowest possible weighting) without influencing the out- come. This allowed the operative severity score to be calculated reasonably accurately. 6 ?Mortality? was defined as death from any cause as a postoper- ative inpatient (the same end-point used in the original POSSUM, P-POSSUM and Cr-POSSUM studies). 1,4,5 Scores for POSSUM, P-POSSUM and Cr-POSSUM were cal- culated by constructing formulas (these were rearranged from the equations for each system as listed in Table 2) on Microsoft Excel (Microsoft Corporation, Redmond, WA, USA). 1,4,5 The Mann?Whitney U-test was carried out on the scores to compare the different sets of continuous variables within each prediction system. Additionally, the risk-stratification model was applied to all three POSSUM scoring systems to identify if a correlation existed between predicted risk and actual mortality. Receiver?operator characteristic (ROC) curves were designed for each of the scoring systems to determine the relative accuracy of each as a predictive tool. The ROC curve has been widely used as a measure of the precision of predictive systems. 7 It determines how good a predicted risk value is at discriminating a bivariable outcome (in this case ?death? or ?no death?) by constructing a graph with sensitivity on the y-axis and specificity on the x-axis. It depicts the inverse correlation between sensitivity and specific- ity (an increase in sensitivity will be followed by a decrease in specificity), as well as accuracy ? gauged by the area under the ROC curve. This area is a measure of the discrimination and is referred to as the c-value of discrimination. An area of 1.0 indi- cates a perfect test and an area of 0.5 indicates an ineffective test, with an area of 0.7?0.8 representing a fair discrimination. Stan- dard error (SE) bars that signified a 95% confidence interval (CI) were included in these curves. RESULTS The postoperative mortality rate was 3.9% (12 patients). Patient demographics are shown in Table 3. Analysis of POS- SUM and P-POSSUM physiologic and operative severity scores Table 1. List of individual parameters required for calculation of the physiologic and operation severity scores in POSSUM and P-POSSUM Physiological score Age Cardiac signs Respiratory signs? ECG findings? Systolic blood pressure Pulse rate Haemoglobin White cell count? Urea Sodium? Potassium? Glasgow coma score? Operative severity score Operation type No. procedures? Operative blood loss? Peritoneal contamination Malignancy status Mode of surgery (CEPOD) ?Indicates parameters, which are not included in Cr-POSSUM. Cr-POS- SUM, colorectal-physiologic and operative severity score for the enumeration of mortality and morbidity; CEPOD, confidential enquiry into perioperative deaths; ECG, electrocardiogram; POSSUM, physiologic and operative sever- ity score for the enumeration of mortality and morbidity; P-POSSUM, Ports- mouth-physiologic and operative severity score for the enumeration of mortality and morbidity. Table 2. Equations used to calculate POSSUM, P-POSSUM and Cr-POSSUM POSSUM ln R/12R = 27.04 + (0.13 · physiologic score) + (0.16 · operative severity score) P-POSSUM ln R/12R = 29.065 + (0.1692 · physiologic score) + (0.1550 · operative severity score) Cr-POSSUM ln R/12R = 29.167 + (0.338 · physiologic score) + (0.308 · operative severity score) R = predicted risk. Cr-POSSUM, colorectal-physiologic and operative severity score for the enumeration of mortality and morbidity; POSSUM, physiologic and operative severity score for the enumeration of mortality and morbidity; P-POSSUM, Portsmouth-physiologic and operative severity score for the enumeration of mortality and morbidity. C211 2006 Royal Australasian College of Surgeons SCORING SYSTEMS IN COLORECTAL SURGERY 813 showed that the physiologic score in patients who died was sig- nificantly higher than in those who survived (median 23 vs 19; Mann?Whitney U-test, P = 0.0021). The operative severity score showed a similar trend (median 15 vs 12; Mann?Whitney U-test, P = 0.0031). Cr-POSSUM physiologic and operative severity scores uses different variables with a different scoring system and hence required a separate analysis. It was found that the physiologic score was higher in those who died than in those who survived (median 10 vs 9; Mann?Whitney U-test, P = 0.0162) with a similar finding for the operative severity score (median 10 vs 9; Mann?Whitney U-test, P = 0.0465). The POSSUM, P-POSSUM and Cr-POSSUM scores were col- lated and plotted on a bar graph that was shown to not have a normal distribution. It was established that there were signifi- cant differences in the median predicted risk values for those who died and for those who survived, for all three scoring systems: POSSUM (median 17.29 vs 7.01%; Mann?Whitney U-test, P = 0.0004), P-POSSUM (median 5.41 vs 1.88%; Mann?Whit- ney U-test, P = 0.0004) and Cr-POSSUM (median 11.55 vs 4.54%; Mann?Whitney U-test, P = 0.0064). A risk-stratification model was applied to each scoring system. This showed a clear correlation between an increase in risk and an increase in mortality (Tables 4?6). The cut-off values for each risk band were arbitrarily adjusted to suit specific scoring sys- tems, as each system differed from the others with respect to median and mean (e.g. POSSUM has been shown to overpredict mortality and as such has a higher stratified risk value than P-POSSUM). Receiver?operator characteristic curves were also designed for each of the predictive scoring systems (Figs 1?3), each with a SE for a 95%CI. The area under the ROC curves (?c-values of discrimination?) were 0.789 for POSSUM (SE = 0.068), 0.786 for P-POSSUM (SE = 0.068) and 0.728 for Cr-POSSUM (SE = 0.071). As the c-value approaches 1.0 it becomes increasingly more reliable and accurate as a predictive tool. It therefore appears that POSSUM and P-POSSUM were slightly better predictors of postoperative mortality in this patient group than Cr-POSSUM, but it is uncertain if this finding is statistically significant. DISCUSSION The purpose of this study was to evaluate the use of POSSUM, P- POSSUM and Cr-POSSUM as predictive tools in a colorectal surgical population in the Australasian clinical setting. All three scoring systems were found to provide a statistically significant prediction of the risk of mortality for a given patient. It was also shown that there was little difference with respect to accuracy between them. However, contrary to recent studies, there appeared to be a trend towards POSSUM and P-POSSUM being better indicators of mortality risk in patients than Cr-POS- SUM. 4,8,9 It is unlikely that this is attributable to the mortality rate of this study (3.9%) which was not different from those found in the previous parent studies; POSSUM (4.0%), P-POSSUM (2.9%) and Cr-POSSUM (5.7%). 1,4,5 The patient population in this study was relatively small. There were only 12 deaths in total and it was felt that there was insuf- ficient power to try and distinguish between the relative accura- cies of the predictive systems. Any pairwise comparisons may have yielded results that were misleading. It is therefore inaccu- rate to state whether the difference in the c-values of discrimina- tion between POSSUM, P-POSSUM and Cr-POSSUM were statistically significant or not. The original POSSUM has been shown to overpredict mortal- ity ? sometimes up to six times greater. This tends to occur in low-risk patients. It also underpredicts mortality in elderly or emergency patients. 2?4 However, this study has shown that POS- SUM is just as accurate at discriminating between death and survival in the group as P-POSSUM, even though absolute risk-value predictions of P-POSSUM have been considered more Table 3. Patient demographics No. patients 308 Male : female 147:161 Mean age (years) (range) 65 (16?91) Type of operation Right hemicolectomy 145 Left hemicolectomy 7 Subtotal colectomy 12 Abdomino-perineal resection 13 High anterior resection 16 Low anterior resection 31 Total colectomy 9 Sigmoid colectomy 37 Hartmann?s procedure 38 Malignant : benign disease 196:112 Table 4. Application of the risk-stratification model to POSSUM scores Predicted risk for POSSUM (%) Patient deaths Total no. operations Deaths/total operations (%) <20 7 268 2.6 20?40 3 32 9.3 >40 2 8 25 POSSUM, physiologic and operative severity score for the enumeration of mortality and morbidity. Table 5. Application of the risk-stratification model to P-POSSUM scores Predicted risk for P-POSSUM (%) Patient deaths Total no. operations Deaths/total operations (%) <20 7 268 2.6 20?40 3 32 9.3 >40 2 8 25 P-POSSUM, Portsmouth-physiologic and operative severity score for the enumeration of mortality and morbidity. Table 6. Application of the risk-stratification model to Cr-POS- SUM scores Predicted risk for Cr-POSSUM (%) Patient deaths Total no. operations Deaths/total operations (%) <20 7 275 2.5 20?40 4 27 14.8 >40 1 6 16.6 Cr-POSSUM, colorectal-physiologic and operative severity score for the enumeration of mortality and morbidity. C211 2006 Royal Australasian College of Surgeons 814 VATHER ET AL. reliable in previous studies from elsewhere. 4,5,9?11 Cr-POSSUM was developed as a scoring system for colorectal surgery, and as such some variables were deemed unnecessary. However, it seems that some of the variables which were excluded may have played a larger part in predicting postoperative mortality, as the accuracy of Cr-POSSUM as a predictive tool was not particularly exceptional. In fact, it appeared as though there was a tendency for this system to have a lower predictive capacity than the others, although it is uncertain whether this finding was statisti- cally significant. With development and refining of these different predictive systems comes an increase in the accuracy with which postoper- ative events can be predicted. However, there is still no perfect system for preoperative risk assessment. Cr-POSSUM comes close but requires ?peritoneal soiling? and ?malignancy status?, intraoperative and postoperative measures, respectively, to be included in its analysis and is hence not useful for preoperative risk assessment. 4 A system which could reliably predict postop- erative events (most importantly mortality) using exclusively pre- operative factors would be especially valuable in preoperative elective surgery consultations. All threePOSSUM scoring systemswere originally designed to be used as prospective tools. This study, however, used them retrospectively in a surgical audit setting. As such, assessment of the factors, which made up the physiologic and operative sever- ity scores could not be based on direct observation, but rather on old surgical records, a recognized problem with retrospective studies. A prospective approach to data collection may show dif- ferent results. In summary, all three POSSUM scoring systems provide some degree of mortality prediction in this population. POSSUM and P-POSSUM tended to be slightly more reliable predictors than Cr-POSSUM, but this was offset by the relative increase in effort required to collect data for all the variables rather than for a selected few. Nevertheless, there still remains a niche for a sys- tem that can accurately predict postoperative mortality using exclusively preoperative factors. REFERENCES 1. Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br. J. Surg. 1991; 78: 355?60. 2. Whiteley MS, Prytherch D, Higgins B, Weaver PC, Prout WG. An evaluation of the POSSUM surgical scoring system. Br. J. Surg. 1996; 83: 812?15. 3. Menon KV, Farouk R. Ananalysis ofthe accuracyofP-POSSUM scoring for mortality risk assessment after surgery for colorectal cancer. Colorectal Dis. 2002; 4: 197?200. 4. Tekkis PP, Prytherch DR, Kocher HM et al. Development of a dedicated risk-adjustment scoring system for colorectal surgery (colorectal POSSUM). Br. J. Surg. 2004; 91: 1174?82. 5. Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br. J. Surg. 1998; 85: 1217?20. 6. Senagore AJ, Warmuth AJ, Delaney CP, Tekkis PP, Fazio VW. POSSUM, p-POSSUM, and Cr-POSSUM: implementation issues in a United States health care system for prediction of outcome for colon cancer resection. Dis. Colon Rectum 2004; 47: 1435?41. 7. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29?36. 8. Ramkumar T, Ng V, Fowler L, Farouk R. A comparison of POSSUM, P-POSSUM and Cr-POSSUM for the prediction of Fig. 1. Receiver?operator characteristic curve for POSSUM scores. Area under curve = 0.7888; standard error (area) = 0.0679. Fig. 2. Receiver?operator characteristic curve for P-POSSUM scores. Area under curve = 0.7855; standard error (area) = 0.0684. Fig. 3. Receiver?operator characteristic curve for Cr-POSSUM scores. Area under curve = 0.7281; standard error (area) = 0.0708. C211 2006 Royal Australasian College of Surgeons SCORING SYSTEMS IN COLORECTAL SURGERY 815 postoperative mortality in patients undergoing colorectal resec- tion. Dis. Colon Rectum 2006; 49: 1?5. 9. Tekkis PP, Kessaris N, Kocher HM, Poloniecki JD, Lyttle J, Windsor AC. Evaluation of POSSUM and P-POSSUM scoring systems in patients undergoing colorectal surgery. Br. J. Surg. 2003; 90: 340?5. 10. Brooks MJ, Sutton R, Sarin S. Comparison of surgical risk score, POSSUM and p-POSSUM in higher-risk surgical patients. Br. J. Surg. 2005; 92: 1288?92. 11. Mohil RS, Bhatnagar D, Bahadur L, Rajneesh, Dev DK, Magan M. POSSUM and P-POSSUM for risk-adjusted audit of patients undergoing emergency laparotomy. Br. J. Surg. 2004; 91: 500?3. C211 2006 Royal Australasian College of Surgeons 816 VATHER ET AL. COMPARISON OF THE POSSUM, P-POSSUM AND CR-POSSUM SCORING SYSTEMS AS PREDICTORS OF POSTOPERATIVE MORTALITY IN PATIENTS UNDERGOING MAJOR COLORECTAL SURGERY