HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): Gary L. Grunkemeier John R. Handy, Jr Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Grunkemeier, G. L. Articles by Handy, J. R. Search for Related Content PubMed PubMed Citation Articles by Grunkemeier, G. L. Articles by Handy, J. R., Jr Related Collections Professional affairs

Ann Thorac Surg 2002;74:301-305
© 2002 The Society of Thoracic Surgeons

---

The statistician’s page

Propensity score analysis of stroke after off-pump coronary artery bypass grafting

^a Providence Health System, Portland, Oregon, USA

* Address reprint requests to Dr Grunkemeier, 9155 SW Barnes Rd, #33, Portland, OR 97225 USA
e-mail: ggrunkemeier{at}providence.org

The article by Stamou and colleagues [1] found that the stroke rate after coronary artery bypass grafting (CABG) was lower for off-pump CABG than for conventional, on-pump CABG. Postoperative stroke occurred in 2.5% of on-pump patients and only 1.2% of off-pump CABG patients, for an (unadjusted) odds ratio (OR) of 2.1 for on-pump versus off-pump CABG (see Appendix for definition of OR). Because the two patient groups were not similar with respect to potential risk factors for stroke, they used a logistic regression, which produced an adjusted OR of 1.6. As an adjunctive analysis, they computed a propensity score and used it to find on-pump matches for 72% of the off-pump CABG patients. For these matched subsets the OR for postoperative stroke was 1.8.

This expository article briefly describes propensity scores and demonstrates another way of using them to compare treatments. We apply this method to another data set, and in the process corroborate the findings of Stamou and colleagues.

Propensity score: the concept

Propensity score analysis is about 20 years old, but has only recently begun to be used to any extent in medical research (Fig 1). One of the originators of this method recently wrote an overview article for physicians [2], and another recent review by Eugene Blackstone [3] provides a thorough discussion of this method.

View larger version (15K): [in this window] [in a new window]   Fig 1. The number of papers from a Medline search using the text string "propensity score" according to the year of publication. This term has had a recent increase, although it is still not widely used.

Many factors influence which treatment patients receive in a nonrandomized observational study. Physicians usually select the treatment for a particular patient based on his or her characteristics (age, severity of disease, type of anatomy, and so on). If we knew this physician-specific selection rule, and had available the characteristics of each patient that went into this deci-sion, we could compute the probability of his or her getting one of the two treatments.

Suppose we did not know the selection rule, but had available data on a group of patients, including which treatment they received. Then we could approximate the rule by producing a "risk model." We regularly use logistic regression to produce risk models that use patient characteristics to predict the probability of postoperative events such as death (although we really know whether each patient in the study actually died or not). The propensity score is just the result of a multivariable risk model for the event "treatment." For each patient it provides the probability he or she received the treatment (although we already know that, too).

In comparing on-pump versus off-pump CABG, we will pattern our model on the one by Stamou and colleagues and make on-pump the "treatment" and off-pump CAB the "control." Then, the propensity score is simply the probability that a patient, given his or her particular set of characteristics, received on-pump CABG. A group of patients with the same propensity score are thus equally likely to have been assigned to the on-pump treatment. And, therefore, they were also equally likely to have been assigned to off-pump CABG. In fact, some of them got on-pump and some off-pump CABG, just as if they had been randomly allocated to whichever treatment they actually received. We might think of them as "randomized after the fact."

Matching

Propensity scores can be used in different ways in the analysis. Stamou and colleagues used matching [4]. They had a large number of on-pump CABG patients (78%) and a smaller number of off-pump CABG patients (22%). They used logistic regression analysis to calculate a propensity score, and then for each off-pump CABG patient they found the on-pump patient with the closest propensity score. They were able to find matches for 72% of the off-pump CABG patients, and then used a second logistic regression to compute the OR for stroke between these two matched samples.

Stratification

Stratification is another way to use the propensity score to adjust comparisons and reduce bias [5]. It has the advantage of potentially using all of the patients. (Stamou and associates were only able to enroll 32% of all patients using the matching technique.) With this method, the propensity scores are consecutively ordered (both groups together), and divided into groups, usually five, of equal size. The result is that the patients in each quintile all have similar probabilities of receiving the treatment. It has been shown that by dividing the propensity scores into quintiles, it is possible to achieve balance between treatment groups within each quintile, and remove more than 90% of the bias due to each of the covariates [5].

Providence off-pump CABG experience

Providence Health System includes nine cardiac surgery programs in four western states that submit data to a common database and meet regularly to discuss quality improvement initiatives. During 1998 to 2000, 7,955 isolated CABG (no previous or concomitant valve replacement) were performed, 15% of which were off-pump CABG. Interestingly, the use of off-pump CABG ranged from 1% to 72% among the nine hospitals (Fig 2). This wide variation in off-pump CABG usage justifies continued research inquiries into the relative efficacy of these two surgical approaches. This variation also makes "hospital" one of the most important "risk factors" for off-pump CABG. There was a natural grouping of hospitals into four groups according to off-pump CABG usage: low, moderate, medium, and high (Fig 2), and the (unadjusted) stroke rates were lower for off-pump CABG in each group (Fig 3). Overall, the stroke rates were 2.2% versus 1.2% (arrows in Fig 3), for an unadjusted OR (95% confidence interval) of 2.0 (1.1, 3.5), almost identical to that of Stamou and colleagues.

View larger version (38K): [in this window] [in a new window]   Fig 2. Off-pump coronary artery bypass grafting (OPCAB) utilization by hospital. The percentage of off-pump coronary artery bypass graftings from 1988 to 2000 varied greatly among the nine hospitals. The total number of operations is shown below each bar. Hospitals with larger numbers of cases tended to have lower off-pump coronary artery bypass grafting usage. Hospitals were put into four groups depending on off-pump coronary artery bypass grafting usage; the low utilization hospitals (<10% off-pump coronary artery bypass grafting) were not used for the comparison of off-pump coronary artery bypass grafting to on-pump results.

View larger version (41K): [in this window] [in a new window]   Fig 3. Percentage of postoperative stroke by off-pump coronary artery bypass grafting (OPCAB) versus on-pump for hospitals with low, moderate, medium, and high off-pump coronary artery bypass grafting usage. In each group, the stroke percentage was lower for patients with off-pump coronary artery bypass grafting. The arrows indicate the mean values for all hospitals: 2.2% for on-pump and 1.1% for off-pump coronary artery bypass grafting. (CVA = cerebrovascular accident; Post-op = postoperative.)

View larger version (24K): [in this window] [in a new window]   Fig 4. Boxplots of the propensity scores for off-pump coronary artery bypass grafting (OPCAB) and on-pump patients, excluding the low usage hospitals. For each distribution, the interquartile range (the values for the 25th and 75th percentiles) form the lower and upper edges of the box. The horizontal line within the box indicates the median (50th percentile) and the outer horizontal lines at the end of the vertical lines indicate the maximum and minimum. The number of patients in each group is shown below the horizontal axis.

View larger version (20K): [in this window] [in a new window]   Fig 5. Boxplots of the propensity scores for off-pump coronary artery bypass grafting (OPCAB) and on-pump patients, excluding the low usage hospitals, grouped according to quintile of propensity score. There is fairly close agreement within each quintile. The number of patients in each group is shown below the horizontal axis.

As did Stamou and colleagues, we performed logistic regression analysis with postoperative stroke as the outcome. A model with the significant risk factors of older age, female gender, prior stroke, and on-pump CABG had a c-statistic of 0.74, indicating fair discrimination. The OR for on-pump in this model was 2.7 (1.3, 5.6), which compared nicely with the pooled value from the propensity analysis. This adjusted OR was higher than the raw OR, because off-pump CABG patients had higher percentages of the other risk factors in the model (Table 1). Hospital was not significant in this model, and using all hospitals, including low off-pump CABG usage, produced virtually the same regression model.

A third way of using propensity score is to use it in a logistic regression, either by itself or in conjunction witha model such as the one above. The propensity score was not a significant addition to the above model and forcing it into the model changed the OR for off-pump CABG only slightly, to 2.6.

Comment

The propensity score methodology shares the limitations of all risk models. It can only account for factors that are available. To be successful may require many variables; one of the original examples used up to 74 variables to produce the score [5]. The result of the regression is only an estimate of the propensity, not the true propensity itself. And at best, propensity scores are only approximations of the real thing, randomization.

Stamou and associates used propensity scores to produce matched samples, but could not use all of the patients; the matched samples included only 32% of the patients. We excluded patients operated on in low off-pump CABG usage hospitals, and produced quintiles using the others. But two of the quintiles had numbers too small in one of the groups to make satisfactory comparisons. Conventional logistic regression analysis produced a more satisfying result, and all of the patients could be used.

Propensity score analysis was recently used in two articles comparing valve repair versus replacement for ischemic mitral valve disease. One of the articles was able to use it successfully [8], whereas the authors of the other article tried to but were unable to find satisfactory matches [9]. This prompted an editorial comment by Craig Miller in the same issue, questioning this apparent inconsistency [10].

Propensity score analysis may or may not be helpful in a particular comparison study. A recent review of the topic concluded: "Finally, it is important to note that we are not advocating the use of only propensity scores in analyses of observational studies, rather we are encouraging the use of propensity scores in addition to traditional methods of analysis" [6]. Stamou and associates used propensity scores and traditional methods that both yielded a significant OR for postoperative stroke in favor of off-pump CABG, and our analysis of an independent data set did the same.

Acknowledgments

Providence Health System provided data from the cardiac surgery programs at: Providence Anchorage Medical Center, Alaska; Providence Everett Medical Center, Washington; Providence Campus, Swedish Medical Center, Seattle, Washington; Providence St. Peter Hospital, Olympia, Washington; Providence Yakima Medical Center, Yakima, Washington; Providence Portland Medical Center, Portland, Oregon; Providence St. Vincent Medical Center, Portland, Oregon; Providence St. Joseph Medical Center, Burbank, California; Providence Holy Cross Medical Center, Mission Hills, California. We also thank Vicki Anderson for managing the database.

Appendix

Probabilities and relative risks
Probabilities are computed as the number of patients who had an event divided by the total number of patients. For example, in the moderate-to-high off-pump CABG usage groups, 49 of 2,120 on-pump patients had a postoperative stroke (Table 2). Thus, the probability is 49/2,120 = 0.0231 (this is an approximate equality, as the answer is rounded off to 4 decimal places). Multiplying this by 100 converts it to a percentage of about 2.3%. The ratio of two probabilities is called a risk ratio or relative risk (RR).

The ratio of two odds is called an odds ratio (OR). The (unadjusted) OR for stroke comparing Providence Health System on-pump to off-pump CABG is 0.0237/0.0112 = 2.11 (Table 2), and the RR is 2.08. Odds are similar to probabilities for low-risk events (such as postoperative stroke), and are often used because the adjusted ORs can be determined from logistic regression coefficients.

Logistic regression
Logistic regression is a method of incorporating multiple risk factors into a model to estimate a binary outcome, in our case stroke. The output consists of coefficients for each risk factor: positive means a risk (as for age, usually) and negative means a protective factor (as for ejection fraction, usually). Taking the exponential function of a coefficient gives the (adjusted) OR for that factor. If it is a risk factor, the OR will be greater than one, and if it is protective the OR will be less than one. A logistic regression for postoperative stroke with the single risk factor on-pump would yield the raw OR for on-pump given in Table 2.

Confidence intervals
There is, in general, a duality between confidence intervals and p values. If the 95% confidence interval for an OR does not contain the value one, then that OR is significantly different from one at the 5% level (p < 0.05). Conversely, if the 95% confidence interval for an OR does contain the value one, then that OR is not significantly different from one (p > 0.05). Confidence intervals, however, are more informative than p values; they not only indicate the statistical significance, but also the clinical range of values. This is true for both significant and nonsignificant results.

References

1. Stamou S.C., Jablonski K.A., Pfister A.J., et al. Stroke after conventional versus minimally invasive coronary artery bypass. Ann Thorac Surg 2002;74:394-399.[Abstract/Free Full Text]
2. Rubin D.B. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757-763.[Abstract/Free Full Text]
3. Blackstone E.H. Comparing apples and oranges. J Thorac Cardiovasc Surg 2002;123:8-15.[Free Full Text]
4. Rosenbaum P.R., Rubin D.B. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. AmerStat 1985;39:33-38.
5. Rosenbaum P.R., Rubin D.B. Reducing bias in observational studies using subclassification on the propensity score. JASA 1984;79:516-524.
6. D’Agostino R.B., Jr Tutorial in biostatistics: propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med 1998;17:2265-2281.[Medline]
7. Lachin J.M. Biostatistical methods: the assessment of relative risks. New York: John Wiley & Sons, 2000.
8. Gillinov A.M., Wierup P.N., Blackstone E.H., et al. Is repair preferable to replacement for ischemic mitral regurgitation?. J Thorac Cardiovasc Surg 2001;122:1125-1141.[Abstract/Free Full Text]
9. Grossi E.A., Goldberg J.D., LaPietra A., et al. Ischemic mitral valve reconstruction and replacement: comparison of long-term survival and complications. J Thorac Cardiovasc Surg 2001;122:1107-1124.[Abstract/Free Full Text]
10. Miller D.C. Ischemic mitral regurgitation redux-to repair or to replace?. J Thorac Cardiovasc Surg 2001;122:1059-1062.[Free Full Text]

This article has been cited by other articles:

				B. F. Meyers, P. K. Sultan, T. J. Guthrie, S. S. Lefrak, G. E. Davis, G. A. Patterson, J. D. Cooper, and R. D. Yusen Outcomes after unilateral lung volume reduction. Ann. Thorac. Surg., July 1, 2008; 86(1): 204 - 212. [Abstract] [Full Text] [PDF]

				A. Ali, Y. Abu-Omar, A. Patel, Z. Ali, A. Y. Sheikh, A. Akhtar, A. Pavlovic, P. Theodorou, T. Athanasiou, and J. Pepper Valve failure following homograft aortic valve replacement: does implantation technique have an effect? Eur. Heart J., June 1, 2008; 29(11): 1454 - 1462. [Abstract] [Full Text] [PDF]

				P. C. Austin Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement. J. Thorac. Cardiovasc. Surg., November 1, 2007; 134(5): 1128 - 1135. [Abstract] [Full Text] [PDF]

				M. Mishra, R. Malhotra, A. Karlekar, Y. Mishra, and N. Trehan Propensity Case-Matched Analysis of Off-Pump Versus On-Pump Coronary Artery Bypass Grafting in Patients With Atheromatous Aorta Ann. Thorac. Surg., August 1, 2006; 82(2): 608 - 614. [Abstract] [Full Text] [PDF]

				A. J. Rastan, J. I. Eckenstein, B. Hentschel, A. K. Funkat, J. F. Gummert, N. Doll, T. Walther, V. Falk, and F. W. Mohr Emergency Coronary Artery Bypass Graft Surgery for Acute Coronary Syndrome: Beating Heart Versus Conventional Cardioplegic Cardiac Arrest Strategies Circulation, July 4, 2006; 114(1_suppl): I-477 - I-485. [Abstract] [Full Text] [PDF]

				G. N. Djaiani Aortic arch atheroma: stroke reduction in cardiac surgical patients. Seminars in Cardiothoracic and Vascular Anesthesia, June 1, 2006; 10(2): 143 - 157. [Abstract] [PDF]

				E. I. Kapetanakis, D. A. Medlam, K. R. Petro, E. Haile, P. C. Hill, M. K.C. Dullum, A. S. Bafi, S. W. Boyce, and P. J. Corso Effect of Clopidogrel Premedication in Off-Pump Cardiac Surgery: Are We Forfeiting the Benefits of Reduced Hemorrhagic Sequelae? Circulation, April 4, 2006; 113(13): 1667 - 1674. [Abstract] [Full Text] [PDF]

				S. C. Stamou, P. C. Hill, E. Haile, S. Prince, M. J. Mack, and P. J. Corso Clinical outcomes of nonelective coronary revascularization with and without cardiopulmonary bypass J. Thorac. Cardiovasc. Surg., January 1, 2006; 131(1): 28 - 33. [Abstract] [Full Text] [PDF]

				D. N. Wijeysundera, W. S. Beattie, G. Djaiani, V. Rao, M. A. Borger, K. Karkouti, and R. J. Cusimano Off-Pump Coronary Artery Surgery for Reducing Mortality and Morbidity: Meta-Analysis of Randomized and Observational Studies J. Am. Coll. Cardiol., September 6, 2005; 46(5): 872 - 882. [Abstract] [Full Text] [PDF]

				J. S. Lawton, H. B. Barner, M. S. Bailey, T. J. Guthrie, N. Moazami, M. K. Pasque, M. R. Moon, and R. J. Damiano Jr Radial Artery Grafts in Women: Utilization and Results Ann. Thorac. Surg., August 1, 2005; 80(2): 559 - 563. [Abstract] [Full Text] [PDF]

				A. Weerasinghe, T. Athanasiou, S. Al-Ruzzeh, R. Casula, P. P. Tekkis, M. Amrani, P. Punjabi, K. Taylor, R. Stanbridge, and B. Glenville Functional Renal Outcome in On-Pump and Off-Pump Coronary Revascularization: A Propensity-Based Analysis Ann. Thorac. Surg., May 1, 2005; 79(5): 1577 - 1583. [Abstract] [Full Text] [PDF]

				E. I. Kapetanakis, D. A. Medlam, S. W. Boyce, E. Haile, P. C. Hill, M. K.C. Dullum, A. S. Bafi, K. R. Petro, and P. J. Corso Clopidogrel administration prior to coronary artery bypass grafting surgery: the cardiologist's panacea or the surgeon's headache? Eur. Heart J., March 2, 2005; 26(6): 576 - 583. [Abstract] [Full Text] [PDF]

				T. L. Frankel, S. C. Stamou, R. C. Lowery, E. I. Kapetanakis, P. C. Hill, E. Haile, and P. J. Corso Risk factors for hemorrhage-related reexploration and blood transfusion after conventional versus coronary revascularization without cardiopulmonary bypass Eur. J. Cardiothorac. Surg., March 1, 2005; 27(3): 494 - 500. [Abstract] [Full Text] [PDF]

				E. I. Kapetanakis, S. C. Stamou, M. K.C. Dullum, P. C. Hill, E. Haile, S. W. Boyce, A. S. Bafi, K. R. Petro, and P. J. Corso The Impact of Aortic Manipulation on Neurologic Outcomes After Coronary Artery Bypass Surgery: A Risk-Adjusted Study Ann. Thorac. Surg., November 1, 2004; 78(5): 1564 - 1571. [Abstract] [Full Text] [PDF]

				D. M. Shahian, E. H. Blackstone, F. H. Edwards, F. L. Grover, G. L. Grunkemeier, D. C. Naftel, S. A.M. Nashef, W. C. Nugent, and E. D. Peterson Cardiac Surgery Risk Models: A Position Article Ann. Thorac. Surg., November 1, 2004; 78(5): 1868 - 1877. [Abstract] [Full Text] [PDF]

				T. Athanasiou, O. Aziz, O. Mangoush, S. Al-Ruzzeh, S. Nair, V. Malinovski, R. Casula, and B. Glenville Does off-pump coronary artery bypass reduce the incidence of post-operative atrial fibrillation? A question revisited Eur. J. Cardiothorac. Surg., October 1, 2004; 26(4): 701 - 710. [Abstract] [Full Text] [PDF]

				M. J. Mack, P. Brown, F. Houser, M. Katz, A. Kugelmass, A. Simon, S. Battaglia, L. Tarkington, S. Culler, and E. Becker On-Pump Versus Off-Pump Coronary Artery Bypass Surgery in a Matched Sample of Women: A Comparison of Outcomes Circulation, September 14, 2004; 110(11_suppl_1): II-1 - II-6. [Abstract] [Full Text] [PDF]

				S. C. Stamou, K. A. Jablonski, J. M. Garcia, S. W. Boyce, A. S. Bafi, and P. J. Corso Operative mortality after conventional versus coronary revascularization without cardiopulmonary bypass Eur. J. Cardiothorac. Surg., September 1, 2004; 26(3): 549 - 553. [Abstract] [Full Text] [PDF]

				P. Sergeant, P. Wouters, B. Meyns, C. Bert, J. Van Hemelrijck, C. Bogaerts, G. Sergeant, and K. Slabbaert OPCAB versus early mortality and morbidity: an issue between clinical relevance and statistical significance Eur. J. Cardiothorac. Surg., May 1, 2004; 25(5): 779 - 785. [Abstract] [Full Text] [PDF]

				J. S. Coselli, S. A. LeMaire, L. D. Conklin, and G. J. Adams Left heart bypass during descending thoracic aortic aneurysm repair does not reduce the incidence of paraplegia Ann. Thorac. Surg., April 1, 2004; 77(4): 1298 - 1303. [Abstract] [Full Text] [PDF]

				R. Sharony, E. A. Grossi, P. C. Saunders, A. C. Galloway, R. Applebaum, G. H. Ribakove, A. T. Culliford, M. Kanchuger, I. Kronzon, and S. B. Colvin Propensity case-matched analysis of off-pump coronary artery bypass grafting in patients with atheromatous aortic disease J. Thorac. Cardiovasc. Surg., February 1, 2004; 127(2): 406 - 413. [Abstract] [Full Text] [PDF]

				S. Al-Ruzzeh, T. Athanasiou, S. George, B. E. Glenville, A. C. DeSouza, J. R. Pepper, and M. Amrani Is the use of cardiopulmonary bypass for multivessel coronary artery bypass surgery an independent predictor of operative mortality in patients with ischemic left ventricular dysfunction? Ann. Thorac. Surg., August 1, 2003; 76(2): 444 - 451. [Abstract] [Full Text] [PDF]

				V. A. Ferraris and S. P. Ferraris Assessing the medical literature: let the buyer beware Ann. Thorac. Surg., July 1, 2003; 76(1): 4 - 11. [Abstract] [Full Text] [PDF]

				R. G. Fuster, J. A. M. Argudo, O. G. Albarova, F. H. Sos, S. C. Lopez, M. J. D. Sorli, M. B. Codoner, and J. A. B. Minano Left ventricular mass index in aortic valve surgery: a new index for early valve replacement? Eur. J. Cardiothorac. Surg., May 1, 2003; 23(5): 696 - 702. [Abstract] [Full Text] [PDF]

This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): Gary L. Grunkemeier John R. Handy, Jr Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Grunkemeier, G. L. Articles by Handy, J. R. Search for Related Content PubMed PubMed Citation Articles by Grunkemeier, G. L. Articles by Handy, J. R., Jr Related Collections Professional affairs