HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): David M. Shahian Fred H. Edwards Victor A. Ferraris Constance K. Haan Jeffrey B. Rich Cynthia M. Shewan Eric D. Peterson Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Shahian, D. M. Articles by Peterson, E. D. Search for Related Content PubMed PubMed Citation Articles by Shahian, D. M. Articles by Peterson, E. D. Related Collections Education Professional affairs

Ann Thorac Surg 2007;83:S3-S12
© 2007 The Society of Thoracic Surgeons

---

Report of the STS Quality Measurement Task Force

Quality Measurement in Adult Cardiac Surgery: Part 1—Conceptual Framework and Measure Selection

David M. Shahian, MD^a,_*,, Fred H. Edwards, MD^b, Victor A. Ferraris, MD^c, Constance K. Haan, MD^b, Jeffrey B. Rich, MD^d, Sharon-Lise T. Normand, PhD^e, Elizabeth R. DeLong, PhD^f, Sean M. O’Brien, PhD^f, Cynthia M. Shewan, PhD^g, Rachel S. Dokholyan, MPH^f, Eric D. Peterson, MD, MPH^f

^a Tufts University School of Medicine, Boston, Massachusetts
^b Division of Cardiothoracic Surgery, University of Florida, Jacksonville, Florida
^c Division of Cardiovascular & Thoracic Surgery, University of Kentucky Chandler Medical Center, Lexington, Kentucky
^d Sentara Cardiovascular Research Institute, Norfolk, Virginia
^e Department of Health Care Policy, Harvard Medical School and ^eDepartment of Biostatistics, Harvard School of Public Health, Boston, Massachusetts
^f Duke Clinical Research Institute, Durham, North Carolina
^g The Society of Thoracic Surgeons, Chicago, Illinois

Accepted for publication January 12, 2007.

^_* Address correspondence to Dr Shahian, The Society of Thoracic Surgeons, 633 N Saint Clair St, Suite 2320, Chicago, IL 60611 (Email: shahian{at}comcast.net).

	Executive Summary

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
The Society of Thoracic Surgeons established a Quality Measurement Task Force to develop a methodology for the comprehensive assessment of adult cardiac surgery quality of care, including both individual measures and an overall composite quality score. In Part 1 of a two-part series, the Task Force describes the conceptual framework, principles, and guidelines used to select and categorize the individual measures that comprise the composite score.

Quality indicators were selected using the following principles:

1 Quality assessment should be at the level of the program or hospital rather than the individual surgeon.

2 Initial quality reports should focus on coronary artery bypass grafting surgery.

3 Quality measures should be chosen from among those endorsed by the National Quality Forum.

4 Quality measure selection should be consistent with the principles and criteria recommended in the 2006 Institute of Medicine report Performance Measurement: Accelerating Improvement.

5 Quality measures should be available as data elements within The Society of Thoracic Surgeons National Adult Cardiac Surgery Database.

6 Quality scores should consider structure, process, and outcomes.

7 Quality scores should assess three temporal domains—preoperative, operative, and postoperative.

8 Quality scores should satisfy multiple criteria for validity.

9 Quality scores should be interpretable and actionable by providers.

Eleven individual measures of coronary artery bypass grafting quality within four domains were selected:

1 Perioperative Medical Care, a process bundle of four medications including preoperative ß-blockade and discharge aspirin, ß-blockade, and lipid-lowering agents.

2 Operative Care, a single process measure—use of at least one internal mammary artery.

3 Risk-Adjusted Operative Mortality.

4 Postoperative Risk-Adjusted Major Morbidity, defined as the risk-adjusted occurrence of any of the following: renal failure, deep sternal wound infection, reexploration, stroke, or prolonged ventilation/intubation.

In summary, The Society of Thoracic Surgeons Quality Measurement Task Force has selected a group of measures to serve as the basis for comprehensive assessment of adult cardiac surgery quality. Part 2 of this report describes the statistical considerations relevant to combining these measures into composite scores, then using such scores as the basis for a provider rating system.

	Introduction

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
Performance measurement is an essential element of quality improvement initiatives and health care reform [1]. For two decades, cardiac surgeons have systematically collected and analyzed outcomes data in order to continually enhance the quality of care they deliver. Central to this effort was the development of The Society of Thoracic Surgeons National Adult Cardiac Surgery Database (STS NCD), which is the largest and most comprehensive single-specialty clinical database in health care. Through this resource, information is available on all aspects of adult cardiac procedures including preoperative patient characteristics, operative management, perioperative care, postoperative outcomes, and discharge medications and venue.

These data are used in numerous ways to improve selection, operative techniques, and perioperative care of cardiac surgical patients. Over the past decade, analysis of such risk-adjusted data has also been used by providers, health policy researchers, payers, and government agencies to assess the quality of cardiac surgical care. Until recently, these efforts were focused on the short-term outcomes (eg, operative mortality and morbidity) of cardiac surgery, particularly coronary artery bypass grafting (CABG) [2–4]. However, it is now recognized that such clinical outcomes are only one measure of overall health care quality. More comprehensive quality indicators are desirable, including intraoperative processes of care as well as perioperative measures that impact hospital outcomes, secondary prevention, and long-term health.

Continuing its tradition of leadership in efforts to assess and improve quality, The Society of Thoracic Surgeons recently established a Quality Measurement Task Force (QMTF) consisting of clinical, health policy, and statistical experts. Its charge is to develop methodologies to assess the overall quality of care delivered by cardiac surgery providers (used generically in these reports to refer to hospitals or cardiac surgical groups that participate in the STS NCD). Such information could serve several important functions: (1) to provide public accountability, a fundamental responsibility of any professional organization; (2) to facilitate quality improvement activities; and (3) to satisfy the reporting requirements of various pay for performance programs.

Commercial organizations, accrediting, and governmental agencies have previously developed cardiac surgery rating systems; however, most of these were designed without the involvement or endorsement of cardiac surgeons. Furthermore, all are based on administrative data with the exception of several state report cards (eg, New York, Massachusetts), and the latter have been limited primarily to operative outcomes. The goal of the QMTF is to develop a more comprehensive set of measures based on clinical data elements that have already received the endorsement of the National Quality Forum. With the widespread support of the provider community as well as broad external endorsement, it is expected that these measures could become the standard, benchmark methodology for assessing adult cardiac surgery quality.

The design of such comprehensive quality measures involved clinical, health policy, and statistical considerations. Part 1 of this report outlines the conceptual framework within which the QMTF conducted its deliberations and the guidelines used to select and categorize a comprehensive set of quality measures. Part 2 will describe the statistical considerations relevant to combining these measures into composite scores and subsequently using these scores as the basis for a provider rating system. It also presents the results of a pilot study of the proposed QMTF methodology using actual 2004 data from the STS NCD.

	The Concept of Quality

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
Quality is an abstract construct that cannot be directly measured. In the nomenclature of modern measurement theory, it is characterized by one or more latent (unobserved) variables or traits. The overt, observable manifestations of such underlying traits may be intuitively apparent at the extremes, but difficult to quantify over the broad range between extremes. Different observers may agree that a luxury automobile is high quality, and that an entry-level automobile is of lower quality. However, what are the characteristics that permit this classification to be made, and how is it possible to scale these defining characteristics, weight their relative importance, and combine them into meaningful overall scores?

To quantify abstract constructs like quality, intelligence, or musical ability, we typically rely upon some combination or composite of measurable surrogates that are thought to be associated with or contribute to that underlying trait. For example, to assess intelligence it might be useful to objectively measure visual spatial ability, critical reasoning, and problem-solving approaches, and then assign a score to each area as well as a composite overall score. Ideally, it should also be possible to validate this score against some end point that is believed to be the product of intelligence, such as academic achievement.

Quality is a major focus of American medicine, but how best to define this elusive concept for health care is particularly challenging. At a minimum, any proposed measures of provider quality should be associated with the results we would expect from excellent health care; for example, fewer hospital admissions for chronic diseases, and improved long-term survival. In current practice, quality is often characterized by some combination of structure, process, and outcome measures, a triad first proposed by Professor Avedis Donabedian at the University of Michigan. Birkmeyer and associates [5] suggest that the suitability of structure, process, or outcome measures to assess quality for a particular procedure is a function of both the potential for adverse consequences and the frequency with which the procedure is performed. For many surgical procedures and medical conditions, small sample sizes [6] or lack of serious adverse sequelae diminish the utility of statistical outcomes comparisons, and in such instances process or structural measures may be more appropriate. In contrast, the evaluation of CABG quality is less subject to such limitations. This procedure has important, quantifiable end points and is performed with sufficient frequency to justify the evaluation of outcomes, as well as the more widely applicable process and structural measures.

Structure
Structural measures refer to inherent characteristics of the provider that are believed to be associated with higher quality. These may include, for example, the volume of a particular type of procedure performed annually, advanced information technology such as electronic health records or computerized order entry, participation in an outcomes database registry, sophisticated radiographic imaging capabilities, number of approved residencies, and nurse staffing ratios. For many such measures, the strength of their relationship with desired health care outcomes is not well defined.

Without question, volume is the most extensively studied structural measure of quality in all of medicine and surgery. Whether this reflects "practice makes perfect" or "selective referral" to institutions perceived to be of higher quality is uncertain. The strength of the volume-outcome relationship varies widely. For complex procedures such as esophagectomy or pancreatectomy that are performed relatively infrequently even at high-volume institutions, the volume-outcome association is strong [7]. For some frequently performed, relatively standardized and mature complex procedures such as CABG, where annual hospital volumes may range from 100 to more than 1000 cases, the volume-outcome relationship is weak [3, 7–10].

Structural measures may be useful proxies for outcomes when the latter data are unavailable. Conversely, their relative utility diminishes when reliable outcomes data are accessible. Furthermore, structural measures are often not readily actionable by providers, and this diminishes their usefulness as quality improvement tools.

Process
Process measures reflect the extent to which a provider complies with evidence-based care guidelines. Ideally, use of process measures should be shown to correlate directly with outcomes of care, although only a limited number of such measures meet this criterion (Table 1). In the acute care setting, appropriate process measures might include the administration of aspirin and ß-blockade to reduce mortality in acute coronary syndromes, as well as "door to balloon" time for acute myocardial infarction. For chronic disease management, process measures include the degree of compliance with desirable practices such as frequent determination of hemoglobin A1c levels to optimize glycemic control. Evidence-based practice guidelines have been developed by professional organizations with the goal of decreasing practice variation and defining best-practice standards of care [11].

In addition to such theoretical concerns, measurement of compliance with process measures also presents practical difficulties. Many patients have legitimate contraindications to a particular measure, such as the use of ß-blockade in patients with significant heart block. Ideally, process measure compliance should reflect the number of eligible patients who received the treatment or therapy. In practice, agreement about what constitutes a legitimate exclusion has often been difficult even for expert panels, and mechanisms to document such exclusions are both problematic and susceptible to gaming. Many patients are ineligible for some or all of the process measures relevant to their condition, and the number of patients available for assessing provider practice is correspondingly reduced. Even for eligible patients, the proportion of outcome variability explained by adherence to process guidelines may be small [13].

Outcomes
Outcomes are the most obvious, intuitive, and historically used measures of quality. They also represent the quality metric that is generally most important to patients. Although known primarily as the founder of modern nursing practice, Florence Nightingale was in fact a highly regarded epidemiologist and statistician in the latter half of the 19th century [14–16]. Her careful recording of British military deaths in Crimea, and her subsequent report to the government, had a profound impact on the care of wounded soldiers and is one of the earliest practical examples of epidemiologic outcomes analysis. Nightingale also introduced the concept of risk adjustment, which is central to modern profiling efforts.

Unquestionably the most important, seminal contribution to the study of surgical outcomes was that of Ernest Amory Codman, a Boston surgeon who advocated the methodical reporting of both short-term and long-term results. Although on the staff of the Massachusetts General Hospital, Dr Codman began his own hospital that was dedicated to the practical application of these principles. His obsession with quality was neither shared nor appreciated by many of his contemporaries, perhaps partly because of the often brusque fashion with which he presented his views. Nonetheless, his efforts contributed substantially to the development of what ultimately became the American College of Surgeons and the Joint Commission on the Accreditation of Healthcare Organizations [16–19].

It was near the end of the 20th century before the next attempt to quantify health care outcomes on a large scale occurred. The Health Care Financing Administration (HCFA), now known as The Centers for Medicare & Medicaid Services (CMS), publicly released hospital mortality data for all institutions that received payment from HCFA. Because of serious deficiencies with this program, including lack of risk adjustment and the use of an administrative database not designed for outcomes analysis, the program was abandoned shortly after its inception. However, a number of specialty organizations concluded that properly executed data collection and analysis could be a powerful tool to assess and improve quality. In the late 1980s, the STS began development of what subsequently became the NCD [20]. This is currently the largest single specialty clinical database in the world, with entries on more than 3 million surgical procedures from more than 780 cardiac surgery practices. Statistical models have been developed for risk-adjusted mortality and morbidity, and the database also includes substantial information on process measures such as the use of the internal mammary artery (IMA) and of recommended medications such as ß-blockers, aspirin, and antilipid agents [4].

	Cardiac Surgery Quality

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
Cardiac surgeons were early adopters of data collection, measurement, and analysis. Their dominant procedure, CABG, is the most commonly performed of all complex operations. It is costly and has broad implications for population health. CABG surgery is the paradigm for outcome measurement given both its frequency and its potential for serious morbidity and mortality.

In recognition of its important public health implications, the National Quality Forum (NQF) recently convened a Cardiac Surgery Performance Measures Steering Committee and associated Technical Advisory Panel, both of which included representatives of the STS, payers, regulators, health policy experts, and consumers. Their charge was to identify and endorse a set of evidence-based measures that would accurately reflect the performance of cardiac surgery programs. Ultimately, a set of 21 structure, process, and outcomes measures were selected (Table 2). It is expected that various subsets or combinations of these measures will become the basis for regional and national quality monitoring and improvement efforts in cardiac surgery. With the exception of those measures pertaining to preoperative antibiotic administration, timing, and duration, the remaining measures are directly related to data elements included in the STS NCD, which further emphasizes the central importance of this instrument.

	Principles for Measure Selection

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

1 Initial public quality reports should focus on programs, not individual surgeons. Although a number of states, notably New York and Pennsylvania, publish surgeon-specific results, this continues to be a highly controversial topic. Performance reports on individual practitioners are fraught with even more vexing conceptual and technical concerns than reports on hospitals. The potential for unintended negative consequences is greater, leading many experts to recommend caution regarding their use [2, 21, 22].
Cardiac surgery is fundamentally a team endeavor, and publicly releasing information on just one element of that team detracts from the more important goals of improving overall team coordination, standardization of care, and clinical effectiveness. Surgeon report cards also focus attention on the one member of the team who ultimately decides which patients to accept for surgery, and this increases the potential for risk aversion. Hospitals do not decide which patients are acceptable risks for surgery—surgeons ultimately bear this responsibility alone. The inevitably higher mortality associated with surgery on high-risk populations will be diluted within the overall results of a hospital or practice group. In contrast, a few such patients may have a substantial negative impact on the results of an individual surgeon. Research indicates that when individual surgeon results are made public, surgeons become more risk averse because they fear the impact of a bad result on their public report card [2, 23, 24]. This may result in denial of care to the very group of high-risk patients who would potentially benefit most from surgery [2, 25, 26].
From a practical standpoint, sample size is also an issue and generally requires accumulating data during a period of 2 to 3 years. The resulting reports include some data that may be as much as 4 years old after the appropriate audit, validation, and analysis have been performed, and some believe this limits their relevance. Some have advocated aggregating data across several measures of individual practitioner performance as a way to mitigate this problem.
Because of the many serious and unresolved concerns regarding this issue, as well as specific prohibitions included in existing STS NCD Participation Agreements, the STS currently does not publicly release information on individual practitioners. It should be noted that alternative approaches to practitioner-level public reporting do exist. For example, it is possible to collect and analyze individual practitioner data confidentially and to have such data reviewed by a committee of experts including clinicians, statisticians, and health policy experts.

2 Initial quality reports should focus on CABG. Coronary artery bypass grafting is the most commonly performed and studied of all cardiac surgical procedures. It is relatively standardized, and it has a broad range of well-defined major and minor adverse outcomes, including death. Results for this procedure have become the benchmark by which overall adult cardiac surgery performance is assessed by consumers, regulators, and payers. It should be noted, however, that excellent risk models and data exist for valve surgery as well [27–29]. Early evidence suggests that the results for such procedures may have greater variability among programs than those for CABG, making them a likely focus for subsequent quality profiling.

3 All measures should be derived from the set of 21 NQF-endorsed adult cardiac surgery measures. The NQF is a private, not-for-profit organization established in 1999. Its mission is to develop and endorse standardized health care performance measures to provide public accountability, support quality improvement efforts, promote patient safety, and to serve as the basis for pay-for-performance programs and value-based purchasing. The NQF has quickly achieved national respect and credibility because of the extensive process undertaken before any candidate quality measure is endorsed, as well as the diverse constituencies included on its panels. These include providers, payers, consumers, accrediting agencies, and health policy experts. NQF-endorsed measures are widely accepted by both payers and government agencies, and they have legal status as national voluntary consensus standards.
The NQF has published a set of National Voluntary Consensus Standards for Cardiac Surgery that includes 21 structure, process, and outcome measures (Table 2), all of which are consistent with the Institute of Medicine goals for health care: safe, effective, patient-centered, timely, efficient, and equitable. These measures have thus become the set of nationally recognized parameters for assessing quality in cardiac surgery. Use of some combination of these NQF-endorsed measures largely mitigates concerns about their selection or appropriateness and also maximizes the acceptability of the measure to all constituencies.

4 Measure selection should be consistent with the principles and criteria recommended by the Institute of Medicine. In its two previous landmark studies, To Err is Human [30] and Crossing the Quality Chasm [31], the Institute of Medicine established the improvement of health care quality as a major national priority. In its 2006 publication Performance Measurement: Accelerating Improvement [1], the Institute of Medicine proposes principles and criteria for the selection of measures by which quality may be assessed. These recommendations include comprehensive; evidence-based; longitudinal; supportive of multiple uses and stakeholders; intrinsic to the care process; providing opportunity for patient input; patient-level, population-based, and systems-level measurement; acknowledgement of shared accountability; a learning system, continually evaluating performance and advancing knowledge; independent and sustainable; scientifically sound (reliable, valid, and explicit); feasible; having important health implications; and aligned with existing leading measure sets [1].
These guidelines will undoubtedly become the national benchmark by which to evaluate quality and performance measures. Accordingly, the QMTF assessed all its proposed measures to assure compliance with pertinent Institute of Medicine recommendations.

5 Measures should be readily accessible from the STS NCD instrument. With the exception of those related to perioperative antibiotics, the NQF-endorsed cardiac surgery measures are based upon or may be derived from data in the STS NCD. The STS NCD is the largest, single-specialty clinical database in the world and is administered by the highly respected Duke Clinical Research Institute. It has unquestioned credibility among providers, payers, and regulators. As a clinical database, it is clearly superior in accuracy to the administrative databases used as the basis for many government and commercial report cards [32, 33].
Nationally, more than 780 hospitals or adult cardiac surgery practices devote extensive time and resources to the accurate collection of STS NCD data elements. It is both desirable and more efficient to develop quality measures from these existing elements without necessitating any separate work process or substantial additional resources. Consistent with its longstanding practice, the STS will modify the NCD as necessary to meet the evolving needs for additional information.
As the STS NCD becomes the basis for quality reports to governmental agencies, payers, referring physicians, and the public, it will be crucial to assure the accuracy and validity of these data. Although preliminary reports [34] are encouraging, the STS has nevertheless embarked upon a major auditing initiative to confirm these findings. Undoubtedly such efforts will continue to evolve, hopefully with additional external funding.

6 Quality assessment should consider, either separately or as a composite, each of Donabedian’s three domains of quality—structure, process, and outcomes. This quality triad is widely accepted in health care, and measures from all three domains are available from the STS NCD. The actual method of presenting scores from individual domains—separately, as a roll-up composite score, or both—has statistical implications that are discussed in Part 2 of this report. Furthermore, the specific report format can be tailored to particular constituencies.

7 Quality scores should assess all three relevant temporal domains of the cardiac surgical experience—preoperative, operative, and postoperative (including long-term) care. Preparation for surgery, conduct of the surgical procedure, postoperative morbidity and mortality, and discharge protocols are all important aspects of the surgical experience. Often, they involve different personnel because cardiac surgery is the quintessential team endeavor. An appropriate comprehensive quality measure should have representation from each of these periods of care. The only time frame that currently is difficult to study is long term, but with better information systems and electronic health records, this too will become an important component of quality scores. Our quality measures begin to address this long-term perspective by including discharge medications that primarily impact secondary prevention (aspirin, ß-blockade, lipid-lowering agents). The time of hospital discharge after CABG may be one of the most effective opportunities to institute such therapy [35].

8 Quality scores should satisfy multiple criteria for validity. Quality scores must be acceptable to both providers, who are being evaluated, and to payers, accrediting agencies, and regulators. The measures should have face validity (the measures appear reasonable for their intended purpose), content validity (the selected measures encompass all relevant aspects of the concept that is being measured), construct validity (the measures accurately reflect the construct they are designed to measure), and attributional validity (risk-adjusted outcome measures adequately compensate for patient factors, so that they reflect quality of care and not case mix) [15, 36].

9 Quality scores should be interpretable and actionable by providers. The goal is a transparent, clearly defined methodology whose results provide some understanding of a provider’s (hospital or program) performance relative to that of other providers. Reporting options would include actual numerical scores, a three-star or five-star rating system, performance tiers with upper and lower bands as currently employed in the CMS Premier Demonstration Project, or statistical techniques that determine with high probability that a given provider’s results differ from the performance of its peers.
Quality scores should enable providers to enact changes that will improve both their scores and, more importantly, the actual quality of care delivered to their patients. In addition to their overall composite score and rating, providers should also receive scores for the four domains and the eleven individual measures, thus enabling them to better focus their quality improvement efforts.
As will be discussed in Part 2 of this report, the QMTF investigated a number of sophisticated statistical methodologies, including approaches derived from modern psychometric or measurement theory (eg, latent variable analysis, item response theory). Unless such methodologies had unequivocally superior scientific justification for our application, or proved to have superior performance when applied to STS data in pilot studies, we chose to use simpler and more intuitive approaches.

	Specific Measure Selection

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
In addition to these nine major principles, the QMTF also considered evolving new trends in health care quality assessment, including the use of bundled measures and "all or none" scoring as advocated by the Institute for Healthcare Improvement and the Institute of Medicine [1, 37]. Consider a bundle of care measures for a particular condition, such as acute myocardial infarction. If experts agree that all components of this bundle (eg, aspirin after acute myocardial infarction, ß-blockade, 90-minute door-to-balloon time, statin on discharge) are important for optimal patient care, then the desired benchmark performance is compliance with all components. Failure to achieve one component of the bundle results in no credit. Partial credit not only sets too low a goal but also requires subjective decisions about how to weight the various components of the bundle.

Although the all or none approach in essence assigns equal weights to each component of the bundle, it is preferable to regard them as a group of important measures that are all essential to providing the highest quality care. By analogy, it would be impossible to weight the relative importance of the heart, kidneys, and liver to the human body. They all perform quite different tasks but all must work in concert for optimal bodily function. Significant failure of any one of these organs is potentially fatal.

The QMTF ultimately chose eleven individual quality measures grouped within four domains (Table 3). This includes all relevant CABG process and outcomes measures currently endorsed by the NQF. Because of the credibility of the NQF process, we did not conduct additional statistical analyses to determine whether any of these measures could be eliminated, although subsequent pilot studies did not demonstrate substantial correlation among these measures. This suggests that they provide complementary rather than redundant information.

1 Perioperative Medical Care Process Measure: All or none compliance with the use of four specific medications (preoperative ß-blockade and postoperative aspirin, ß-blockade, and lipid-lowering therapy) shown to be associated with improved short or long-term outcomes for CABG patients. These are typically ordered by the surgical care team (staff and resident physicians, physician assistants, nurse practitioners) or cardiologists, often through the use of standardized order sheets.

2 Operative Process Measure: Use of the IMA, the single most important and universally accepted process measure in CABG surgery.

3 Risk-Adjusted Operative Mortality: The most important outcomes measure, and one for which CABG surgery is particularly well suited because of its frequency and the availability of excellent risk models.

4 Postoperative Risk-Adjusted Major Morbidity: Serious complications can be devastating for patients and costly for payers. The STS QMTF morbidity measure is the risk-adjusted rate of patient discharge from a given institution with any of the following five major complications: stroke, deep sternal wound infection, reexploration for any cause, prolonged intubation/ventilation, or renal insufficiency. Definitions of these five morbidities are available from the STS and NQF Websites [38, 39].

The following describes the QMTF rationale for selection of these measures and for the rejection of several others, in the context of the principles established previously.

Perioperative Medical Care
To assess preoperative and postoperative medical care, four NQF-endorsed process measures were selected, all of which are included as data elements in the STS NCD. These include preoperative ß-blockade and postoperative aspirin, ß-blockade, and lipid-lowering therapy. All these measures are supported by extensive studies, either specifically pertaining to CABG surgery or to the secondary prevention of disease progression in patients with established coronary atherosclerosis [40–45]. Each measure is also included in the 2006 American College of Cardiology (ACC)/American Heart Association (AHA) Guidelines for Secondary Prevention for Patients with Coronary and Other Atherosclerotic Vascular Disease [46].

The QMTF discussed the problem of legitimate exclusions to process measures, as noted previously. Two options for dealing with such exclusions were considered. Ideally, a full list of acceptable exclusions would be agreed upon, included in the STS NCD instrument, and documented as present by the provider when appropriate; however, how to make such an approach operational proved to be problematic. There were concerns regarding what constituted legitimate exclusions and how to prevent gaming of such exclusions. The second option, which we chose, is to assume that the percentage of patients with legitimate exclusions to each process measure should not vary substantially among programs (in contrast to case mix, which can be quite variable). Accordingly, consistent with current NQF specifications, only a few exclusions are recognized for these medical process measures and for the operative process and postoperative outcome measures. The three discharge medication measures are calculated only among patients who survive until hospital discharge; the IMA measure excludes patients undergoing repeat CABG surgery; and the permanent stroke measure excludes patients with a previous cerebrovascular accident.

Because of this limited number of recognized exclusions, all programs will fall short of 100% compliance with the entire bundle of perioperative measures for all patients, but no program should be disproportionately disadvantaged. In a pilot study of data from more than 500 STS NCD participants (Part 2 of this report), application of this all or none approach to perioperative process measures produced a variable and relatively symmetric distribution of compliance percentages. Given that each of these 4 medical process measures was deemed by the QMTF to be efficacious and important for the care of CABG patients, it is noteworthy that current all or none compliance rates vary substantially among STS NCD participant programs. Hence, there is significant room for process improvement. Furthermore, none of these measures would require any additional funding or resources by institutions. They require only the will to change, a standardized order set, and an audit mechanism.

Operative Care
For a surgical procedure regarded as being relatively standardized, operative processes of care for CABG vary widely among cardiac surgery programs. For many of these processes, there is no generally accepted gold standard. Examples of such practice variation include: on-pump versus off-pump CABG; conduct of cardiopulmonary bypass including temperature, hematocrit, flow rates, anticoagulation management, and the use of heparin-bonded circuits; hemostatic and antiinflammatory adjuncts such as aprotinin; single aortic cross clamp versus side clamp; and cardioplegia composition, temperature, and route of administration. For many of these areas, it would be difficult to select a performance benchmark that had broad support, and that could be implemented by most programs without substantially altering otherwise successful operative protocols. This could have the potential for producing significant unintended negative consequences.

We chose not to base our operative quality measure on such indicators that are often a matter of individual preference. However, there is one measure of CABG operative process quality—use of single or multiple IMA—whose importance has been universally recognized for well over two decades [47, 48]. There may be no other single surgical measure that so positively impacts the long-term survival of CABG patients, and it is for this reason that use of at least one IMA was chosen as the operative process measure. Consistent with the NQF definition, exclusion is permitted for repeat CABG procedures.

Risk-Adjusted Operative Mortality
This is the most familiar and widely used of all quality measurements [2], and it has broad acceptance as a valid measure among providers, regulators, and payers. We use the NQF operative mortality definition adapted from the STS NCD. It includes all deaths occurring in-hospital, regardless of timing, as well as all deaths occurring within 30 days of surgery regardless of venue, unless clearly unrelated to the operation.

Postoperative Risk-Adjusted Major Morbidity
Studies by Hannan [49] and Silber [50] and their colleagues suggest that complications and readmissions, which are delayed complications, may measure different aspects of care than does mortality. Furthermore, the same serious complication may result in death at one institution because of "failure to rescue," whereas superior care at another institution may result in patient salvage. For all these reasons, we chose to include a bundle of NQF morbidity measures as a separate outcomes domain in addition to operative mortality.

Patients, providers, and payers have a common goal after CABG surgery—discharge from the hospital alive, and without the occurrence of any serious postoperative complication. We selected five postoperative complications from the NQF cardiac surgery measure set (Table 2), all of which are derived from and defined by the STS NCD. These include stroke, renal insufficiency, deep sternal wound infection, reexploration for any cause, and prolonged intubation/ventilation. There are several NQF-approved exclusions for renal failure, stroke, and prolonged ventilation. Because absence of all such complications is the desired outcome, they are again considered as a bundle, in this case any or none. If even one major complication occurs, no credit is received for the bundle. A hierarchical risk model is developed using this dichotomous morbidity measure (1 = presence of any of the five major complications, 0 = no major complications) as the dependent variable, and an adjusted any or none morbidity rate is determined (see Part 2 and Technical Appendix).

Measures Considered but Not Included
Two structural measures (database participation and procedural volume) are included in the NQF cardiac surgery measure set but were not specifically chosen for inclusion in our quality score. As all our chosen measures will be derived from the STS NCD, participation in this database will be required for all programs that are rated, and thus this structural measure is included de facto. The other NQF-endorsed structural measure is procedural volume. Based on the studies cited previously, the volume-outcome relationship for CABG is weak. Furthermore, volume is most useful as a surrogate for outcome when accurate determination of the latter is impractical. In the case of CABG, we have highly credible, direct measures of both risk-adjusted mortality and morbidity. For all these reasons, we have chosen not to include volume as a structural measure.

Three NQF cardiac surgery process measures deal with the selection of appropriate preoperative antibiotics, their timing of administration, and duration of use. Because these are not currently included in the STS NCD, they were not selected for our quality measure set. In addition, compliance with these measures is already being closely monitored through multiple other quality measurement and improvement initiatives (eg, Surgical Infection Prevention Project, Surgical Care Improvement Project, American College of Surgeons National Surgical Quality Improvement Program, Joint Commission on Accreditation of Healthcare Organizations).

Finally, although the ACC/AHA endorses the use of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers for secondary prevention of atherosclerotic coronary disease [46], they are not NQF-endorsed and are therefore not included in our measure set.

	Conclusion

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
The QMTF selected a set of 11 individual quality measures within four domains of care. These are nationally recognized metrics that are also consistent with the guiding principles established by the STS QMTF. This measure set will be the foundation for a comprehensive, composite quality scoring methodology for cardiac surgery. The QMTF envisions this as a dynamic process that will undergo progressive refinements over time.

Part 2 of this report describes the statistical considerations for analyzing and combining these measures and for using the resulting composite scores to characterize provider performance [51].

	Footnotes

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 
Dr Shahian is the Quality Measurement Task Force Chair and Writing Group Leader.

	References

Top Executive Summary Introduction The Concept of Quality Cardiac Surgery Quality Principles for Measure Selection Specific Measure Selection Conclusion Footnotes References

 

1. Institute of Medicine Performance measurement: accelerating improvement. Washington, DC: The National Academies Press; 2006.
2. Shahian DM, Normand SL, Torchiana DF, et al. Cardiac surgery report cards: comprehensive review and statistical critique Ann Thorac Surg 2001;72:2155-2168.[Abstract/Free Full Text]
3. Shahian DM. Improving cardiac surgery quality—volume, outcome, process? JAMA 2004;291:246-248.[Free Full Text]
4. Shahian DM, Blackstone EH, Edwards FH, et al. Cardiac surgery risk models: a position article Ann Thorac Surg 2004;78:1868-1877.[Abstract/Free Full Text]
5. Birkmeyer JD, Dimick JB, Birkmeyer NJ. Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg 2004;198:626-632.[Medline]
6. Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size JAMA 2004;292:847-851.[Abstract/Free Full Text]
7. Birkmeyer JD, Siewers AE, Finlayson EVA, et al. Hospital volume and surgical mortality in the United States N Engl J Med 2002;346:1128-1137.[Abstract/Free Full Text]
8. Shahian DM, Normand SL. The volume-outcome relationship: from Luft to Leapfrog Ann Thorac Surg 2003;75:1048-1058.[Abstract/Free Full Text]
9. Peterson ED, Coombs LP, DeLong ER, Haan CK, Ferguson TB. Procedural volume as a marker of quality for CABG surgery JAMA 2004;291:195-201.[Abstract/Free Full Text]
10. Shroyer AL, Marshall G, Warner BA, et al. No continuous relationship between Veterans Affairs hospital coronary artery bypass grafting surgical volume and operative mortality Ann Thorac Surg 1996;61:17-20.[Abstract/Free Full Text]
11. Gibbons RJ, Smith S, Antman E. American College of Cardiology/American Heart Association clinical practice guidelines: Part I: where do they come from? Circulation 2003;107:2979-2986.[Free Full Text]
12. Werner RM, Asch DA. The unintended consequences of publicly reporting quality information JAMA 2005;293:1239-1244.[Abstract/Free Full Text]
13. Bradley EH, Herrin J, Elbel B, et al. Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality JAMA 2006;296:72-78.[Abstract/Free Full Text]
14. Cohen IB. Florence Nightingale Sci Am 1984;250:128-137.[Medline]
15. Iezzoni LI. Risk adjustment for measuring health care outcomes. Chicago, IL: Health Administration Press; 2003.
16. Spiegelhalter DJ. Surgical audit: statistical lessons from Nightingale and Codman J R Stat Soc (series A) 1999;162:45-58.
17. Mallon WJ. Ernest Amory Codman: the end result of a life in medicine. Philadelphia, PA: WB Saunders Company; 2000.
18. Passaro Jr E, Organ Jr CH. Ernest A. Codman: the improper Bostonian Bull Am Coll Surg 1999;84:16-22.[Medline]
19. Codman EA. A study in hospital efficiency. As demonstrated by the case report of the first two years of a private hospital. Oakbrook Terrace, IL: Joint Commission on Accreditation of Healthcare Organizations; 1995.
20. Edwards FH. Evolution of the Society of Thoracic Surgeons National Cardiac Surgery Database J Invasive Cardiol 1998;10:485-488.[Medline]
21. Berwick DM. Public performance reports and the will for change JAMA 2002;288:1523-1524.[Free Full Text]
22. Landon BE, Normand SL, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers JAMA 2003;290:1183-1189.[Abstract/Free Full Text]
23. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care—a survey of cardiovascular specialists N Engl J Med 1996;335:251-256.[Abstract/Free Full Text]
24. Burack JH, Impellizzeri P, Homel P, Cunningham Jr JN. Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons Ann Thorac Surg 1999;68:1195-1200.[Abstract/Free Full Text]
25. Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better?The effects of "report cards" on health care providers. J Polit Econ 2003;111:555-588.
26. Jones RH. In search of the optimal surgical mortality Circulation 1989;79:I132-I136.[Medline]
27. Edwards FH, Peterson ED, Coombs LP, et al. Prediction of operative mortality after valve replacement surgery J Am Coll Cardiol 2001;37:885-892.[Abstract/Free Full Text]
28. Nowicki ER, Birkmeyer NJ, Weintraub RW, et al. Multivariable prediction of in-hospital mortality associated with aortic and mitral valve surgery in Northern New England Ann Thorac Surg 2004;77:1966-1977.[Abstract/Free Full Text]
29. Shahian DM, Edwards FH, Peterson ED. Risk models for cardiac valve surgery ACC Curr J Rev 2005;14:52-56.
30. Institute of Medicine To err is human: building a safer health system. Washington, DC: National Academies Press; 1999.
31. Institute of Medicine Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001.
32. Torchiana DF, Meyer GS. Use of administrative data for clinical quality measurement J Thorac Cardiovasc Surg 2005;129:1223-1225.[Free Full Text]
33. Mack MJ, Herbert M, Prince S, Dewey TM, Magee MJ, Edgerton JR. Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes? J Thorac Cardiovasc Surg 2005;129:1309-1317.[Abstract/Free Full Text]
34. Welke KF, Ferguson Jr TB, Coombs LP, et al. Validity of the Society of Thoracic Surgeons National Adult Cardiac Surgery Database Ann Thorac Surg 2004;77:1137-1139.[Free Full Text]
35. Khanderia U, Townsend KA, Eagle K, Prager R. Statin initiation following coronary artery bypass grafting: outcome of a hospital discharge protocol Chest 2005;127:455-463.[Medline]
36. Daley J. Criteria by which to evaluate risk-adjusted outcomes programs in cardiac surgery Ann Thorac Surg 1994;58:1827-1835.[Abstract]
37. Nolan T, Berwick DM. All-or-none measurement raises the bar on performance JAMA 2006;295:1168-1170.[Free Full Text]
38. The Society of Thoracic Surgeons. STS National Database. Available at: www.sts.org/sections/stsnationaldatabase/datamanagers/adultcardiacdb. Accessed Sep 6, 2006.
39. National Quality Forum. National voluntary consensus standards for cardiac surgery. Available at: www.qualityforum.org/pdf/reports/cardiac.pdf. Accessed, Sep 6, 2006.
40. Denton TA, Fonarow GC, LaBresh KA, Trento A. Secondary prevention after coronary bypass: the American Heart Association "Get with the Guidelines" program Ann Thorac Surg 2003;75:758-760.[Free Full Text]
41. Ferraris VA, Ferraris SP, Moliterno DJ, et al. The Society of Thoracic Surgeons practice guideline series: aspirin and other antiplatelet agents during operative coronary revascularization(executive summary) Ann Thorac Surg 2005;79:1454-1461.[Free Full Text]
42. Ferguson Jr TB, Peterson ED, Coombs LP, et al. Use of continuous quality improvement to increase use of process measures in patients undergoing coronary artery bypass graft surgery: a randomized controlled trial JAMA 2003;290:49-56.[Abstract/Free Full Text]
43. Ferguson Jr TB, Coombs LP, Peterson ED. Preoperative beta-blocker use and mortality and morbidity following CABG surgery in North America JAMA 2002;287:2221-2227.[Abstract/Free Full Text]
44. Lazar HL. Role of statin therapy in the coronary bypass patient Ann Thorac Surg 2004;78:730-740.[Abstract/Free Full Text]
45. Okrainec K, Platt R, Pilote L, Eisenberg MJ. Cardiac medical therapy in patients after undergoing coronary artery bypass graft surgery: a review of randomized controlled trials J Am Coll Cardiol 2005;45:177-184.[Abstract/Free Full Text]
46. Smith Jr SC, Allen J, Blair SN, et al. AHA/ACC guidelines for secondary prevention for patients with coronary and other atherosclerotic vascular disease: 2006 Update: endorsed by the National Heart, Lung, and Blood Institute Circulation 2006;113:2363-2372.[Free Full Text]
47. Loop FD, Lytle BW, Cosgrove DM, et al. Influence of the internal-mammary-artery graft on 10-year survival and other cardiac events N Engl J Med 1986;314:1-6.[Abstract]
48. Lytle BW, Blackstone EH, Sabik JF, Houghtaling P, Loop FD, Cosgrove DM. The effect of bilateral internal thoracic artery grafting on survival during 20 postoperative years Ann Thorac Surg 2004;78:2005-2012.[Abstract/Free Full Text]
49. Hannan EL, Racz MJ, Walford G, et al. Predictors of readmission for complications of coronary artery bypass graft surgery JAMA 2003;290:773-780.[Abstract/Free Full Text]
50. Silber JH, Rosenbaum PR, Schwartz JS, Ross RN, Williams SV. Evaluation of the complication rate as a measure of quality of care in coronary artery bypass graft surgery JAMA 1995;274:317-323.[Abstract]
51. O’Brien SM, Shahian DM, DeLong ER, et al. Quality measurement in adult cardiac surgery: Part 2—Statistical considerations in composite measure scoring and provider rating Ann Thorac Surg 2007;83:S13-S26.[Medline]

This article has been cited by other articles:

				S. C. Stamou, S. L. Camp, R. M. Stiegel, M. K. Reames, E. Skipper, L. T. Watts, M. Nussbaum, F. Robicsek, and K. W. Lobdell Quality improvement program decreases mortality after cardiac surgery. J. Thorac. Cardiovasc. Surg., August 1, 2008; 136(2): 494 - 499.e8. [Abstract] [Full Text] [PDF]

				D. M. Shahian and S.-L. T. Normand Low-volume coronary artery bypass surgery: Measuring and optimizing performance. J. Thorac. Cardiovasc. Surg., June 1, 2008; 135(6): 1202 - 1209. [Full Text] [PDF]

				W. W.L. Li and B. A.J.M. de Mol Hospital volume as a quality standard in lung cancer surgery: an unfinished debate. Ann. Thorac. Surg., May 1, 2008; 85(5): 1840 - 1841. [Full Text] [PDF]

				A. Brunelli, G. Varela, P. Van Schil, M. Salati, N. Novoa, J. M. Hendriks, M. F. Jimenez, P. Lauwers, and on behalf of the ESTS Audit and Clinical Excellenc Multicentric analysis of performance after major lung resections by using the European Society Objective Score (ESOS) Eur. J. Cardiothorac. Surg., February 1, 2008; 33(2): 284 - 288. [Abstract] [Full Text] [PDF]

< This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): David M. Shahian Fred H. Edwards Victor A. Ferraris Constance K. Haan Jeffrey B. Rich Cynthia M. Shewan Eric D. Peterson