| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Ann Thorac Surg 2004;77:1137-1139
© 2004 The Society of Thoracic Surgeons
a Division of Cardiothoracic Surgery, University of Iowa Hospitals and Clinics, Iowa City, Iowa, USA
b Louisiana State University Health Sciences Center, New Orleans, Louisiana, USA
c Outcome Research and Assessment Group, Duke Clinical Research Institute, Durham, North Carolina, USA
d Iowa Foundation for Medical Care, West Des Moines, Iowa, USA
* Address reprint requests to Dr Welke, Division of Cardiothoracic Surgery, University of Iowa Hospitals and Clinics, 200 Hawkins Dr, Iowa City, IA 52242, USA.
e-mail: karl-welke{at}uiowa.edu
Abbreviations: CABG • Coronary artery bypass grafting • IASTS • Iowa Society of Thoracic Surgeons • IFMC • Iowa Foundation for Medical Care • MEDPAR • Medicare Provider and Analysis Review • NCD • National Adult Cardiac Surgery Database • STS • The Society of Thoracic Surgeons
In 1989, The Society of Thoracic Surgeons (STS) created a national voluntary cardiac surgery database as a means of supporting national quality improvement efforts [1]. The database has grown to become the largest clinical database of its kind, and includes 638 hospitals and clinical information from 2,164,079 surgical procedures. This provider-supported database allows participants to benchmark their risk-adjusted results against regional and national standards. In addition, it provides data for research projects that can improve the process of patient care and outcomes. It has also been used to support quality improvement efforts. Each of these uses, however, is dependent on an accurate and complete clinical database. The purpose of this article is to provide the STS membership with a description of the ongoing efforts to assure validity of the National Adult Cardiac Surgery Database (NCD). In particular, we will highlight examples of data quality assurance mechanisms occurring at three levels: internally, regionally, and nationally.
Internal data quality improvement efforts
The site data managers play a crucial role in ensuring the quality of the data being submitted to the NCD. To support these key individuals, the STS has initiated annual site data manager meetings. These well-attended meetings generate in-depth discussions and suggestions about how to best collect the data, the application of clinical data definitions, and other topics related to the overall quality of the database.
In addition, several steps have been added to the data collection process of the NCD with the intent of improving data quality. First, the STS has worked extensively with database vendors to develop point of entry data quality controls. For instance, if a patient is identified as having died during the original hospitalization and the field "date of death" is left blank, the software requires that the expected date of death be entered. These checks assure internal data consistencies within a given procedural record. Second, at the point of file submission to the Database Warehouse and Analysis Center at the Duke Clinical Research Institute, sites receive a detailed automated data quality report that warns site data managers about potential problems with various characteristics of the data file including records with inconsistent data, missing data values, and outlier or extreme data values. This proactive system allows sites the option of resubmitting their data as many times as is necessary within the harvest window to correct initial data issues. Experience has shown that a majority of STS sites take advantage of this option and submit their data multiple times per harvest. Official final sign-off from the data managers at each site on data to be included in analyses for that harvest is an additional safeguard to ensure that the appropriate data are entering the NCD. At the Duke Clinical Research Institute, the site data go through a third round of data quality control checks to assure completeness and consistency before being integrated into the NCD. In addition, after each harvest a detailed letter summarizing key data quality concerns for a particular site is sent from STS headquarters to the primary surgeon at each site
As a result of these efforts, the quality of the data in the NCD has increased remarkably [2]. For example, in 2002 the average percent of missing values on the 28 clinical variables used for mortality risk modeling was 1.6%. Of these, only two variables were missing more than 5% of the time: New York Heart Association functional class (6.9%) and ejection fraction (8.9%)
Regional validation efforts in Iowa
The STS data quality is also assisted by many regional collaborative quality improvement efforts. One such example is the collaboration between the Iowa Society of Thoracic Surgeons (IASTS), and the Iowa Foundation for Medical Care (IFMC). The IASTS is a voluntary, not-for-profit organization whose purpose is to bring together the thoracic surgeons of Iowa to engage in activities of mutual benefit including continuous quality improvement activities aimed at improving outcomes for patients who have had cardiac surgery. Currently, all sites performing cardiothoracic surgery in Iowa submit data to the Society of Thoracic Surgeons NCD. The IFMC is a private, nonprofit organization composed of health care professionals and is the Centers for Medicare and Medicaid Services Quality Improvement Organization for Iowa. In March 2000, the IASTS and the IFMC entered into an agreement to work together in support of the Society's quality improvement goals and as part of an Agency for Healthcare Research and Quality sponsored continuous quality improvement grant to the STS.
As a part of this effort, the IFMC performed a series of independent external data quality audits in the spring of 2001. Initially, five randomly selected isolated records of patients who had undergone coronary artery bypass grafting (CABG) were audited from each of the 12 Iowa sites. These audits focused on CABG operative risk of mortality data elements. After a masked review, a comparison was made between data collected by the IFMC staff and data submitted by the hospital staff. The overall agreement rate for the audited data variables was 95.8%, ranging from 80% to 100% for individual items.
After initial review by the IASTS, a second audit process was repeated by the IFMC in the spring of 2002. This audit also assessed coding for postoperative atrial fibrillation owing to the wide range of occurrence noted in the state. Ten randomly selected records were reviewed at all sites. The overall agreement rate for the audited data variables was again high at 96.2%, with postoperative atrial fibrillation at 96.4%. A third round of audits are planned for 2003 for data elements that fell below a 95% agreement rate in 2001 or 2002.
National comparison of the NCD to Medicare part A data files
As a final level of data quality assurance, the STS has joined with the University of Iowa to compare the NCD to Medicare Provider and Analysis Review (MEDPAR) Part A public use data files. This effort will allow validation of the yearly CABG volumes of cases submitted to NCD with those claims submitted to Medicare for patients aged 65 years or older. Additionally, this will allow comparison of unadjusted mortality rates between the two data sets. In the first round of analyses, all Medicare patients 65 years of age and older who underwent CABG in US hospitals from 1994 to 1999 were identified in the MEDPAR Part A public use data files, obtained from the Center for Medicare and Medicaid Services. Only patients covered by fee-for-service arrangements are included in MEDPAR files (excluding up to 10% of Medicare patients enrolled in health maintenance organizations during this time period). Isolated CABG cases were identified by the appropriate diagnosis-related group codes (106, 107).
A comparable sample of patients was created from the NCD by identifying all patients 65 years of age or older who underwent isolated CABG from 1994 to 1999.
Hospitals common to the MEDPAR and NCD data sets were identified by matching on hospital name and city (n = 519 hospitals).
Findings from this comparison are summarized in Table 1. The volume of patients was consistently slightly higher in the NCD. In-hospital mortality, while following the same trend toward lower mortality over time, was consistently higher in the NCD. The lesser number of patients in the MEDPAR data may be explained in part by the omission of patients enrolled in health maintenance organizations from Medicare files. In addition, the identification of CABG patients by diagnosis-related group code in the MEDPAR data may have resulted in the exclusion of patients who underwent CABG, but were assigned a higher diagnosis-related group code owing to another procedure. As this latter group is likely to represent higher risk patients, this may in part explain the higher mortality rate in the NCD.
|
Comment
The validity of the NCD is crucial if it is to continue to serve as a benchmark for participants and a resource for outcomes research and quality improvement efforts. The STS has made and continues to make efforts to assure the validity of the NCD. Prospectively defined and collected clinical data are more detailed and accurate than administrative data, but both can suffer from the question of validity. The STS validation initiatives have been directed at maximizing the benefits of a clinical data set: collection of data by clinical personnel who should have a better understanding of the data than administrative personnel, more accurate recording of data of interest in cardiac surgery because of a database specifically designed for that purpose, and standardization of definitions. While efforts to confirm the validity of the NCD are in progress, work done to date suggests that the NCD is accurate.
The goal of the STS is not just to produce a high quality clinical data set, but also to improve the process and outcomes of cardiac surgical care. Unlike administrative data, clinical process and outcomes data derived from and immediately relevant to surgeons' patients can be distributed in a timely manner to support quality improvement efforts. Indeed, the STS has just completed the first successful national randomized trial of continuous quality improvement in medicine, built on the backbone of the STS NCD [5]. In addition, clinical data sets can be modified to include and exclude data elements to support specific areas of interest. The NCD has made possible numerous research projects and quality improvement initiatives. Continual efforts to assure validity of the STS NCD make such work possible.
References
This article has been cited by other articles:
|
|
 |
|
  N. Motomura, H. Miyata, H. Tsukihara, M. Okada, S. Takamoto, and Japan Cardiovascular Surgery Database Organization First Report on 30-day and Operative Mortality in Risk Model of Isolated Coronary Artery Bypass Grafting in Japan. Ann. Thorac. Surg., December 1, 2008; 86(6): 1866 - 1872. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. Gongora and T. M. Sundt III Myocardial Revascularization with Cardiopulmonary Bypass Card. Surg. Adult, January 1, 2008; 3(2008): 599 - 632. [Full Text]
|
|
|
|
 |
|
 S. M. O'Brien, E. R. DeLong, R. S. Dokholyan, F. H. Edwards, and E. D. Peterson Exploring the Behavior of Hospital Composite Performance Measures: An Example From Coronary Artery Bypass Surgery Circulation, December 18, 2007; 116(25): 2969 - 2975. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. F. Welke, E. D. Peterson, M. S. Vaughan-Sarrazin, S. M. O'Brien, G. E. Rosenthal, G. J. Shook, R. S. Dokholyan, C. K. Haan, and T. B. Ferguson Jr Comparison of Cardiac Surgery Volumes and Mortality Rates Between The Society of Thoracic Surgeons and Medicare Databases From 1993 Through 2001 Ann. Thorac. Surg., November 1, 2007; 84(5): 1538 - 1546. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. F. Hernandez, J. D. Grab, J. S. Gammie, S. M. O'Brien, B. G. Hammill, J. G. Rogers, M. T. Camacho, M. K. Dullum, T. B. Ferguson, and E. D. Peterson A Decade of Short-Term Outcomes in Post Cardiac Surgery Ventricular Assist Device Implantation: Data From the Society of Thoracic Surgeons' National Cardiac Database Circulation, August 7, 2007; 116(6): 606 - 612. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. K. Haan, S. Milford-Beland, S. O'Brien, D. Mark, M. Dullum, T. B. Ferguson, and E. D. Peterson Impact of Residency Status on Perfusion Times and Outcomes for Coronary Artery Bypass Graft Surgery Ann. Thorac. Surg., June 1, 2007; 83(6): 2103 - 2110. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. R. Bridges, S. M. O'Brien, J. C. Cleveland, E. B. Savage, J. S. Gammie, F. H. Edwards, E. D. Peterson, and F. L. Grover Association between indices of prosthesis internal orifice size and operative mortality after isolated aortic valve replacement J. Thorac. Cardiovasc. Surg., April 1, 2007; 133(4): 1012 - 1021. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian, F. H. Edwards, V. A. Ferraris, C. K. Haan, J. B. Rich, S.-L. T. Normand, E. R. DeLong, S. M. O'Brien, C. M. Shewan, R. S. Dokholyan, et al. Quality Measurement in Adult Cardiac Surgery: Part 1--Conceptual Framework and Measure Selection Ann. Thorac. Surg., April 1, 2007; 83(4_Supplement): S3 - S12. [Full Text] [PDF]
|
|
|
|
|
|
E. B. Savage, J. D. Grab, S. M. O'Brien, A. Ali, E. J. Okum, R. A. Perez-Tamayo, D. S. Eiferman, E. D. Peterson, F. H. Edwards, and R. S.D. Higgins Use of Both Internal Thoracic Arteries in Diabetic Patients Increases Deep Sternal Wound Infection Ann. Thorac. Surg., March 1, 2007; 83(3): 1002 - 1006. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. J. DiSesa, S. M. O'Brien, K. F. Welke, S. M. Beland, C. K. Haan, M. S. Vaughan-Sarrazin, and E. D. Peterson Contemporary Impact of State Certificate-of-Need Regulations for Cardiac Surgery: An Analysis Using the Society of Thoracic Surgeons' National Cardiac Surgery Database Circulation, November 14, 2006; 114(20): 2122 - 2129. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. K. Haan, S. O'Brien, F. H. Edwards, E. D. Peterson, and T. B. Ferguson Trends in emergency coronary artery bypass grafting after percutaneous coronary intervention, 1994-2003. Ann. Thorac. Surg., May 1, 2006; 81(5): 1658 - 1665. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
W. A. Cooper, S. M. O'Brien, V. H. Thourani, R. A. Guyton, C. R. Bridges, L. A. Szczech, R. Petersen, and E. D. Peterson Impact of Renal Dysfunction on Outcomes of Coronary Artery Bypass Surgery: Results From the Society of Thoracic Surgeons National Adult Cardiac Database Circulation, February 28, 2006; 113(8): 1063 - 1070. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. G. Fowler Jr, S. M. O'Brien, L. H. Muhlbaier, G. R. Corey, T. B. Ferguson, and E. D. Peterson Clinical Predictors of Major Infections After Cardiac Surgery Circulation, August 30, 2005; 112(9_suppl): I-358 - I-365. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. K. Young, D. A. Foster, S. T. Heller, D. J. Ballard, F. H. Edwards, K. F. Welke, S. Levitsky, P. Cram, G. E. Rosenthal, and M. S. Vaughan-Sarrazin Cardiac revascularization in specialty and general hospitals. N. Engl. J. Med., June 30, 2005; 352(26): 2754 - 2756. [Full Text] [PDF]
|
|
|
|
 |
|
 R. H. Jones The Year in Cardiovascular Surgery J. Am. Coll. Cardiol., May 3, 2005; 45(9): 1517 - 1528. [Full Text] [PDF]
|
|
|
|
|
|
N. E. Taylor, S. O'Brien, F. H. Edwards, E. D. Peterson, and C. R. Bridges Relationship Between Race and Mortality and Morbidity After Valve Replacement Surgery Circulation, March 15, 2005; 111(10): 1305 - 1312. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. L. Mehta Acute Renal Failure and Cardiac Surgery: Marching in Place or Moving Ahead? J. Am. Soc. Nephrol., January 1, 2005; 16(1): 12 - 14. [Full Text] [PDF]
|
|
|
|
|
|
C. K. Haan, C. I. Cabral, D. A. Conetta, L. P. Coombs, and F. H. Edwards Selecting patients with mitral regurgitation and left ventricular dysfunction for isolated mitral valve surgery Ann. Thorac. Surg., September 1, 2004; 78(3): 820 - 825. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
