|
|
||||||||
Ann Thorac Surg 2000;69:680-691
© 2000 The Society of Thoracic Surgeons
a Committee to Establish a National Database in Cardiothoracic Surgery, The Society of Thoracic Surgeons, Chicago, Illinois, USA
Address reprint requests to Dr Ferguson, Department of Surgery, LSU School of Medicine, 1542 Tulane Ave, 7th Floor, New Orleans, LA 70112-2822
e-mail: tbruceferg732{at}pol.net
Presented at the Thirty-fifth Annual Meeting of The Society of Thoracic Surgeons, San Antonio, TX, Jan 2527, 1999.
| Abstract |
|---|
|
|
|---|
Methods and Results. This report explains the numerous changes in the NDB and its structure that have occurred over the past 2 years. It highlights the benefits of these changes, both to the individual member participants and to the STS overall. Additionally, the vision changes to the NDB and reporting structure are identified. The individuals who have participated in this effort since 1989 are acknowledged, and the STS owes an enormous debt of gratitude to each of them.
Conclusions. Because of their collective efforts, the goal to establish the STS NDB as a "gold standard" worldwide for process and outcomes analysis related to cardiothoracic surgery is becoming a reality.
| Introduction |
|---|
|
|
|---|
| National Cardiac Surgical Database, 1990 to 1997 |
|---|
|
|
|---|
|
|
|
|
|
|
|
| Limitations of the National Cardiac Database effort, 1997 |
|---|
|
|
|---|
Definition issues
From 1989 until 1998, the original definitions associated with the 512 data elements in the Summit DOS and Windows Vista software were formally revised one time [13]. In early 1997 the Definitions Subcommittee of the Database documented that many no longer reflected current clinical circumstances that were significant in the risk assessment analysis. Among these were unstable angina, myocardial infarction, chronic obstructive pulmonary disease (COPD), and elective, urgent, and emergent status at operation. In this revision process, the opportunity to achieve synergy with the evolving American College of Cardiology National Registry [14] as well as other cooperative efforts [15] ultimately was the catalyst for the restructuring of the entire STS NDB.
Access to information in the database
Two primary goals for the NDB effort have been (1) that it provide to the participant membership of the STS site-specific performance evaluation and comparison with national aggregate statistics, and (2) that aggregate analyzed data be available for scientific investigation. In 1997 neither of these goals was being met. The quality of the National Report had not kept pace with other concurrent database reports [1618].
Data entry and acquisition software
The sophistication of data entry software had failed to keep pace with hardware and software developments in the microcomputer arena during the years 1990 to 1997. For example, there were still 58 participant sites using DOS-based data entry software at the time of the 1997 harvest, despite the fact that there has been no DOS-based risk algorithm developed or software support available for the past 2 years. This software limitation resulted in a number of difficulties for the site data managers. These included the size of the data form (512 data elements per record), the lack of software-driven data quality checks, and the outdated/unclear/inaccurate definitions (eg, unstable angina, left main disease) discussed above. Finally, there was essentially no technical support for data managers until the first Annual Data Managers Meeting was convened in January 1997.
Data quality
Doctor Fred Grover, as incoming Chair of the Database Committee, instituted a program to address data quality issues for this voluntary database in 1996. Based upon outside evaluation of the quality issues related to the voluntary database, efforts were implemented to set data quality standards and to improve local site data quality [19, 20].
During this process, it became clear that the best opportunities to assure data quality and completeness were through: (1) education and technical support for the data managers, (2) incorporation of data quality and completeness checks into data entry software, and (3) local and regional efforts to improve data quality.
Developing regional activities
Although STS regional activities began with the Minnesota STS in 1994, there has been an explosion of interest in regional activities within the National Society over the 2 years. The National Database Committee recognized in 1997 that the database was poorly positioned to respond to the information needs of these regional efforts.
| Restructuring of the STS National Cardiovascular Database |
|---|
|
|
|---|
During the spring of 1997 the Definition Subcommittee worked closely with representatives of the American College of Cardiology (ACC) to revise the definitions for contiguous variables present in the two databases [16]. From the perspective of the STS, this included primarily those data elements that had been statistically significant in the univariate or multivariate risk analyses for CABG mortality in the 1990 to 1995 models [7].
With the new definitions as the impetus, the restructuring process resulted in the creation of Core and Extended Data Sets of the Database (Fig 8). The Core Set comprises a standard grouping of "fields" that are necessary for accurate representation of clinical practice, risk model development and analyses, and participation in national and regional database activities. This concept of a core set assumes that collection of all of these data fields (related to a procedure and its complications) is mandatory for all patients entered. The corollary to this is that the size of the Core Set must be reduced to a manageable number of variables (Table 1). The Extended Data Set includes other less critical but nevertheless important fields; the collection of these fields is strongly encouraged but is not mandatory for participation in national and regional analyses; the individual sites can select variables from the Extended Set to collect according to their individual practice patterns. Finally, creation of custom fields for information specific to a site is still possible.
|
|
A request for proposal (RFP) was offered by the Database Liaison Committee in the summer of 1998 for development of front-end (data entry, local site analysis) software. The STS software specifications for the data set (version 2.35) are the basis of the data entry component of the software packages. Eleven vendors responded and began to develop software for certification; eight vendors are undergoing certification, which has been outsourced to Tri-Analytics Inc. (Baltimore, MD), the firm that certified ACC Database software. The membership will thus have the opportunity to evaluate and choose between different software packages based upon the unique features offered by the vendors, while being assured that at a minimum the Core Data Set specifications will have been integrated successfully into the software package.
| Database warehousing and analysis |
|---|
|
|
|---|
The Duke Clinical Research Institute (DCRI) was selected as the new warehouse facility for the STS NDB from among seven RFP responses. A 13-member core team within the DCRI, with Eric Peterson, MD, MPH, as Principal Project Director, is responsible for management of all facets of the National Database warehouse and analysis efforts. DCRI has extensive experience in management of other large database projects, both private and government-sponsored.
Site contact personnel at DCRI are responsible for all communication between the Institute and participant sites within six geographic areas. It is expected that this will greatly enhance the level of warehouse and analysis support available to each site participating in the NDB.
| Software transition options for the STS NDB participants |
|---|
|
|
|---|
| Benefits of these changes for STS participant members |
|---|
|
|
|---|
Data input
The software vendors have great flexibility in the design of their front-end data acquisition packages. At a minimum, all packages will have incorporated the Core Data Set fields. Of the 217 total Core fields, only 114 fields are required for an elective CABG and 112 fields for a primary single valve (Fig 9). These fields must be completed, however, for a record to meet quality completeness checks and for uploading data in the biannual harvest.
|
Outcomes and process measures reports
Beginning in 2000, data will be harvested from participant sites by DCRI twice a year. Semiannual site-specific feedback, comparing site data benchmarked against local/regional and national data for all important data elements, will be published by DCRI (Fig 10). In addition, an executive national aggregate report will be produced by DCRI from the full-year harvest. This semiannual harvest process will shorten the time interval from data submission to output generation back to the participant sites to approximately 3 to 4 months, allowing for quality improvement implementation closer to the temporal point of care.
|
Data quality and completeness
Two goals for data quality have been achieved as a result of the changes made to the database structure. The first of these is the clarification of definitions for the data fields in the Core and Extended Data Sets. The second is the inclusion of Data Quality Checks as the forms are entered into the software, to highlight missing data, conflicting data elements, and require certain critical fields to be present before the data file can be uploaded for harvesting.
A third goal, local/regional data quality improvement, will be addressed as well by these changes. Data Quality Reports will be generated by DCRI for participant sites twice a year. These reports will highlight those areas of missing/erroneous data, possible misinterpretation of data element definitions, and other issues of data quality benchmarked as outlined above. In addition, education and support of the site data managers to facilitate and assure the most accurate and complete data entry will be provided. The STS Web resources will be substantively augmented to help in this process.
Data analysis and risk model development
The Data Analysis Subcommittee, under the co-chairmanship of Dr Fred Edwards and Dr A. Laurie W. Shroyer, has developed and implemented new models for adult valve (aortic valve replacement [AVR], mitral valve replacement [MVR]) and valve + CABG (AVR + CABG, MVR + CABG). These have been developed in conjunction with the DCRI statistical group, under the direction of Elizabeth R. DeLong, PhD. In addition, the CABG model (from 1996) has been revised to "bridge" the transition from old software to new software [21]. All five models have be incorporated into the newly certified STS software available in 1999. The Subcommittee and resources at the DCRI will assist in further model development for the adult, thoracic, and congenital databases.
Access to database information
To deal with the issue of access to the NDB information, a mechanism was developed to facilitate querying the database which would be applicable for member participants, for the DCRI personnel, and for third-party entities, either governmental organizations or commercial product developers within cardiovascular disease. This process is supervised by the Access and Publication Steering Subcommittee of the Database, chaired by Bruce Ferguson, which will review all requests for access to the NDB.
Minor access
This is an ad hoc, focused query to determine the merits of an idea, to obtain specific clinical information, or to generate a specific data output set from the database (Table 2). It is arbitrarily defined as less than 8 hours of analysis support from the DCRI to generate the results of the Minor Access request. Most often this will not involve complex data analysis including risk-adjusted outcomes data.
|
The STS has included in the contractual budget with the DCRI financial support for both Minor and Major Access requests. To equitably distribute this resource, all requests will be evaluated by the Access and Publication Steering Committee for scientific merit, required resources, source of the request, and intended use of the requested information. It is anticipated that the review process will take approximately 1 month from the time of submission to notification of the Principal Investigator of a decision regarding prioritization and funding. It is the intention of the Database Committee and the DCRI to fully support those access requests that are approved and prioritized to fully accomplish the goals of the project.
It is anticipated that 25 Minor Access and 14 Major Access projects can be funded in 2000. In addition, Major and Minor Access projects with appropriate scientific merit but which do not meet available funding criteria will be supported upon provision of outside funding.
STS member database participants are eligible for Funded and Nonfunded Major and Minor Access approaches. Members who are nonparticipants in the database as well as industry and government organizations are eligible for Nonfunded Minor and Major Access approaches.
Information about the Steering Committee Review Process, the Access format and process, as well as copies of the Minor Access and Major Access forms are available on the STS Database Web Page (http://www.sts.org/database/access.html). In addition to submission of the appropriate forms, principal investigators for projects that are approved for support by the STS and the DCRI will be asked to sign appropriate disclaimer forms.
National General Thoracic Database
The Thoracic Database has been revised completely under the direction of Dr David Harpole, Chair of the General Thoracic Subcommittee. The data sets for lung cancer, esophageal cancer, and mediastinal masses have been prepared for release. Plans are to develop a data dictionary and specifications for software and to offer an RFP for general thoracic database software development. Ultimately, risk-adjusted models will be developed for lung and esophageal cancer.
Congenital Database
Under the direction of Dr Constantine Mavroudis the Congenital Database Subcommittee has performed an enormous amount of work on a National Congenital Database. The first Congenital Report was published on data that had been collected previously. An International Conference on Nomenclature and Definitions was convened in Chicago in September 1998, and two subsequent conferences have continued work on these issues. The results of these conferences will be published in the Annals of Thoracic Surgery early in 2000. A limited "core" congenital set has been designed, and plans are to develop a data dictionary and specifications for software development.
Financial and Cost Subcommittee
One of the original goals of the NDB was to eventually link clinical information and outcomes with cost data. Doctor Richard Prager and Dr Peter Smith co-chair the Cost Subcommittee, with a goal to link the STS clinical database with cost analysis database efforts [22]. The goal of these efforts ultimately will be to predict the cost of delivered care based upon patient risk factors, just as the clinical outcomes of care can be predicted now.
International initiatives
Many of these efforts have developed to involve the European cardiothoracic surgical community. The European Cardiac Surgery Registry (ECSUR) principals, under the guidance of Prof Kenneth Taylor and Dr Richard Wyse, have been regular participants in the NDB efforts. ECSUR has incorporated the Core Set variables into their data set. In conjunction with Dr Bruce Keough from Great Britain and Dr Paul Sergeant from Belgium, a Joint STS-ECSUR Committee has developed the International Adult Cardiac Surgery Data Collection Form, which is an international data set for outcomes for cardiac surgical outcomes reporting.
| Database participation: requirements and costs |
|---|
|
|
|---|
Deliberately, the software and warehousing components of the NDB structure have been uncoupled to maximize flexibility for the participants and return control of the overall database direction to the STS. The participation fees for software and warehousing have been uncoupled as well.
Software
By the end of 1999, NDB participants will be able to choose from among a minimum of eight certified software products that will be available to replace the DOS, Vista, or Crescendo versions of the Summit software, or upgrade to new versions of other STS participant software (eg, Axis, Seattle Systems). Technical support for DOS, Vista, and Crescendo ended with the December 31, 1998, termination of participation agreements.
DCRI has worked with the Site Data Managers to convert 1998 data (collected on "old" software) to a format that can be harvested, using a data conversion tool developed by Stan Dziuban. This same process will be used for the 1999 harvest. Beginning January 1, 2000, data should be collected on the new (version 2.35) data forms until software packages have been selected and installed at each institution.
The harvesting of 1998 data was completed during the summer by DCRI, and the site-specific reports will be distributed by December 1, 1999. In the spring of 2000 the first semiannual harvest will occur, on "old format" data from 1999. Beginning with the fall 2000 harvest, all data will be in the new STS Core Data Set format.
Software packages at a range of pricing levels ($8,000 to $150,000) are available for this new front-end software. Annual licensing fees ($350/year in 2000) will be paid directly to the software vendor for the software and support services; as in the past, the STS will receive a royalty from the vendors to offset costs. In addition, any locally developed software package that meets the certification standards can be used for data entry.
Warehousing and analysis
An annual warehouse fee ($1,700 for Member Participant Site in 1999) will be paid to the STS by participants for the annual harvest, analysis, and site-specific report generation activities of the DCRI. This fee will offset the DCRI contract annual costs. This warehouse fee is analogous to that portion of the annual fee paid to Summit Medical Inc which was used for harvest and warehouse expenses. Under this old arrangement, the STS in essence underwrote the analysis costs incurred from the NDB. Although this worked well in the initial years of the NDB, more recently this became a case of "you get what you pay for" due to the lack of Summit Medical resources that were available to Drs Edwards and Shroyer. The quality of data generated by DCRI will be available for evaluation by prospective participants with the publication of the 1998 report in December 1999.
New database revenue sources
This revised structure creates two new sources of potential revenue for the STS from the database activity. The first is from Nonfunded Major or Minor Access requests, in which investigators/third party/industry/government entities desire to obtain information from the database. The second is from regional activities. It is anticipated that the quality of regional information customized for presentation by DCRI and supplied through the NDB mechanism will be superior to any other available data. As such, the distribution of these data to third-party and government organizations can become an additional source of revenue to offset the Participant and Society costs associated with the NDB. However, cost only becomes an issue if and when the quality of the information is no longer in question.
Cost strategy
It is the opinion of the Database and Liaison Committees that in this effort the physician provider is linked in most cases with the hospital or hospitals at which he or she practices. The new Participation Agreements were constructed to allow the physician(s) to cooperatively partner with the hospital(s) in the acquisition of these software products. This site-specific database information is most beneficial to the physician/provider consortium (the physician and the hospital). It therefore seems most reasonable that the hospitals should partner with the physician groups to build these database costs into the operating expenses for the Cardiovascular (Surgical) Service Line or its equivalent at that hospital or hospital system, including software and hardware acquisition costs and data personnel costs. It is recognized, however, that there may be some potential loss of autonomy by the surgeons in these circumstances.
| Regional data reporting within the NDB |
|---|
|
|
|---|
There is a fundamentally important distinction between these voluntary efforts and the mandatory collection and reporting systems that have been implemented in New York [27] and Pennsylvania [28], and are being implemented in New Jersey and "voluntarily" in California. The Minnesota and NNE efforts involved full participation in voluntary databases on the part of providers, and not externally mandated and implemented data collection and analysis.
Support for publishing information and forming quality improvement consortia to improve outcomes in cardiac surgery practice is well established [29]. Although the importance of the connection between reporting, quality evaluation, and outcomes has been questioned [30], it is no longer valid to question whether outcomes and process measures data should be collected and analyzed [31]. Three fundamental issues [32] remain unsettled: (1) who should collect these data, (2) what data should be collected and how should it be analyzed, and (3) how should the information be used to improve results? It is the reality (threat or opportunity) of these three issues that has played a major role in stimulating regional activity within the STS. Newcomer [33] has termed the recognition of this circumstance "accountable autonomy." Regaining the autonomy that physicians as providers of care lost to third-party and governmental agencies will require (1) performance measurement and (2) acceptance of the responsibility for improvement in the process of care. Meeting these requirements results in two profound benefits: (1) the quality of care improves for the individual patient under a providers supervision, and (2) the need for outsiders to manage medicine evaporates.
If the NDB is to achieve its full potential then it is incumbent upon the STS, on behalf of its membership, to establish the Database as the de facto standard for outcomes reporting for cardiovascular surgical care [34]. In this way, data that are collected as part of participation in the NDB can be used to supply regional groups with analyzed data of sufficient quality in terms of accuracy, completeness, analysis, and presentation such that it can be passed on directly to outside organizations that request or demand clinical outcomes data. Only through this mechanism can the providers exert some influence over who collects the data and how it is used [32].
However, there are four hurdles to overcome with this approach:
As mentioned above, the goals of the Database Committee and Dr Kit Aroms Regional Subcommittee on behalf of the STS and the goals of the emerging regional activities are mutually synergistic. The NDB can now supply the highest quality, lowest cost information through the National Structure to the regional groups. Conversely, this regional activity provides a mechanism to assure and improve the data quality of the Database, including inclusion of all cases and deaths in the recorded data. Review and implementation of data quality issues are most effectively handled at the local and regional level rather than the national level. Information, policies, and procedures successful in one region can be efficiently shared with other regions.
Finally, centralization and organization of the regional database activities through the national effort permit the Government Affairs representatives to hold up the NDB as the de facto standard in their negotiations with federal government health officials and agencies.
| Comment |
|---|
|
|
|---|
| Appendix |
|---|
|
|
|---|
|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
A. Ernst, M. Simoff, D. Ost, Y. Goldman, and F. J. F. Herth Prospective Risk-Adjusted Morbidity and Mortality Outcome Analysis After Therapeutic Bronchoscopic Procedures: Results of a Multi-institutional Outcomes Database Chest, September 1, 2008; 134(3): 514 - 519. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. O. Jacobson, M. N. Neuss, K. K. McNiff, P. Kadlubek, L. R. Thacker II, F. Song, P. D. Eisenberg, and J. V. Simone Improvement in Oncology Practice Performance Through Voluntary Participation in the Quality Oncology Practice Initiative J. Clin. Oncol., April 10, 2008; 26(11): 1893 - 1898. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. S. Gammie, M. Haddad, S. Milford-Beland, K. F. Welke, T. B. Ferguson Jr, S. M. O'Brien, B. P. Griffith, and E. D. Peterson Atrial Fibrillation Correction Surgery: Lessons From The Society of Thoracic Surgeons National Cardiac Database Ann. Thorac. Surg., March 1, 2008; 85(3): 909 - 914. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Gongora and T. M. Sundt III Myocardial Revascularization with Cardiopulmonary Bypass Card. Surg. Adult, January 1, 2008; 3(2008): 599 - 632. [Full Text] |
||||