Ann Thorac Surg 2003;75:1128-1131
© 2003 The Society of Thoracic Surgeons
Original article: general thoracic
Effect of a cardiac-specific didactic course on Thoracic Surgery In-Training Examination performance
Marc R. Moon, MDa*,
Ralph J. Damiano, Jr, MDa,
G. Alexander Patterson, MDa,
William A. Gay, Jr, MDa,
Joel D. Cooper, MDa
a Division of Cardiothoracic Surgery, Washington University School of Medicine, St. Louis, Missouri, USA
Accepted for publication October 24, 2002.
* Address reprint requests to Dr Moon, Division of Cardiothoracic Surgery, Washington University School of Medicine, 3108 Queeny Tower, #1 Barnes-Jewish Plaza, St. Louis, MO 63110-1013, USA
e-mail: moonm{at}msnotes.wustl.edu
 |
Abstract
|
|---|
BACKGROUND: The purpose of this study was to determine which factors influenced performance on the Thoracic Surgery In-Training Examination (TSITE) and whether the addition of a cardiac-specific didactic study course improved scores.
METHODS: Between 1989 and 2002, 59 TSITE scores (overall, cardiac [C-TSITE], and thoracic [T-TSITE]) were collected from 33 residents (23 cardiac track, 10 thoracic). Factors assessed with univariate and multivariate analysis included calendar year, year of training (31 year I, 28 year II), standardized test-taking history (average National Board and American Board of Surgery in-training scores), subjective faculty assessment of cardiothoracic knowledge, months on cardiac versus thoracic service, clinical performance, and participation in a cardiac-specific didactic series with faculty lectures and board question reviews (12 residents).
RESULTS: Cardiac-track residents had higher C-TSITE percentile scores (53% ± 27% versus 38% ± 27%, p < 0.05), whereas thoracic-track residents had higher T-TSITE scores (70% ± 24% versus 51% ± 25%, p < 0.01). Multivariate analysis identified 3 factors associated with higher overall TSITE scores: standardized test-taking history (p < 0.001), subjective faculty assessment of knowledge (p < 0.001), and year of training (p < 0.007). Inclusion in the cardiac-specific didactic series did not affect C-TSITE scores (50% ± 30% versus 48% ± 27%, p > 0.82) or overall TSITE scores (p > 0.23).
CONCLUSIONS: Standardized test-taking history and subjective faculty assessment of knowledge were associated with higher TSITE scores, but implementation of a cardiac-specific didactic series had no influence. These findings suggest that independent study and reading may be the best way to improve TSITE scores.
|
Introduction
|
|---|
In 1994, the Thoracic Surgery Directors Association (TSDA) published the Comprehensive Thoracic Surgery Curriculum as a study guide for residents [1]; however, TSDA leadership quickly recognized that the printed outline was only the beginning of the process [2]. In May 1998, the TSDA established the Curriculum Implementation Committee, responsible for developing the optimal methodology whereby the comprehensive curriculum could be taught to residents [2, 3]. In an attempt to improve the educational milieu of our thoracic surgery residents, we performed an internal review of our program and found that performance on the Thoracic Surgery In-Training Examination (TSITE) was consistently lower on cardiac-specific topics than on thoracic-specific topics. Despite an extensive conference schedule on the general thoracic surgery service, we found that residents had limited formal didactic conferences dedicated specifically to adult and congenital cardiac surgery. Therefore, we implemented a faculty-driven didacticprogram that encompassed the major cardiac topics outlined in the TSDA comprehensive curriculum. The purpose of the current investigation was to determine which factors most influenced performance on the TSITE and whether the addition of a cardiac-specific didactic study course improved scores.
|
Material and methods
|
|---|
Between 1989 and 2002, data were collected from 33 cardiothoracic surgery residents at Washington University Medical Center. There were 23 cardiac-track residents and 10 thoracic-track residents. The cardiothoracic training program lasts 2 years, with 3 residents (2 cardiac track, 1 thoracic track) in each year of training. The distribution of time on the clinical services is equal during year I (6 months adult cardiac, 6 months adult thoracic) but differs during year II for the cardiac- and thoracic-track residents. During year II, cardiac-track residents spend 8 months on adult cardiac and 4 months on congenital cardiac; thoracic-track residents spend 6 months on adult thoracic, 2 months on adult cardiac, and 4 months on congenital cardiac. During the 14-year period, 59 TSITE scores were collected from 31 year-I and 28 year-II trainees. Percentile rankings were recorded including overall TSITE, cardiac-specific (C-TSITE), and thoracic-specific (T-TSITE) scores.
For each resident, an objective assessment of their base line underlying intelligence (standardized test-taking history) was obtained by averaging their scores on all parts of the United States Medical Licensing Examination (USMLE) and the American Board of Surgery In-Training Examination (ABSITE) for the 3 years immediately preceding the resident’s application to the cardiothoracic surgery training program. Subjective faculty evaluations of resident performance were obtained biannually using a 13-question survey with a 5-point grading scale. The survey included a subjective faculty assessment of cardiothoracic knowledge and an overall assessment of clinical performance. Individual scores were converted to a standard score based on the scores obtained for all residents during the period of study for statistical comparison.
Resident educational activities
During the entire study period, all residents were required to attend weekly grand rounds (formal faculty or resident-prepared lectures), and while on the thoracic service, residents attended routine case review and teaching conferences (two to three per week). In an attempt to strengthen formal educational activities specific to cardiac surgery, starting in July 2000, all residents were required to participate in a cardiac-specific didactic study course, regardless of their clinical rotation (9 residents, 12 TSITE scores). The didactic series consisted of bimonthly faculty lectures with a 5-question board review quiz. All major adult and congenital cardiac topics were included during the 12-month course as outlined in the TSDA comprehensive curriculum, and the course was repeated yearly.
Data analysis
Continuous data are reported as mean ± 1 standard deviation (SD) and were compared between groups using Student’s t test. Univariate linear regression analysis and multivariate stepwise regression analysis were used to determine the factors that were significant, independent predictors of higher C-TSITE, T-TSITE, and overall TSITE scores (SigmaStat 2.03, SPSS Inc, Chicago, IL). Ten variables were analyzed: calendar year, year of training, cardiac versus thoracic track, standardized test-taking history, subjective faculty assessment of cardiothoracic knowledge, months on cardiac service, months on thoracic service, ratio of time on cardiac-to-thoracic services, subjective faculty assessment of clinical performance, and participation in the cardiac-specific didactic series.
|
Results
|
|---|
Resident characteristics
Average USMLE and ABSITE scores (standardized test-taking history) were 570 ± 89 for the entire group. The difference in prior standardized test scores between thoracic- and cardiac-track residents (603 ± 74 versus 557 ± 92) did not reach statistical significance (p > 0.11), but the subjective faculty assessment of cardiothoracic knowledge was higher for thoracic than cardiac residents (546 ± 53 versus 468 ± 112, p < 0.008). The faculty assessment of clinical performance was not significantly different between thoracic and cardiac residents (506 ± 67 versus 488 ± 119, p > 0.56).
Thoracic surgery in-training examination scores
Average TSITE percentile scores were 53% ± 27% overall, 48% ± 28% C-TSITE, and 57% ± 26% T-TSITE. Year-II residents had higher overall TSITE (60% ± 25% versus 47% ± 27%, p < 0.06) and C-TSITE (57% ± 28% versus 40% ± 25%, p < 0.05) scores than year-I residents, but T-TSITE scores were not significantly different (60% ± 23% year II versus 54% ± 28% year I, p > 0.43). Cardiac-track residents had higher C-TSITE percentile scores (53% ± 27% versus 38% ± 27%, p < 0.05), and thoracic-track residents had higher T-TSITE scores (70% ± 24% versus 51% ± 25%, p < 0.01). Overall scores were similar between cardiac- and thoracic-track residents (53% ± 28% versus 54% ± 26%, p > 0.89). Calendar year did not correlate with TSITE scores (p > 0.29).
Standardized test-taking history was associated with higher C-TSITE (r2 = 0.25, p < 0.001), T-TSITE (r2 = 0.29, p < 0.001), and overall TSITE (r2 = 0.36, p < 0.001) scores (Fig 1). Subjective faculty assessment of resident knowledge was also associated with higher C-TSITE (r2 = 0.11, p < 0.02), T-TSITE (r2 = 0.30, p < 0.001), and overall TSITE (r2 = 0.25, p < 0.001) scores (Fig 2).
Subjective faculty evaluation of clinical performance correlated with T-TSITE (r2 = 0.10, p < 0.03) and overall TSITE (r2 = 0.09, p < 0.04) scores, but did not correlate with C-TSITE scores (r2 = 0.04, p > 0.15). Number of months on the cardiac service did not correlate with C-TSITE scores (r2 = 0.06, p > 0.09), nor did the ratio of cardiac-to-thoracic service time (r2 = 0.01, p > 0.51). Similarly, T-TSITE scores did not correlate with the number of months on the thoracic service (r2 = 0.01, p > 0.60) or the ratio of cardiac-to-thoracic service time (r2 = 0.00, p > 0.90). Univariate analysis revealed that participation in the cardiac-specific didactic series did not improve C-TSITE scores (50% ± 30% didactic versus 48% ± 27% no didactic, p > 0.82) or overall TSITE scores (62% ± 22% versus 51% ± 28%, p > 0.23).
View larger version (20K):
[in this window]
[in a new window]
|
Fig 1. Correlation between standardized test-taking history (average scores on the United States Medical Licensing Examination and the American Board of Surgery In-Training Examination) and Thoracic Surgery In-Training Examination (TSITE) score. r = 0.50, p < 0.001.
|
|
View larger version (20K):
[in this window]
[in a new window]
|
Fig 2. Correlation between subjective faculty assessment of residents’ cardiothoracic knowledge (see Material and Methods for details on subjective faculty scoring) and Thoracic Surgery In-Training Examination (TSITE) score. r = 0.60, p < 0.001.
|
|
Multivariate regression analysis identified four factors that predicted higher C-TSITE scores: standardized test-taking history (p < 0.001), faculty knowledge assessment (p < 0.02), year of training (p < 0.004), and cardiac track (p < 0.001). Three factors predicted higher T-TSITE scores: standardized test-taking history (p < 0.009), faculty knowledge assessment (p < 0.002), and thoracic track (p < 0.03). Three factors predicted higher overall TSITE scores: standardized test-taking history (p < 0.001), faculty knowledge assessment (p < 0.001), and year of training (p < 0.007). Multivariate analysis revealed that participation in the cardiac-specific didactic series did not improve C-TSITE (p > 0.40) or overall TSITE (p > 0.61) scores.
|
Comment
|
|---|
Previous investigators have examined the effect of conference attendance on ABSITE scores with varying results [4–7]. Shetler [4] found that conference attendance was weakly related to ABSITE scores (r2 = 0.16, p = 0.08), and although Godellas and Huang [5] reported a significant correlation between conference attendance and ABSITE results (p < 0.001), their findings were difficult to interpret. During the period in which conference attendance increased, Godellas also instituted a structured reading program and a retention policy based on ABSITE scores that surely affected resident effort on the examination [5]. In the current study, the addition of a cardiac-specific didactic lecture series did not affect cardiac or overall TSITE scores. Instead, a history of exemplary performance on standardized tests (USMLE, ABSITE), likely representing a superior base line fund of knowledge and underlying intelligence, was the most important factor affecting TSITE scores. Itani and associates [6] similarly found that ABSITE performance was most dependent on the individual resident’s motivation and self-study efforts, concluding that traditional conferences, even if popular among residents, had no influence on ABSITE results. These data do not, however, suggest that didactic sessions or educational conferences should be abandoned, but rather that formal teaching conferences are not the best way to improve TSITE scores. Thus, other means should be undertaken if the goal is to increase the factual knowledge base of the average thoracic surgery resident.
More significant improvements in ABSITE and TSITE scores have been associated with self-study reading programs than with diligent conference attendance. Bull and associates [7] noted a threefold improvement in TSITE performance after a change in their educational curriculum from a faculty-driven didactic lecture series to a resident-driven self-study program. Residents who participated in the didactic series increased percentile scores from year I to year II by 11% ± 12% compared with a 31% ± 21% increment for residents who participated in the self-study program (p < 0.05). Hirvela and Becker [8] reported similar findings after instituting a structured self-study program with weekly reading assignments and discussion sessions. They found that ABSITE scores correlated significantly with reading effort (r2 = 0.62, p < 0.001) but not with attendance at the discussion sessions (r2 = 0.10, p > 0.16). In their study, 90% of residents who completed more than one half of the assigned readings had an incremental change in ABSITE scores during successive years that was greater than the national average. In contrast, only 20% of residents who completed less than one half of the readings had an incremental change that was greater than predicted. Therefore, the development of a structured reading and self-study course based on the TSDA comprehensive curriculum would likely be beneficial, as long as the residents actually completed the program. Residents must take an active role in the educational process; independent study is a requisite to the development of a satisfactory database of knowledge in thoracic surgery.
Teaching conferences and didactic sessions generally address clinical management issues that may be more applicable to practical oral examinations than standardized multiple-choice tests. For oral examinations, residents not only need an adequate factual knowledge base, but must possess the clinical acumen that comes from preoperative, intraoperative, and postoperative experience and focused clinical teachings. Without knowledge of the results of the current study, our residents acknowledged that although they did not believe that the didactic program improved their TSITE scores, they found the lectures valuable from a practical standpoint. The small didactic sessions offered them the opportunity to ask questions that may not have been appropriate, or that otherwise may have been addressed inadequately during an operation.
In summary, the current report demonstrated that TSITE performance was more dependent on the individual resident’s underlying fund of knowledge and intelligence level than on participation in a cardiac-specific didactic lecture series. Residents who had higher scores on prior standardized tests generally performed better on the TSITE. In addition, we found that the level of day-to-day knowledge that the residents demonstrated on the wards and in the operating room correlated with their database of factual information as assessed by the TSITE. The correlation between the subjective faculty assessment of resident knowledge and TSITE scores was significant, and residents who received better clinical performance evaluations tended to score higher on the examination. These findings suggest that reading and independent study may be the best way to improve TSITE scores.
|
Acknowledgments
|
|---|
The authors gratefully acknowledge the contributions of Hendrick B. Barner, MD, Richard J. Battafarano, MD, Lawrence L. Creswell, MD, Thomas B. Ferguson, MD, Charles B. Huddleston, MD, Jennifer S. Lawton, MD, Eric N. Mendeloff, MD, Bryan F. Meyers, MD, Nader Moazami, MD, Nabil Munfakh, MD, Michael K. Pasque, MD, and Charles L. Roper, MD.
|
References
|
|---|
- In: Nolan S.P., Salley R.K., eds. Comprehensive thoracic surgery curriculum. Arlington, VA: Thoracic Surgery Directors Association, 1994.
- Thoracic Surgery Directors Association historical perspective. Thoracic Surgery Directors Association Web site. Available from: URL: http://www.tsda.org/doc/3990. Accessed July 15, 2002
- Curriculum implementation task force update. Thoracic Surgery Directors Association Web site. Available from: URL: http://www.tsda.org/news/cic_update_1999.htm. Accessed July 15, 2002
- Shetler P.L. Observations on the American Board of Surgery in-training examination, board results, and conference attendance. Am J Surg 1982;144:292-294.[Medline]
- Godellas C.V., Huang R. Factors affecting performance on the American Board of Surgery in-training examination. Am J Surg 2001;181:294-296.[Medline]
- Itani K.M., Miller C.C., Church H.M., McCollum C.H. Impact of a problem-based learning conference on surgery residents’ in training exam scores. J Surg Res 1997;70:66-68.[Medline]
- Bull D.A., Stringham J.C., Karwande S.V., Neumayer L.A. Effect of a resident self-study and presentation program on performance on the thoracic surgery in-training examination. Am J Surg 2000;181:142-144.
- Hirvela E.R., Becker D.R. Impact of programmed reading on ABSITE performance. Am J Surg 1991;162:587-590.
Top
Abstract
Introduction
