Corrected: Predictive Value of Credit Score on Surgery Resident and Fellow Academic and Professional Performance

1 2, 3 1 4 1 5, 6 7 8 9 10 Open Access Original Article DOI: 10.7759/cureus.15946 How to cite this article Berry J A, Marotta D A, Savla P, et al. (June 26, 2021) Predictive Value of Credit Score on Surgery Resident and Fellow Academic and Professional Performance . Cureus 13(6): e15946. DOI 10.7759/cureus.15946

In good faith, we write to report an error and intellectual blind spot in our article that needs to be corrected and clarified. The abstract and conclusion contain the statements: 1) "This study suggests credit score may have a utility as a companion to traditional metrics used in identifying candidates for surgery residencies and fellowships who will have positive performance in the domains of research productivity, written examination performance, and professional awards and recognition"; and 2) "This study suggests credit score may be a valuable tool in identifying candidates for surgery residencies and fellowships who will have positive performance in the domains of research productivity and examination performance." These statements are not supported by the data contained within the study. As discussed below, these sentences from the former abstract and conclusion should more appropriately read, "Although our data may have indicated that higher credit scores may be associated with increased residency academic performance on examinations and research productivity we are not recommending any implementation of using credit scores as a metric for selecting individual surgical candidates for a position in residency or fellowship due to extensive socioeconomic variables and historical context of credit scores, which must be taken into consideration." While credit scores have been applied as a construct of historical decision-making in certain instances, it is undeniable that sociodemographics, socioeconomics, and life events confound substantive interpretations and preclude practical utility as a metric of suitability, as we mentioned in the paper. The paper reports the survey of residents and fellows currently in surgical programs and their reported credit score and their reported performance. There was no survey of candidates applying for residency or fellowship programs. There were no reported credit scores below 611, therefore there were no poor credit scores.
However, when considering credit score as a relative gauge of financial well-being in the context of this study, the data more appropriately shed light on the impact of financial well-being on resident and fellow performance. Furthermore, we feel it is necessary to reiterate the data do not support the use of credit score for identification or stratification of surgical residents and fellows. In fact, we posit that the use of such a metric for applicant suitability has discriminatory implications that could serve as a barrier for individual residents and undermine racial and class representation within surgery, which in turn could negatively impact patient care. Any additional research on this topic should focus on providing financial support and resources to those who may be at risk for financial well-being as described by the data.

Introduction
Surgical specialties consistently remain among the most competitive residency and fellowship programs with some of the highest rates of unmatched applicants. Attrition in surgical specialties is as high as 30% and particularly problematic given the extended duration of training and limited number of positions. Applicants are traditionally evaluated using a streamlined set of objective metrics, such as board scores, class rank, leadership, letters of recommendation, research productivity, and volunteerism. Consumer credit scores have been shown to be predictors of personality and work performance, however, the literature has yet to explore consumer credit histories in the context of surgical resident and fellow performance. This study aims to determine whether consumer credit scores of surgery residents and fellows are predictive of academic and professional performance.

Methods
This is a multi-institutional observational survey study across all American Council of Graduate Medical Education and Royal College of Physicians and Surgeons accredited surgical residency and fellowship programs in the United States and Canada. Ninety-nine surgical residents and fellows with educational status of post-graduate year two or higher participated in this study. Dichotomous (yes or no) survey items were formulated to assess performance indicators in the domains of notable achievements and awards, research output, written examination performance, professionalism, and surgical/technical skills. Threedigit Fair Isaac Corporation (FICO) credit scores, a widely accepted consumer reporting score, were collected to avoid calculation variability between algorithms.

Results
Surgical residents and fellows reported credit scores between 611( fair) and 853 (exceptional) with a median (interquartile range) of 774 (715-833). The majority of participants 51.5%(51) reported very good credit scores. Those with higher credit scores (very good/exceptional) were 377% more likely to have one or more positive performance indicators OR (95% CI) = 3.77 (1.43-9.97). Similarly, residents with lower credit scores (fair/good) were only 40% more likely to have one or more negative performance indicators. The credit score has a moderate ability to distinguish between the presence and absence of positive performance indicators (area under the curve {AUC} = 0.70, p = 0.001). The use of 753 as a credit score cutoff is 78.9% sensitive and 52.4% specific for discerning surgery residents and fellows with one or more positive performance indicators. The credit score did not significantly discern those with negative performance indicators.

Introduction
Graduating medical students and senior residents vie for a limited number of residency and fellowship programs every year in the United States and Canada. These programs vary in size, often consisting of between one and 220 residents and fellows. Program size and years of training change by specialty and program type. For example, surgical specialties offer fewer overall positions and have extended lengths (five to seven years) of training compared to non-surgical programs, which have more positions and shorter durations (one to five years). In the United States, only 20% of the 140,000 residency and fellowship positions available through the National Resident Match Program were surgically related. As such, surgical specialties consistently remain among the most competitive programs with some of the highest rates of unmatched applicants [1].
Given the popularity of surgical specialties and the high number of competitive applicants, program directors are tasked with an annual dilemma: selecting residents and fellows from a set of uniquely qualified individuals. The applicants are traditionally evaluated using a streamlined set of objective metrics such as board scores, class rank, leadership, letters of recommendation, research productivity, and volunteerism. A pragmatic approach to resident and fellow selection melds these objective measures with subjective impressions gleaned through direct candidate interaction during elective rotations and interviews. Nevertheless, the resident and fellowship selection process remain imperfect and fallible. While the majority of positions at competitive surgery programs are filled each year, attrition in surgical specialties is particularly problematic with rates as high as 30% [2]. Uncontrollable lifestyle, lack of support, and length of training were all cited as factors for attrition; none of which have been shown to be predicted by the abovementioned objective and subjective measures [3,4]. This prompts the need for further investigation into new assessment measures to predict the success, performance, and longevity of future residents and fellows.
Historically, consumer credit scores have been used by financial lenders to assess financial capacity, namely the ability to accept and repay debt. On the surface, credit scores provide a three-digit numeric value using mathematical algorithms with data from an applicant's financial history. Many scoring models exist. Depending on the scoring model used, weights are applied to several factors, such as the number of credit accounts, types of accounts, ages of accounts, debt versus available credit ratio, and payment history. The scores often range from 300 (bad) to over 850 (excellent). Over time, credit score was found to be a predictor of candidate personality and job performance in the workplace [5,6]. Studies emerged evaluating credit scores as surrogates for performance, extraversion, agreeableness, openness, conscientiousness, and neuroticism [7,8]. However, the literature lacks a detailed characterization of consumer credit score in the setting of graduate medical education. Therefore, we set out to evaluate the utility of credit score as a predictor of positive and negative performance indicators in surgical residents and fellows.

Materials And Methods
This is a multi-institutional observational survey study across all American Council of Graduate Medical Education and Royal College of Physicians and Surgeons accredited surgical residency and fellowship programs in the United States and Canada. Surgical residents and fellows with educational status of postgraduate year two or higher were invited to take part in this study. Contact information for 1089 program coordinators of general surgery, integrated plastic surgery, integrated vascular surgery, neurosurgery, orthopedic surgery, otolaryngology, and urology programs were compiled using the database from the American College of Surgeons, American College of Osteopathic Surgeons, American Association of Neurological Surgeons and other professional societies. Program coordinators were sent electronic requests soliciting resident study participation, 50 of which returned undeliverable. Electronic communications included instructions and a hyperlink to complete the online survey. This study was approved as exempt status by the Arrowhead Regional Medical Center Institutional Review Board. Individual consent was implied through survey completion.

Survey
The survey consisted of an online 10-question survey link that was emailed to the individual program coordinators. A list of the survey questions by performance indicator type (positive vs. negative) is displayed in Table 1. The information contained within the survey was inherently private and sensitive. Thus, the survey was designed to be completed without gathering any of the participant's personal identification information and demography (i.e., name, social security number, address, age, sex, and residency/fellowship program name) to minimize the risk of financial or educational information disclosure and maximize participation. Dichotomous (yes or no) survey items were formulated to assess performance indicators in the domains of notable achievements and awards, research output, written examination performance, professionalism, and surgical/technical skills. One item served to identify participants with histories of financial hardship by means of a disadvantaged background (i.e., public aid, food stamps, welfare, etc.). Three-digit Fair Isaac Corporation (FICO) credit scores, a widely accepted consumer reporting score, were collected to avoid calculation variability between algorithms.

Indicator Type
Question Text

Procedure
The survey items were sorted by positive and negative performance indicators. Three positive performance indicators measured a participant's history of achievements awarded during residency, research output, and top in-service examination scores. Five negative performance indicators measured history of below average in-service examination scores, academic probation, probation for professionalism or behavioral violations, repeat residency rotations, and below average surgical or technical skills. Positive and negative scores were tallied by participants. A net performance value was calculated by subtracting the cumulative positive performance indicators from negative performance indicators. Participant performance was classified as the net negative, net neutral, and net positive according to negative, neutral (zero), and positive net performance values, respectively. The credit scores were sorted by value into a categorical variable based on poor (<580), fair (580-669), good (670-739), very good (740-799), and exceptional (+800). Additionally, categories were grouped into high (exceptional and very good) and low (good, fair) credit score groups.

Data analyses
Chi-squared tests were utilized to compare the proportion of surgery residents with positive and negative performance indicators across high (exceptional, very good) and low (good, fair) credit score groups. Results were interpreted as OR where values of 2.0, 3.0, and 4.0 correspond to minimum, moderate, and strong risk estimates, respectively [9]. Here, we used the phi (φ) coefficient to interpret effect size (φ = [ 2 /N] 1/2 ), where values of 0.2, 0.5, and 0.8 correspond to minimal, moderate, and strong effect sizes, respectively [10].
Kolmogorov-Smirnov and Shapiro-Wilk tests were used to assess normality for credit score continuous data; analyses revealed evidence of non-normality of the distribution of credit scores (D{99} = 0.13 , p < 0.001; W{99} = 0.92, p < 0.001) [11]. As such, nonparametric tests were used for between-group comparisons. We first compared differences in credit scores across the performance indicator groups (i.e., net negative, neutral, and positive) using a Kruskal-Wallis H test [12]. We used the eta 2 (η 2 ) coefficient and an index of Kruskal-Wallis H effect size (η 2 = 2 / (n-1)) with 0.4, 0.25, 0.64 correspondings to minimal, moderate, and strong effect sizes, respectively [10]. We treated the Kruskal-Wallis H test analysis as an omnibus test, where a significant overall effect was then followed up by pair-wise post hoc Mann-Whitley U tests to determine pair-wise differences among the groups [13]. We used the r coefficient to index Mann-Whitley U effect sizes (r = z / (n) 1/2 ), which were interpreted using 0.2, 0.5, and 0.8 as cutoffs for minimal, moderate, and strong effect sizes, respectively [10].
Lastly, receiver operating characteristic curves (ROC) analyses were used to assess the practical utility of credit score in the selection of surgery residents/fellows to discern performance indicators. These analyses were interpreted using the area under the curve (AUC) values of < 0.70 indicating low classification utility, AUC values between 0.70 and 0.90 indicating moderate classification utility, and AUC over 0.90 indicating high classification utility. The data were analyzed in SPSS software version 27 (Armonk, NY: IBM Corp.).

Results
Ninety-nine surgery residents participated in the study.

Credit scores
Participants reported credit scores between 611 and 853 with a median (interquartile range) of 774 (715-833). The majority of participants, 51 (51.5%), reported very good credit scores. Exceptional and good credit scores were reported by 24 (24.2%) and 20 (20.2%) of respondents, respectively. None of the participants reported poor credit history.

Performance indicator survey results
Among the positive performance indicators, 41 (41.4%) of the participants reported an average of one research publication per year, while only 19 (19.2%) reported an achievement award or top examination score within his or her residency program. An academic achievement award includes categories such as an individual programs resident of the year award, state and national achievement awards for research publications, etc. In general, positive performance indicators were reported more frequently than negative performance indicators. Twenty-nine (29.3%) of the respondents reported below-average examination scores on a national residency examination while none of the participants reported a history of academic probation. A modicum of participants reported prior behavioral or professional probation, 4 (4.0%), residency evaluations documenting below average surgery skills, 2 (2.0%), or having to repeat a residency rotation, 1 (1.0%). Figure 1 below displays the frequency of participants with net negative, neutral, and positive performance values. The majority of participants had a total of zero cumulative positive indicators, zero cumulative negative indicators, and zero net performance indicators. Only 2% of participants had either three cumulative positive indicators, two negative net indicators, or three positive net indicators each.

Quantitative analyses
Comparisons between those with high credit scores (exceptional, very good) and low credit scores (good, fair) by positive and negative performance predictors are displayed in Table 3. Chi-squared analyses revealed a greater proportion of residents with one or more positive performance indicator in those with high credit scores compared to those with low credit scores, 65.3% vs. 33.3%; χ 2 (1, n = 57) = 7.62, p = 0.006, φ = 0.37. The analysis also indicated that residents with high credit scores had 377% higher odds of having one or more positive performance indicators, OR (95% CI) = 3.77 (1.43-9.97). In contrast, significant differences in the number of negative performance indicators between those with low and high credit scores was not observed, 41.7% vs. 30.7%; χ 2 (1, n = 33) = 0.99, p = 0.33 φ = 0.17.

Practical utility
To assess the credit score's ability to discern resident/fellow performance, credit scores were subjected to AUC analysis of ROC curves (Figure 2). Credit score has a moderate ability to distinguish between the presence and absence of positive performance indicators (AUC = 0.70, p = 0.001). The optimal cutoff for this ROC analysis was a credit score of 753, with values above or equal this being 78.9% sensitive and 52.4% specific for discerning surgical residents with one or more positive performance indicators. In contrast, ROC analysis indicated that credit score did not significantly detect negative performance, and as such classification accuracy rates were poor (AUC = 0.548, p = 0.436).

Discussion
We used a survey-based multi-institutional approach to evaluate the utility of credit scores as a predictor of positive and negative performance indicators. Overall, we found that surgical residents and fellows had fair to exceptional credit scores. Surgical residents and fellows with high credit scores (exceptional, very good) were much more likely to have positive performance indicators, such as top written examination performance, multiple research publications, and professional awards and recognition. We found that the use of 753 as a credit score cutoff is 78.9% sensitive and 52.4% specific for discerning surgery residents with one or more positive performance indicators. However, credit scores did not significantly discern those with negative performance indicators. We found that low credit scores (good, fair) did not predict negative performance indicators such as below-average performance (written examinations and surgical skills), probation status (i.e., academic, behavioral, professional), and repeated rotations. To our knowledge, this is the first study in the literature to investigate credit history in the context of graduate medical education.
The utility of credit scores as a potential screening method relies on its potential function as a construct of historical decision-making. That is, the idea that credit scores represent habits, character traits, and decision-making leading to financial stability, rather than socioeconomic status alone. Generally, negative factors (i.e., short duration of credit history, many credit inquiries, high debt to credit ratios, and late payments) reduce credit score, while the opposite trends increase credit score. Although each factor is related to fiscal responsibility, they also can serve as surrogates of behaviors and habits. Late payments on outstanding balances may suggest poor organization, irresponsibility, and lack of follow-through. High credit card balances secondary to overspending may reveal a lack of self-discipline and restraint. Numerous credit inquiries and outstanding debts may suggest poor planning for the future. Considering these scores can date back as far as ten years, credit scores can give rise to a unique assessment of a candidate which is not currently available.
In this study, we found higher credit scores significantly predicted those with top written examination performance, multiple research publications, and professional awards and recognition. This is consistent with past literature showing that higher credit scores translate to higher work-related performance [7]. Taking part in extracurricular activities, such as research, and studying enough to score highly on written examinations during the rigors of post-graduate education takes persistent motivation, commitment, and emotional stability. Similar discipline is necessary to achieve and maintain high credit scores. It is also conceivable that those who commit to success in one domain (i.e., creditworthiness) have the innate drive to remain consistent in other domains of life (i.e., positive performance during residency) [14,15]. Thus, in a functional sense, credit scores may have the ability to detect candidates who are simply driven to succeed.
In contrast, lower credit scores did not predict below-average performance in academic and surgical skills, probationary status, or repeated clinical rotations. This may arise from the positive skew of credit scores represented by those within surgery-related programs. The paucity of lower credit scores in this cohort (i.e., poor, fair) may prevent such a prediction. That is, credit scores included in the cohort are not truly "lower" credit scores required for such an association. Moreover, this study surveyed residents and fellows who have already attained a position in residency and fellowship, and those with negative credit scores may have already been eliminated or succumbed to attrition [16]. Nevertheless, those who attain graduate medical education in surgical specialties share many personality traits including extroversion, conscientiousness, and emotional intelligence; all of which have been associated with positive credit scores [7,17,18]. Those who are extroverted tend to be increasingly energetic, can effectively network, and engage in interpersonal activities such as collaborative research, while those who are conscientious may be more reliable and selfcontrolled with a focus on planning, organizing, and satisfying responsibilities [7,8]. Finally, those with a high degree of emotional intelligence are less likely to succumb to burnout [19,20]. Taken together, the narrow range of relatively high credit scores of surgery trainees who successfully matched may explain the lack of significance for predicting low performance with lower credit history.
Current metrics for assessing medical residents and fellows are aimed at predicting candidate success during residency. While the framework by which individual surgery programs define success varies, many attributes are widely accepted and emphasized as important. We found that credit scores above 753 had a 79% sensitivity in discerning candidates with one or more positive performance indicators, however, specificity was lacking at 52%. On a practical level, the utility of credit score may function better as a supplement rather than a stand-alone metric in predicting future success to avoid issues with false negatives. Nevertheless, it is important to point out that credit score is subject to influence by socioeconomic status, unforeseen life events, and even fraud. As such, ethical and legal controversies exist when using this score as an indicator of merit [21]. In these instances, it is unclear how the predictive value of credit score holds up. Careful consideration of life events, particularly those which negatively impact a credit score, would likely need to be considered independently.

Limitations and future studies
This study is not without limitations. To start, sociodemographic, socioeconomic, and professional variables were not collected in this study to encourage participation. Surgery programs are relatively small (sometimes training one resident per year) and the potential risk of identifying a study participant or exposing a participant's credit score may deter individuals from participating. Future studies should include these variables for additional comparisons and to assess for potential confounding variables. Secondly, this study relies on self-reported credit scores. While self-reported credit scores have been shown to be a reliable measure of actual credit scores, it is possible that some scores were not accurately or honestly reported [22]. Next, the cross-sectional method of the study reveals results from a single time point. Credit scores are constantly in flux and a longitudinal study may be useful to determine associations between changes in credit scores and changes in performance for residents and fellows as they progress throughout their careers. Finally, the inclusion of residents and fellows who succumbed to attrition may also help determine if financial strain may have been a factor in dropping out.

Conclusions
Overall, we found that surgical residents and fellows in our study had fair to exceptional credit scores. Surgical residents and fellows with high credit scores (exceptional, very good) were 377% more likely to have one or more positive performance indicators. Similarly, residents with low credit scores (good, fair) were only 40% more likely to have one or more negative performance indicators. While credit score is significantly functional in discerning between those with and without positive performance indicators, sensitivity and specificity rates must be considered. The use of 753 as a credit score cutoff is 78.9% sensitive and 52.4% specific for discerning surgery residents with one or more positive performance indicators. Credit scores did not significantly discern those with negative performance indicators. Although our data may have indicated that higher credit scores may be associated with increased residency academic performance on examinations and research productivity we are not recommending any implementation of using credit scores as a metric for selecting individual surgical candidates for a position in residency or fellowship due to extensive socioeconomic variables and historical context of credit scores, which must be taken into consideration. However, additional data and research is needed to fully explore the use of credit score in additional domains such as probationary status, surgical skill, and professionalism.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. Arrowhead Regional Medical Center Institutional Review Board issued approval N/A. Approval of the research study was accepted as exempt because the collection or study of existing data, documents, records, pathological specimens, or diagnostic specimens were recorded by the investigator in such a manner that subjects cannot be identified, directly or through identifiers linked to the subjects. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.