Factors Predictive of Publication Among Medical Students Participating in School-Sponsored Research Programs

Introduction: Publishing research is an important component of medical students’ career development and becoming a more competitive residency applicant. Many medical schools offer structured programs to enable students to participate in research during their preclinical and clinical years, but the majority of student-mentor partnerships do not culminate in publication across a variety of institutions and medical specialties. The primary objective of this study is to determine if any factors associated with mentee-mentor partnerships are predictive of publication from two school-sponsored research programs at a single US medical school. Methods: The PubMed-indexed publications of all student-mentor pairings from a summer internship (after year 1 of medical school) or research elective (during year 4 of medical school) at a single institution from 2008 to 2018 were retrospectively reviewed. Student/mentor demographic information was associated with the probability of publication. Results: A total of 124 students participated in the summer internship with 32 (26%) achieving publication. The publication was significantly more likely for students that were from highly ranked undergraduate institutions (p = 0.04; likelihood ratio (LR) = 5.788), were future Alpha Omega Alpha (AOA) members (p = 0.03; LR = 4.597), or worked with a mentor focused on clinical rather than basic science research (p = 0.02; LR = 5.662). Forty-four students participated in the fourth-year elective with 11 (25%) achieving publication. The publication was more likely if the student worked with a mentor without a Doctor of Medicine (MD)/Doctor of Osteopathic Medicine (DO) degree (p = 0.001; LR = 7.051), with a PhD degree (p = 0.002; LR = 7.820), or a mentor with prior publication(s) with prior mentee(s) (p = 0.03; LR = 5.368). Conclusion: Only one-quarter of mentor-mentee research pairings resulted in publication, with student-related factors more predictive for publication from the internship and mentor-related factors more predictive of publication from the elective. Approaches to promote successful completion of medical student research projects should be considered to yield the greatest value from students’ work and strengthen the development of future physician-scientists.


Introduction
Medical students in the United States often engage in research with faculty mentors, with the potential for both mentee and mentor to derive a variety of benefits from the experience [1][2][3]. A students' knowledge of research methodology can contribute toward a better understanding of interpreting and critiquing medical literature so that scientific evidence can be applied appropriately to clinical practice [4]. Some students may develop an interest in becoming physician-scientists themselves and improving patient care through discovery. While a research experience has value in and of itself, seeing projects through to publication is an important determinant of its success. Independent of the effect of a publication on changing clinical practice, publications are a critical component of a student becoming a competitive applicant for many specialties in the residency match [5], and research productivity in medical school has been associated with increased research productivity in residency and thus better career placement, especially in cultivating academic physicians [6][7][8][9][10][11].
Many medical schools offer structured programs to enable students to participate in research during their preclinical and clinical years, but the majority of student-mentor partnerships do not culminate in publication across a variety of institutions and medical specialties [1][2][11][12][13]. It is likely that not all student-mentor partnerships and projects have a similar probability of publication from the outset, but little is known about what factors may impact this probability. Having this knowledge could better inform students, mentors, and academic administrators on ways to optimize the value of similar research experiences in the future. The primary objective of this study is to determine if any factors related to the student, mentor, or project are predictive of publication in two research programs at a single US medical school.

Study setting and participants
This retrospective review included medical students paired individually with a research mentor through (1) a summer research internship or (2) a research elective, at a single institution from 2008 to 2018. This study only included MD candidates and excluded MD/PhD candidates since the latter is on a different track with a greater focus on research in their education. The research internship consists of an optional 10-week paid experience (funding from Dean's office) during the summer following a student's first-year curriculum where mentors were chosen individually by students or assigned based on their specialty of interest. The research elective consists of an optional one-month experience during the fourth year of medical school where mentors were chosen individually by students. Prior to starting both programs, students were expected to submit proposals (under the guidance of their mentor) detailing the hypothesis, objectives, background, methods, and IRB approval information for the project. Upon completion of the program, students were expected to formally present their research results via presentation at an institution-sponsored research event or regional/national meeting.
The primary endpoint of this study was the mentee and mentor collaboration resulting in a publication, whereas a secondary endpoint was to determine the factors associated with multiple publications. Each endpoint was evaluated separately for the summer internship and fourth-year research elective.
Demographics of students and faculty mentors were collected to determine characteristics associated with a higher likelihood of publication. The evaluated independent variables related to the mentors included research focus (basic science or clinical), type of degree, clinical/tenure track level, prior number of publications prior to collaboration, years of institutional employment prior to collaboration, gender (deduced by name and online pictures, including gender discordance from student), prior research mentee(s) through internship or elective, and prior publication with research mentee(s) through internship or elective.
The evaluated independent variables related to the students included gender, number of prior publications, and undergraduate institution ranking (based on US News and World Report 2020 college rankings) [14]. Individual students' grades and standardized test scores were not available for this study; however, to distinguish high-performing medical students, Alpha Omega Alpha (AOA) membership was used as a surrogate, even though students do not actually become AOA members until later in medical school. The primary factor considered for AOA membership at this institution is having grades in the top quartile of one's class during their first two years of medical school. For those students who meet these criteria, secondary factors that are considered include receiving at least two letters of recommendation for academic achievement, attaining a high Step 1 score, taking on leadership positions, and being productive in research.

Data collection
A list of student and mentor names and titles of their project for each research program were obtained from the directors of each research program. Data related to mentor and mentee demographics were collected via internet search using Google to identify the individual on the website of their current employer, which was typically the primary source of information. This was supplemented and cross-checked with any additional information available on LinkedIn, Google, or prior residency/employment websites. Mentors' information was also frequently available from curriculum vitae posted online. There were minimal discrepancies between the information on the websites that were available. The National Center for Biotechnology Information (NCBI) PubMed database was used to collect publication information. A publication was accredited to a given student-mentor pairing if both were listed as co-authors on the publication and the title of the publication was related to the title of the project in the research program's records. In order to account for surname changes due to marriage, the mentor's publication record was reviewed for both the first name and surname of their female student mentees, and any suspected surname changes were confirmed using similar internet searches as above. If a collaboration resulted in a publication, then the student authorship number, total number of authors, publication journal, and that journal's associated impact factor were also recorded. Publications in "predatory journals," which commonly describe openaccess medical journals that publish articles online with little or no peer review, low academic standards, and little credibility, were not excluded if listed in PubMed.

Statistical analyses
Associations between the above endpoints and independent variables were evaluated using the Chi-squared, Fisher's exact, and Mann-Whitney U tests, where appropriate. Fisher's exact test was used for comparisons in which 20% of the cells had expected frequencies less than five; if the contingency tables were large, the Freeman-Halton extension was used [15]. Statistical analysis was carried out using Statistical Package for Social Sciences version 24.0 (SPSS Inc., Chicago, IL). A p-value less than 0.05 was considered statistically significant.

Study participants and characteristics
After approval by the local institutional review board, data were collected on a total of 336 individual students and their mentors. It included 168 students, with 124 (74%) participating in the summer internship and 44 (26%) participating in the research elective as presented in Table 1. Seven students participated in both programs and were included in the analysis of each. Gender distribution among medical students in both programs was different. Medical school enrollment at this institution is 48% females, but twice as many male medical students participated in the summer program compared to females (66.9% versus 33.1%, respectively); however, the elective year had a similar gender distribution between males and females (52.3% versus 47.7%, respectively). A large majority of students who participated in the summer research program (83.1%) had no previous publication record prior to entering the internship program. Across programs, fewer than 25% of medical students became members of the AOA Medical Honor Society. Students ultimately pursued a wide variety of medical and surgical residency programs after graduating from medical school.

Internship, n (%)
Elective, n (%) To Be Determined 23 (18.5%) ----Not Applicable 6 (4.8%) ---- The distribution and characteristics of mentors included in the study are presented in Table 2. More male mentors were available to medical students in both the internship (81.5%) and elective program (72.7%). In the internship program, students selected more tenure track mentors with a PhD for basic science projects (62.1%), but this was reversed in the elective program where the majority of mentors were on a clinical track (68.2%). A total of 72.7% of mentors in the elective program had not mentored a student in either program previously, and 86.4% had a mentee with no prior research publications.

Publication outcomes of medical students in school-sponsored programs
For the summer internship cohort (n = 124), 32 collaborations resulted in publication (26%), with a median time until publication of 21 months (IQR, 17-30 months). Seven students (22%) were the first authors on the publication's byline, 12 students (38%) were the second authors, and 13 students (41%) were the third (or higher) authors. The median impact factor of the journal of publication was 1.8 (IQR, 1.4-3.3). Among the students that published, 11 (34%) had multiple publications with their respective mentor, among which five students (16%) had three publications with their mentor, and two students (6%) had four publications with their mentor. At the time of this analysis, 78 of the 124 students (63%) who participated in the summer internship had at least one publication (with any mentor).

Factors predictive of student-mentor publication
Categorical and continuous variables associated with student-mentor publication for each cohort are displayed in Tables 3, 4, respectively. Publication was significantly more likely for students who participated in the internship if they studied at a more highly ranked undergraduate institution (p = 0.04), would become an AOA member (p = 0.03), and worked with a mentor focused on clinical rather than basic science research (p = 0.02). Publication was significantly more likely for students who participated in the elective if they worked with a mentor without a Doctor of Medicine (MD)/Doctor of Osteopathic Medicine (DO) degree (p = 0.001), with a PhD degree (p = 0.002), or who had publication(s) with prior mentee(s) (p = 0.03).

Publication (IQR) No Publication (IQR) p-value Publication (IQR) No Publication (IQR) p-value
Mentor Years at Institution 6 (4-11) 7 (4-12) 0.58 6 (5-10) 5 (  Categorical and continuous variables associated with multiple mentor-mentee publications for both internship and elective cohorts are displayed in Tables 5, 6, respectively. Multiple publications were significantly more likely for students who participated in the internship if they worked with a mentor focused on clinical rather than basic science research (p = 0.050). Multiple publications were significantly more likely for students who participated in the elective if the student had published previously (p = 0.045).

Discussion
Research is a critical component of medical education, both in the context of practicing evidence-based medicine and for the purpose of scholarship and discovery. As such, opportunities for students to pursue mentored research projects are common at many medical schools [1,12,13] as well as through many national specialty societies hoping to promote greater academic interest in their field [11,16]. As in prior studies evaluating general research programs for all medical students regardless of their specialty of interest (or the specialty of their mentor), we found that approximately one in four mentor-mentee research pairings resulted in a manuscript publication in a PubMed-indexed medical journal, and two-thirds of participants published an article on any topic in their subsequent careers. This is comparable to what has been reported for a similar program at other institutions [2,11,17] and suggests that participation in any organized research program is associated with the pursuit of future research [16,18]. Our findings are unique in reporting outcomes of two different medical student research experiences at a single institution and in describing factors associated with publication.
Interestingly, student-related factors were most predictive of publication from summer internship projects, and mentor-related factors were most predictive of publication of fourth-year elective projects. It is perhaps not surprising that early in medical school, students who attended a more highly ranked college, or who have higher grades (as suggested by future AOA membership), would also be more successful in their research projects [19]. Attending a more highly ranked college may be associated with prior opportunities and exposure to research leading to a better understanding of how to perform in a research setting during medical school. Or alternatively, since most publications did not occur for many months afterward the research period, a student's ability to merit co-authorship may have more to do with sustained engagement in the project even while balancing it with their medical school course work before and after the research period.
The type of project pursued is another important factor, with clinical projects typically being less timeconsuming and perhaps easier to sustain during the school year than basic science laboratory work. Furthermore, although authorship criteria are relatively standardized across medical and biomedical journals, mentors' views on what merits authorship may vary. For instance, it is possible that basic scientists may be more rigorous than their clinical counterparts in their authorship requirements since the former is perhaps more used to engaging MS or PhD degree candidates for whom publishing a manuscript is a component of earning their degree. Notably, predictors of a more prestigious first-author publication were also not assessed in this study due to the relatively small number of them and because the faculty mentors' criteria to make a student first author are likely to vary substantially from one project to another based on the type of project, number and type of other people involved in it, and the relative contributions of each person. Unexpectedly, prior publications from the student or mentor did not impact the probability of publication from the pairing for the internship program.
The factors associated with publication in the fourth-year student elective program differed, likely related to the type of student who pursues this very different experience. Elective projects were published more quickly than internship projects and eight out of 11 students with publications in this cohort had multiple publications with their elective mentor. Both of these findings suggest students may have had pre-existing projects with mentors and completed them during the elective period. On the other hand, new elective projects with a clinical mentor, who a student perhaps met during the third-and fourth-year clinical rotations rather than selected specifically for research purposes, were less likely to result in publication. This may account for the fact that publication was more common with mentors with a PhD than with an MD or DO degree in the elective program. For instance, five physicians who served as elective research mentors had zero publications, including one mentor who worked with four students, and it is not surprising that none of these pairings resulted in a student publication. Unfortunately, some students may participate in the fourth-year elective for purposes other than research, like having free time for residency interviews. In general, we believe that if elective programs like this are going to be meaningful for students, there needs to be a more structured program with goals and expectations as well as significant vetting of experienced mentors who have a track record of publications, particularly with students they have worked with in the past. These types of programs have been implemented at some medical schools [1,13]. Other support structures like a mentor development program may also be worthwhile to help generate a larger pool of high-quality mentors to best serve students' needs. Otherwise, students would be better served on any clinical rotation that the research elective took the place of. Clearly, clinical mentors and clinical projects can work well for students as shown in the internship data, and perhaps more clinicians need to promote their availability as a mentor during the summer internship when more basic science projects were pursued, despite more clinical projects being published.
There are several limitations to this study. Most importantly, it was conducted at a single institution, which is not among the top 40 US medical schools with the highest National Institute of Health research funding. Factors associated with publication for these types of medical student research programs may differ at more robust research institutions [20]. Furthermore, being a public medical school, the majority of students were from the same state or received an undergraduate degree at this state university and may have more homogeneous backgrounds than at some private medical schools, despite the wide variety of specialties that the students ultimately pursued. Also, PubMed was the only database searched, and it is possible that medical students' articles were published in non-indexed journals or potentially using a different name.
Likewise, online searches to obtain demographic information has limitations for certain variables. For instance, while we believe that most surname changes for female students were captured (particularly related to publications with their mentor), some may not have been. Also, while the variables clinical/tenure track level and years of institutional employment provide some information on how advanced each faculty member was in their academic career, we could not determine the actual length of each faculty members' academic career due to inability to obtain reliable prior employment history. There are also potentially other variables related to the mentor-mentee relationship and the extent of students' contribution to the research that may have influenced publication and may have been better assessed using qualitative approaches that were not feasible in this retrospective study. Finally, there were only fewer students who participated in the research elective compared to the summer internship, which limits the power of our analysis for the elective cohort.

Conclusions
The findings from this study may assist medical student administrators in looking critically at the research programs that they offer to their students and provide useful information to students and mentors as they select a research collaboration. Publication after the summer internship was more likely for students that were from highly ranked undergraduate institutions, were future AOA members, or worked with a mentor focused on clinical rather than basic science research. Publication after the research elective was more likely if the student worked with a mentor without an MD/DO degree, with a PhD degree, or a mentor with prior publication(s) with prior mentee(s).