Effects of Organ System Courses of the First Two Years of Medical School on the Performance of the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 2-Cognitive Evaluation

Objectives: The Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 2-Cognitive Evaluation (CE) is a board examination that medical students usually take in the third or fourth year of medical school. A few researchers have investigated the prediction of COMLEX Level 2-CE scores based on the performance in third-year clerkships. However, given how close the clerkships are to the board exam, this type of prediction is too late for students to have adequate time to get assistance to prepare for COMLEX Level 2-CE. We aimed to investigate the predictive value of each organ system course during the first two years in predicting COMLEX Level 2-CE performance. Our findings will help students at risk focus on important basic and clinical sciences much earlier before preparing for COMLEX Level 2-CE. Methods: Academic data from students enrolled at Rocky Vista University College of Osteopathic Medicine from 2011 to 2017 were retrieved. Data included the Medical College Admission Test (MCAT) scores, course grades in the first two years of medical school, COMLEX Level 1 scores, and COMLEX Level 2-CE scores. Pearson correlation coefficients, a multiple linear regression model, and a backward stepwise regression model were generated for analysis. Results: The highest correlation with COMLEX Level 2-CE scores was the COMLEX Level 1 score, followed by the performances in the third-semester Cardiovascular System II (CV II) and Renal System II (REN II) courses. Multiple linear regression and backward stepwise regression predictive models found that scores on third-semester CV II and Principles of Clinical Medicine III (PCM III) were the most significant predictors of performance on Level 2-CE. Both models explained 46% of the variance in COMLEX Level 2-CE scores. Conclusions: Performances in third-semester courses are the most important predictors of COMLEX Level 2-CE scores.


Introduction
Students in osteopathic medical schools must pass the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) series Level 1 and Level 2-Cognitive Evaluation (CE) to graduate and match to residency programs. These examinations are equivalent to the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) tests in allopathic medicine, respectively [1]. Students usually take COMLEX Level 2-CE in the third or fourth year of medical school. All board examinations, including the COMLEX series and USMLE series, are important in the residency application. Higher scores on COMLEX Level 2-CE are associated with an increased likelihood of matching to first choices of residency programs and more competitive specialties [2]. Furthermore, COMLEX Level 2-CE scores are more crucial than ever for residency placement since reporting "pass or fail only" has been applied to USMLE Step 1 and COMLEX Level 1 on and after January 26, 2022, and May 10, 2022, respectively [3,4]. Therefore, early prediction of performance on COMLEX Level 2-CE is critical, so that students who are at risk of scoring poorly will have enough time to get assistance and perform better on this exam.
Although some studies have explored the prediction of performance on COMLEX Level 1 [1,5], only a few studies have been focused on the prediction of COMLEX Level 2-CE performance [6][7][8]. Preadmission undergraduate science grade point average (sciGPA) or Medical College Admission Test (MCAT) scores show a small but significant correlation with COMLEX Level 2-CE performance [6][7][8]. Academic performance during the first two years of medical school has a stronger correlation with COMLEX Level 2-CE scores as compared to preadmission sciGPA and MCAT scores [7]. In addition, Evans et al. investigated academic data from 1,254 students from 12 osteopathic medical schools. They found that the highest correlation with the COMLEX Level 2-CE score was the COMLEX Level 1 score, followed by the performance in the first two years of medical school, total grade point average (GPA) as a fourth-year student, and clinical GPA [9]. Furthermore, since the Comprehensive Osteopathic Medical Achievement Test (COMAT) series of subject examinations was installed in 2011, two studies indicated that scores on COMAT explained 63-68% of the variance in Level 2-CE scores, with COMAT Internal Medicine and Emergency Medicine displaying the highest correlations [10,11]. Using the COMAT performance to predict COMLEX Level 2-CE is too late for students since they will not have enough time to modify and improve their preparation for COMLEX Level 2-CE.
There is no research investigating the impact of each organ system course during the preclinical years on the performance of COMLEX Level 2-CE. Our goal is to find the earliest predictors of COMLEX Level 2-CE scores in the preclinical years so that students can focus on improving earlier.

Data
As described in our previous publication [1], Rocky Vista University College of Osteopathic Medicine (RVUCOM) has a stepwise organ system-based curriculum for the first two preclinical years. Each organ system is covered twice, with the first course in the first year focusing on anatomy and physiology, and the second course in the second year focusing on pathology, pharmacology, and diagnosis and treatment. Details of the major courses in the first three semesters are listed in our previous study [1].

Participants
Academic data of students who enrolled at RVUCOM between 2011 and 2017 were retrieved. The project entitled "Using Simulation Modeling to Predict Failure on COMLEX 1 and 2 at First Attempt Through a Longitudinal Investigation" was approved by the IRB committee of RVUCOM (IRB number: IRB #2019-0079) [1]. Informed consent was waived because the IRB committee determined that this project was exempt. De-identified data were disclosed to the investigators.

Independent variables
The MCAT scores, final grades in each course in the first two years of RVUCOM, and COMLEX Level 1 scores at the first attempt were used. As described in our previous publication [1], if students took the MCAT more than once, average scores across all MCAT attempts were calculated. The average scores in all courses of the first year, all courses of the second year, and all preclinical courses in the first two years of medical school were calculated and were used as independent variables as well.

Dependent variable
Scores on COMLEX Level 2-CE from 586 students on their first attempt were collected.

Correlations between independent variables and scores of COMLEX Level 2-CE
Pearson correlation coefficients (R) were used to measure the correlations between independent variables (MCAT scores, preclinical course grades, the average scores in all courses of the first year, second year, and first two years, and COMLEX Level 1 scores) and the dependent variable (COMLEX Level 2-CE scores).

Multiple linear regression model
A multiple linear regression model was generated. This predictive model is based on academic performances in organ system courses of preclinical years. The independent variables (27 in total) included all final grades in each course in the first two years of medical school and did not include MCAT and COMLEX Level 1 scores. The dependent variable was the score on COMLEX Level 2-CE. The final formula consisted of 27 variables.

Backward stepwise regression model
Independent variables included scores in each course in the first two years and did not include MCAT and COMLEX Level 1 scores. The dependent variable was the score on COMLEX Level 2-CE. In the final formula, only significant variables have remained because insignificant independent variables were deleted sequentially.

Statistical analysis
All analyses were done using either IBM SPSS version 20 (IBM Corp., Armonk, NY) or SigmaPlot 14 (Systat Software Inc., San Jose, CA).

The correlations between MCAT scores, the average scores in the first two years, and performances in board examinations
A total of 906 students finished COMLEX Level 1, and 586 students took COMLEX Level 2-CE. As shown in Table 1, MCAT scores and performances in the first two years were significantly correlated with COMLEX Level 1 scores and COMLEX Level 2 scores, respectively. The highest correlation with the COMLEX Level 1 score was the average score in second-year courses (r = 0.76), followed by the average score in the first two years' courses (r = 0.73) and the average score in first-year courses (r = 0.66). The highest correlation with performance on COMLEX Level 2-CE was the COMLEX Level 1 score (r = 0.76), followed by the average score in second-year courses (r = 0.71), the average score of all the first two years' courses (r = 0.66), and the average score in first-year courses (r = 0.59). MCAT scores were weakly correlated with the COMLEX Level 1 score (r = 0.18) and COMLEX Level 2-CE performance (r = 0.14), respectively.

The correlations between performance in each course and COMLEX Level 2-CE scores
Since we found that performance throughout the first two years was highly correlated with COMLEX Level 2-CE, we further investigated the effects of each organ system course on COMLEX Level 2-CE performance.

Discussion
From our study, the highest correlation with COMLEX Level 2-CE performance was COMLEX Level 1 scores, followed by average scores in second-year courses. Our predictive regression models showed CV II and PCM III scores were the most significant predictors. Both models explained approximately 46% of the variance in the COMLEX Level 2-CE scores.
Our current study found that COMLEX Level 1 scores had the highest correlation with COMLEX Level 2-CE scores, with an R-value of 0.76. This was consistent with a 0.76 correlation reported by Evans et al. [9]. Therefore, it is reasonable to infer that a student who performs poorly on COMLEX Level 1 is at risk for poor performance on COMLEX Level 2-CE.
We found that third-semester and fourth-semester courses had higher correlations with performance on COMLEX Level 2-CE as compared to first-semester and second-semester courses. The highest correlation was the third-semester CV II course. Our two regression models provided convincing evidence that performance on CV II was the most important predictor. The backward stepwise regression model demonstrated that additional third-semester courses REN II and PCM III, fourth-semester NEURO II, and first-semester MCM were significant predictors of COMLEX Level 2-CE performance as well. In addition, our previous predictive models for COMLEX Level 1 have found that third-semester courses were most essential for predicting the COMLEX Level 1 score [1]. Since there is a high correlation between COMLEX Level 1 and Level 2-CE, it is not a surprise that third-semester courses are also strong predictors of COMLEX Level 2-CE scores. However, no explanation exists in the literature as to why these courses are so pivotal to Level 2-CE. The authors hypothesize that CV II and REN II courses involve physiology, pathology, and pharmacology, all of which contribute to building up the capacities to understand and correctly diagnose diseases. Those capacities are assessed in Level 2-CE, which evaluates the application and integration of knowledge between foundational biomedical sciences and clinical medicine. Students who perform poorly in third-semester courses, especially CV II, may need early intervention to prepare for COMLEX Level 2-CE.
Furthermore, another third-semester course (PCM III) was deemed an essential predictor according to both regression models. This is because PCM III primarily teaches internal medicine, emphasizing the diagnosis and treatment of diseases. It parallels the finding that the National Board of Osteopathic Medical Examiners (NBOME) COMAT Internal Medicine and Emergency Medicine performances in the third-year rotation are the most significant predictors for COMLEX Level 2-CE because Level 2-CE emphasizes primary care as a generalist examination [10,11]. If students perform poorly in PCM III, they may need additional assistance when preparing for Level 2-CE.
To our knowledge, our current study is the first to investigate the effect of performance in each preclinical organ system course on the COMLEX Level 2-CE scores. Our regression models provide early prediction of COMLEX Level 2-CE scores. Our study has the advantage of predicting COMLEX Level 2-CE performance one to two years in advance compared to predictive models based on COMAT performance. Thus, students who perform poorly in CV II and PCM III in the third semester will have enough time to get assistance and ultimately improve their performance on COMLEX Level 2-CE.

Limitations
Although our regression models explain 46% of the variance in COMLEX Level 2-CE scores, there is still 54% of the variance that cannot be attributed to performance during the first two years, which could be attributed to performance in the last two years of clinical rotations. We did not include performance on clinical rotations, clerkships, or clinical subject examinations, as we aimed to generate predictive models based on preclinical academic performance. Our study is based on academic data from one single institution and may not necessarily apply to other institutions with different curricula. In addition, some medical schools have changed to a pass/fail grading system, and our findings may not be as valuable for a medical school with this new grading system.

Conclusions
There are significant correlations between COMLEX Level 1 scores, performance in the first two years of medical school, and COMLEX Level 2-CE scores. Among all courses, third-semester courses, especially CV II and PCM III, are the key predictors according to our regression models. Our regression models based on the performances during the first two years explain around 46% of the variance in COMLEX Level 2-CE scores. Our findings provide models for the earliest prediction of COMLEX Level 2-CE scores. At the end of the third semester, students who perform poorly in third-semester courses should receive assistance and intervention to better prepare them for COMLEX Level 2-CE that they must take one to two years later. In the future, investigating the effects of the third-semester courses on clinical subject performances may be necessary.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. The IRB Committee of Rocky Vista University College of Osteopathic Medicine (RVUCOM) issued approval IRB #2019-0079. Our research project, "Using Simulation Modeling to Predict Failure on COMLEX 1 and 2 at First Attempt Through a Longitudinal Investigation," was approved by the IRB Committee of Rocky Vista University College of Osteopathic Medicine (RVUCOM), and the IRB number was IRB #2019-0079. IRB determined the study was exempt. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.