"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead

Original article
peer-reviewed

An Evaluation of the Postgraduate Physician Assistant/Associate and Nurse Practitioner Orthopedic Surgery Fellowship and Residency Websites in the United States



Abstract

Background

Although considerable research on physician fellowship training program websites has demonstrated critical deficiencies in accessibility and content, there is no published study regarding the website content of physician assistant/associate (PA) and nurse practitioner (NP) orthopedic surgery postgraduate residency/fellowship programs in the United States. Therefore, this study aims to conduct a web-based content analysis to assess the available information on program websites and identify the potential areas for improvement.

Methods

The methodology for this study was replicated from prior research on graduate medical education (GME) websites. Twenty-two PA and joint PA/NP orthopedic surgery postgraduate residency/fellowship training program websites were assessed between August and September 2022. The criteria comprised 17 items related to postgraduate education and recruitment content. All program websites were evaluated for pertinent content relevant to prospective applicants applying to orthopedic surgery residency/fellowship programs. Descriptive statistics were performed.

Results

Out of the 22 PA and joint PA/NP orthopedic postgraduate websites evaluated, all had a functional website link. Only one orthopedic surgery postgraduate website met all 17 criteria. All program websites assessed contained program descriptions and contact information. Other information present on websites included program admission requirements 95% (21/22), salary/benefits 77% (17/22), interview requirement 73% (16/22), faculty listing 68% (15/22), journal club 59% (13/22), program objectives/goals 59% (13/22), and acceptance process 54% (12/22). Less than half of the program websites contained the following information: listing of current PA/NP postgraduate trainees, sample rotation schedule, orientation activities, curriculum, wellness/well-being strategy, graduation and research requirements, and link to program handbook. However, accredited PA orthopedic surgery program websites contained more content relevant to prospective applicants than non-accredited programs.

Conclusion

This is the first study to provide a detailed analysis of PA and joint PA/NP postgraduate orthopedic surgery residency/fellowship program websites. Not surprisingly, there is a wide variation in the available content on these websites. Study results provide further evidence to support the urgent need to improve program website content for prospective applicants.

Introduction

Physician assistants/associates (PAs) and nurse practitioners (NPs) often referred to as advanced practice providers (APPs) have been a linchpin of the American Health Care system since the 1960s. PAs are educated as generalists in the medical model, whereas NPs are educated in at least one of six population foci [1,2]. The growth of PAs and NPs is partly motivated by industry changes including projected physician shortages, expanded access to care, an upward trend in value-based payment models, and improved profitability among hospitals and health systems [3,4]. Like physicians, PAs and NPs practice in all areas of medicine. While not required for state licensure or entry-level clinical practice, PA and NP postgraduate specialty training usually (12 months in length) is available in a variety of medical and surgical specialties [5-7]. According to a 2021 report, 5,851 PAs have completed some type of postgraduate training program [8]. However, the clinical and net economic benefits of completing a PA or NP residency/fellowship program remain a subject of debate.

Postgraduate specialty training opportunities are designed to provide greater depth in a specialty or subspecialty area not covered in an entry-level PA and NP education program. Additionally, newly minted PAs and NPs may find these programs attractive because of the opportunity to hone their clinical skills or acquire new ones. For example, Hart and Bowen surveyed several hundred licensed NPs and reported that 90% of them felt either “well prepared” or “somewhat prepared” for practice upon completion of their NP program and expressed interest in transition-to-practice opportunities through a formalized training program such as a residency [9]. Although, there are several descriptive cross-sectional studies describing programmatic functions and educational characteristics of PA and NP postgraduate fellowship/residency programs [7,10-13], the effect of PA and NP formalized specialty training on patient outcomes is unknown. Notwithstanding the fact that PA postgraduate residency/fellowship training has been around for nearly 50 years and NP residency training for almost two decades, some prospective applicants may be unaware of how to obtain information about these specialty training opportunities.

Currently, there is no national resident matching program or centralized application-processing service available to licensed PAs and NPs interested in pursuing postgraduate residency/fellowship training [10]. Therefore, prospective orthopedic surgery PA and NP applicants often use the internet as a source of information when researching and applying for these specialty training programs. Hence, the primary aim of this study is to perform a website content analysis of PA and joint PA/NP postgraduate orthopedic surgery residency/fellowship programs within the United States.

Materials & Methods

A comprehensive list of PA and joint PA/NP postgraduate orthopedic surgery residency/fellowship programs was compiled from the following sources: the Association of Post Graduate Advanced Practice Registered Nurse (APRN) Programs (APGAP), the Association of Postgraduate PA Programs (APPAP) websites, a Google search strategy, and from prior research on PA and NP postgraduate orthopedic surgery programs [6]. The Google search strategy was conducted using the search terms such as “physician assistant orthopedic surgery residency” and “nurse practitioner orthopedic surgery residency” to identify additional programs. Only programs with a functional website were included in this study. After a complete list of PA and joint PA/NP postgraduate orthopedic surgery residency/fellowship programs was finalized, a second Google search strategy using the program’s name was conducted to determine the accessibility of each program’s dedicated website (Table 1).

Postgraduate Programs (n = 22)
Duke University Physician Assistant Orthopedic Residency Program
Mayo Clinic's Orthopedic Sports Medicine Physician Assistant Fellowship
University of California San Francisco Orthopedic Surgery Physician Assistant Residency
University of California Davis Advance Practice Provider Fellowship
University of Rochester Medical Center Advanced Practice Provider Fellowship Program in Orthopedic Surgery
Wake Forest Orthopedic Surgery Physician Assistant Fellowship
The Medical College of Wisconsin Nurse Practitioner and Physician Assistant Postgraduate Orthopedic Fellowship
Ohio State Advanced Practice Provider Orthopedic Fellowship Postgraduate Program
Texas Children’s Hospital Orthopedic Surgery Physician Assistant Fellowship Program
Atlanta and Mercer University Physician Assistant Orthopedic Surgery Residency Program
Illinois Bone and Joint Institute Orthopedic Surgery Physician Assistant Residency
Carilion Clinic Orthopedic Surgery Nurse Practitioner and Physician Assistant Fellowship
Northwell Advanced Clinical Provider Fellowship in Orthopedics
U.S. Army/Baylor University Orthopedic Physician Assistant Residency
Arrowhead Regional Medical Center Orthopedic Surgery Fellowship Program
Indiana University Health-Postgraduate Advanced Practice Provider Training Program
New England Baptist Hospital Orthopedic Surgery Physician Assistant Fellowship
Riverside University Health System Medical Center Orthopedic Surgery Physician Assistant Fellowship Program
The Steadman Philippon Research Institute Sports Medicine Physician Assistant Fellowship Program
Navy Physician Assistant Graduate Training: Orthopedics
Direct Orthopedic Care Physician Assistant Residency Program: Choosing Orthopedics as a Career Specialty
Orlando Health Orthopedic Advanced Practice Provider Fellowship

Other external sources of information outside of the program’s dedicated website were not examined due to the potential for outdated and/or incomplete information. Each program's website home page was scanned along with content searches within each website to identify the presence of 17 criteria based on two categories. The postgraduate recruitment category included a listing of faculty and current fellows/residents, admission requirements, program description, acceptance process, salary/benefits, contact information, interview requirement, and program objectives/goals. The postgraduate education category included journal club, sample rotation schedule, orthopedic curriculum, orientation activities, graduation and research requirements, resident wellness/well-being, and hyperlink to program handbook. All 17 criteria were weighted appropriately. For example, when a content item was found on the program’s website homepage and/or subpages, it was awarded 1 point, but if absent, it was awarded 0 points. The rationale for including certain criteria elements in this study is based on the findings from previous research. For instance, in one study, medical students applying to surgical specialties indicated that fellowship acquisition, faculty information, application contact information, and resident wellness are the most important website content [14]. Another study of interventional radiology program applicants indicated that applicants were most interested in website content that contained didactics and information about facilities [15]. A study of pediatric orthopedic fellowship program websites found that information about research, affiliated hospital information, and rotations were the most prevalent education criteria on program websites [16]. Consistent with the prior research on orthopedic surgery residency websites, the data elements from all PA and joint PA/NP program websites were uniformly collected into an extraction grid template for analysis, which was modeled based on the one developed by Sherman et al., 2020 [17].

This study does not require institutional review board approval since the information assessed is from publicly available websites and there was no contact with residency/fellowship programs. 

Results

The 22 PA and joint PA/NP postgraduate orthopedic surgery residency/fellowship programs are located in seven specific US census regions/divisions (Table 2).

United States Divisions/Region Number of Postgraduate Programs
South Atlantic 6
East North Central 4
Pacific 4
West South Central 3
Middle Atlantic 2
Mountain 2
New England 1
West North Central 0
East South Central 0

There was variation in the content covered across websites. Out of the 22 PA and joint PA/NP orthopedic postgraduate websites evaluated, all had a functional website link. Only one orthopedic surgery postgraduate website met all 17 criteria. All program websites assessed contained program descriptions and contact information. Other information present on websites included program admission requirements 95% (21/22), salary/benefits 77% (17/22), interview requirement 73% (16/22), faculty listing 68% (15/22), journal club 59% (13/22), program objectives/goals 59% (13/22), and acceptance process 54% (12/22). Less than half of the program websites contained the following information: listing of current PA/NP postgraduate trainees, sample rotation schedule, orientation activities, curriculum, wellness/well-being strategy, graduation and research requirements, and link to program handbook (Table 3). Additionally, one program had as few as three of the 17 (18%) criteria identified on their website.

Program Website Content Criteria Number of Websites, n% = 22
Program contact information 100% (22/22)
Program description 100% (22/22)
Admission requirements 95% (21/22)
Salary/benefits 77% (17/22)
Interview requirement 73% (16/22)
Faculty listing 68% (15/22)
Journal Club 59% (13/22)
Program objectives/goals 59% (13/22)
Acceptance process 54% (12/22)
Curriculum  45% (10/22)
Rotation schedule 32% (7/22)
Orientation activities 32% (7/22)
Research requirement 32% (7/22)
Residents' wellness/well-being strategy 27% (6/22)
Listing of current PA/NP fellows or residents 18% (4/22)
Graduation requirements from postgrad program 18% (4/22)
Program Handbook 14% (3/22)

However, 14% (3/22) accredited PA orthopedic surgery program websites contained more content criteria relevant to prospective applicants than non-accredited programs. Not surprising, given that external accreditation requires strict compliance with administrative and educational standards. Nevertheless, PA and NP postgraduate residency/fellowship specialty programs are not required to seek accreditation. Although not part of the original criteria, four websites (18%) included online video content, which offered a compelling view of postgraduate specialty training.

Discussion

This is a rare study to examine the accessibility and presence of website content pertinent to PA and NP orthopedic surgery residency/fellowship applicants. Usability testing and quality assessment of the information provided on program websites were not conducted. While the results of this study indicate that the most prevalent information on program websites were program descriptions, contact information, salary/benefits, and interview requirement, there were substantial gaps in education and recruitment content. Information details about wellness, graduation requirements, sample rotation schedule, curriculum, link to program handbook, and listing of current residents/fellows were least readily available. These important findings coupled with existing research on GME programs highlight the value of enhancing program website content, which may improve orthopedic surgery recruitment efforts [17,18]. Moreover, research seems to suggest that website content and quality may influence an applicant’s program choice. For example, a study of emergency medicine applicants showed that 41% reported not applying to certain programs based on information from their websites, and slightly over one-third of plastic surgery applicants stated that program website quality influenced their decisions to interview at a program [19,20]. Another study showed that 40% of prospective applicants to an internal medicine residency program found websites most useful when preparing for their interviews [21].

A call to action

Our world today is undeniably digital, where data, people, and devices converge. Information is readily available and highly sought after by prospective applicants. Therefore, postgraduate programs should regularly audit their website content to ensure their online information is easily accessible, loads quickly, and is up to date. As an immediate step, orthopedic surgery postgraduate programs should clearly articulate on their website for prospective applicants the minimum graduation requirements and essential knowledge, skills, and competencies that must be satisfied to earn the certificate of completion from the program. Also, program websites should include a listing of current fellows/residents along with faculty photos and biography that present faculty members' education and research interests to prospective applicants. This is underscored by a retrospective analysis using website analytics of an accredited emergency medicine residency training program, which found that the most popular content accessed was the “people” section of the website, which includes photos of current residents and faculty [22].

Additionally, postgraduate programs should display an example of a weekly rotation and call schedule on their websites for prospective applicants to help codify training expectations. This is extremely important as some applicants prefer lifestyle controllability over traditional extended shifts as burnout rates are reportedly higher among those undergoing overnight call rotations [23]. Furthermore, a link to a program handbook may help to demystify didactic and clinical expectations, program policies and procedures, approval of travel and vacation requests, wellness/well-being programs, remediation, and other pertinent information that could potentially assist prospective orthopedic surgery applicants in making informed application and enrollment decisions. Lastly, due to the absence of research on PA and NP postgraduate specialty training websites, this novel study may serve as a guide for orthopedic surgery PA and joint PA/NP postgraduate programs to improve and prioritize their online content. Moreover, other PA and NP postgraduate specialty training programs may find this study useful in examining their websites for improvement. 

Limitations

This study is not without limitations. First, data elements were initially completed by a single researcher on two separate occasions, which may have led to some degree of bias. To reduce bias, an independent reviewer blinded to the results of the first review extracted data on 12 websites (54.5%) to analyze the online search results. Any data collection discrepancies led to the author and independent reviewer reexamining the specific website to resolve any errors. Additionally, this study was conducted on websites from a single specialty, thus limiting generalizability to other PA and NP postgraduate specialties. Moreover, the content analysis did not account for any program website updates or development of any new orthopedic surgery residency/fellowship programs that occurred after the review period. Finally, 17 content items were chosen for this study, but there may have been other content items that PA and NP applicants also find important when considering whether to pursue postgraduate training such as facilities information, housing/neighborhood information, postgraduate job prospects, diversity statement, moonlighting opportunities outside of the program, and loan deferment options. 

Conclusions

Website recruitment and education content appear uneven across orthopedic surgery postgraduate programs. Given the variation in the online program information, there is an urgent need to optimize the program website content for prospective PA and NP orthopedic surgery applicants. Future research is needed to identify whether specific website content influences the decision of PA and NP applicants to apply to a postgraduate program. Moreover, whether prospective applicants search for residency/fellowship program information from online resources offered by APPAP or APGAP is an area of emerging interest. Also, an investigation into the quality, usability, and accessibility of PA and NP postgraduate residency/fellowship website content across all specialties is warranted.


References

  1. Institute of Medicine, Committee on the Robert Wood Johnson Foundation Initiative on the Future of Nursing, at the Institute of Medicine: The Future of Nursing: Leading Change, Advancing Health. National Academies Press, Washington, DC; 2011.
  2. Vasco Deon K: Redeployment of orthopaedic advanced practice providers at academic medical centers during the COVID-19 pandemic. Orthop Nurs. 2020, 39:215-7. 10.1097/NOR.0000000000000689
  3. Urbanowicz J: APRN transition to practice: program development tips. Nurse Pract. 2019, 44:50-5. 10.1097/01.NPR.0000605520.88939.d1
  4. Kidd VD, Liu JH, Reamer-Yu A, Wang JH, Deng M: The development of a visual dashboard report to assess physician assistant and nurse practitioner financial and clinical productivity. BMC Health Serv Res. 2022, 22:882. 10.1186/s12913-022-08216-7
  5. Kidd VD, Tankersley D, Virginia McCoy H: Postgraduate fellowships and residency programs. The Health Professions Educator: A Practical Guide for New and Established Faculty. Kayingo G, Virginia McCoy H (ed): Springer International Publishing, New York; 2017. 389-96. 10.1891/9780826177186.0036
  6. Kidd VD, Hooker RS: Postgraduate programs in orthopaedic surgery for physician assistants and nurse practitioners. Orthop Nurs. 2021, 40:235-9. 10.1097/NOR.0000000000000772
  7. Kidd VD, Vanderlinden S, Hooker RS: A national survey of postgraduate physician assistant fellowship and residency programs. BMC Med Educ. 2021, 21:212. 10.1186/s12909-021-02613-y
  8. Statistical profile of certified physician assistants: an annual report. (2020). https://www.nccpa.net/wp-content/uploads/2022/04/Statistical-Profile-of-Certified-PAs-2020.pdf.
  9. Hart AM, Bowen A: New nurse practitioners' perceptions of preparedness for and transition into practice. J Nurse Pract. 2016, 12:545-552. 10.1016/j.nurpra.2016.04.018
  10. Kidd VD, Vanderlinden S, Spisak JM: An analysis of the selection criteria for postgraduate physician assistant residency and fellowship programs in the United States. BMC Med Educ. 2021, 21:621. 10.1186/s12909-021-03059-y
  11. Kidd VD, Spisak JM, Vanderlinden S, Kayingo G: A survey of implicit bias training in physician assistant and nurse practitioner postgraduate fellowship/residency programs. BMC Med Educ. 2022, 22:598. 10.1186/s12909-022-03664-5
  12. Kesten KS, El-Banna MM, Blakely J: Educational characteristics and content of postgraduate nurse practitioner residency/fellowship programs. J Am Assoc Nurse Pract. 2019, 33:126-32. 10.1097/JXX.0000000000000341
  13. Kesten KS, El-Banna MM: Facilitators, barriers, benefits, and funding to implement postgraduate nurse practitioner residency/fellowship programs. J Am Assoc Nurse Pract. 2020, 33:611-7. 10.1097/JXX.0000000000000412
  14. Lambdin J, Lin RP, DeAngelis EJ, Vaziri K, Lin P, Lee J, Jackson HT: Analysis of surgery residency website content: implications during the COVID-19 Era. J Surg Educ. 2022, 79:904-8. 10.1016/j.jsurg.2022.03.006
  15. Charalel RA, Pua BB, Galla N, Trehan SK, Madoff DC: Interventional radiology fellowship website content: what is the relevance to potential applicants?. Clin Imaging. 2016, 40:1070-4. 10.1016/j.clinimag.2016.06.011
  16. Cohen SA, Shea K, Imrie M: An update on the accessibility and quality of online information for pediatric orthopaedic surgery fellowships. Cureus. 2021, 13:e17802. 10.7759/cureus.17802
  17. Sherman NC, C Sorenson J, M Khwaja A, L DeSilva G: The content and accessibility of orthopaedic residency program websites. JB JS Open Access. 2020, 5:e20.00087. 10.2106/JBJS.OA.20.00087
  18. Gu A, Lehman JD, Sardana A, Cohen JS, Richardson SS, Sculco PK: Adult reconstruction hip and knee fellowship program content and accessibility. J Arthroplasty. 2018, 33:1630-3. 10.1016/j.arth.2018.01.075
  19. Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ: Aspects of residency programs' web sites important to student applicants. Acad Emerg Med. 2005, 12:89-92. 10.1197/j.aem.2004.08.047
  20. Chen VW, Hoang D, Garner W: Do websites provide what applicants need? Plastic surgery residency program websites versus applicant self-reported needs. Plast Reconstr Surg Glob Open. 2018, 6:e1900. 10.1097/GOX.0000000000001900
  21. Embi PJ, Desai S, Cooney TG: Use and utility of web-based residency program information: a survey of residency applicants. J Med Internet Res. 2003, 5:e22. 10.2196/jmir.5.3.e22
  22. Dyer S, Dickson B, Chhabra N: Utilizing analytics to identify trends in residency program website visits. Cureus. 2020, 12:e6910. 10.7759/cureus.6910
  23. Block L, Wu AW, Feldman L, Yeh HC, Desai SV: Residency schedule, burnout and patient care among first-year residents. Postgrad Med J. 2013, 89:495-500. 10.1136/postgradmedj-2012-131743

Original article
peer-reviewed

An Evaluation of the Postgraduate Physician Assistant/Associate and Nurse Practitioner Orthopedic Surgery Fellowship and Residency Websites in the United States


Author Information

Vasco D. Kidd Corresponding Author

Orthopaedic Surgery, University of California, Irvine, School of Medicine, Orange, USA


Ethics Statement and Conflict of Interest Disclosures

Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Acknowledgements

A special thanks to Michael Burney, EdD, PA-C, Program Director/Chair, Clinical Associate Professor, MMS PA Studies Program at Chapman University, for conducting an independent review of program websites. 



Original article
peer-reviewed

An Evaluation of the Postgraduate Physician Assistant/Associate and Nurse Practitioner Orthopedic Surgery Fellowship and Residency Websites in the United States


Figures etc.

SIQ
-
RATED BY 0 READERS
CONTRIBUTE RATING

Scholarly Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.