A Novel Learner- and Patient-Oriented Needs Assessment for a Point-of-Care Ultrasound Curriculum in Family Medicine

New Accreditation Council for Graduate Medical Education (ACGME) program requirements for family medicine graduate medical education¹ and American Board of Family Medicine (ABFM) procedural competency recommendations² encourage training in point-of-care ultrasonography, representing a shift to bring point-of-care ultrasound (POCUS) into primary care to improve the efficiency and accuracy of health care delivery. Existing studies have demonstrated the development of POCUS curricula in primary care specialties;^3-10 however, many of these studies focused on the perceived needs of learners without attention to the needs of the patients or communities in which these residents practice. In accordance with ACGME recommendations for residencies to provide care that meets the needs of communities,¹ we performed a needs assessment to inform POCUS curriculum development within a family medicine residency.

We employed a triangulated needs assessment approach to capture learner and patient needs in POCUS. Specifically, we analyzed patient needs through the family medicine practice (FMP) site imaging orders as an indicator of community demand, recognizing that patient interviews may not fully capture the utility of this underutilized tool. We paired this with a traditional learner-centered assessment by replicating established survey methodologies.^11,12 By integrating patient-oriented data with learner perspectives, we sought to identify convergences and divergences in training needs. We hypothesized that curricular priorities derived from these complementary sources would differ and that triangulating assessments would provide a richer, more inclusive foundation for POCUS curriculum development and inform future research in this topic.

This study occurred at an unopposed single family medicine residency in an urban community hospital affiliated with a large academic health system. Residents completed introductory didactics in POCUS and approximately two to four interactive POCUS workshops to improve their familiarity with POCUS modalities and techniques prior to the study. Musculoskeletal and skin infection workshops were the only specific modalities taught before the study. Institutional review board approval was obtained through Northwestern University (STU00220400). All data were analyzed using SPSS version 28 for Windows (IBM Corp).

The survey was developed de novo because no existing instruments capture POCUS needs in family medicine residencies. We reviewed the American Academy of Family Physicians (AAFP) POCUS curriculum guidelines¹³ and extracted 36 POCUS modalities to include in our survey instrument. Participants were asked to rate the importance of and their perceived skill in performing each modality on a 5-point Likert scale as performed in prior studies.^11,12 The skill gap for each modality was calculated as the mean difference between the importance and confidence ratings, with greater differences indicating important yet unfamiliar US modalities. The survey was generated in REDCap (REDCap Consortium) and distributed to all faculty and residents at the program via email over a 30-day period. All items were analyzed using descriptive statistics.

We analyzed imaging orders placed over a 1-year period (2023). Each order was categorized by imaging type (eg, ultrasound, CT, MRI) and clinical indication (associated ICD-10 code). We then determined whether the clinical question could be answered using basic or advanced POCUS modalities, as defined by the AAFP curriculum guidelines.¹³ Chart review was performed when these data were unclear. If the clinical question was not identifiable with the given information, it was excluded from the analysis (N = 98).

Of 24 eligible participants, 17 completed the survey (71% response rate). Survey participation represented all levels of training (five postgraduate year [PGY] 1, four PGY-2, five PGY-3, three faculty). Of the participants, 59% had some POCUS training prior to joining the residency program. We found were no statistical differences when comparing responses between different training levels. The three most important ultrasound modalities rated by learners were second and third trimester obstetrics ultrasound, assessment for joint effusion, and ultrasound-guided joint injection being. Skill gap averages are summarized in Figure 1. Assessment for pleural effusion and assessment for hemothorax had the highest average skill gap among the POCUS modalities (2.47 and 2.29, respectively), while hepatobiliary and skin/soft tissue categories carried the highest skill gaps among system groupings.

A total of 1,271 imaging studies were ordered at the FMP site in 2023, with the percentage of each imaging type summarized in Figure 2. Of these, 7.7% were excluded from analysis due to an unclear clinical question. Of the remaining 1,173 orders, 30% were found to be answered by a POCUS modality in the AAFP curriculum guidelines (18.7% basic, 11.3% advanced). Obstetrics and gynecology, hepatobiliary, and vascular studies were the most common imaging classifications identified, while evaluation for cholelithiasis, cholecystitis, and breast mass were the most common specific clinical questions for imaging studies, with more detailed analysis summarized in Figure 3.

POCUS is a tool that stands to enhance efficiency and accuracy of care in the primary care setting. New ACGME and ABFM requirements support this call for additional training and implementation within the field of family medicine. Our study demonstrated a useful methodology for multisource needs assessment when designing and implementing POCUS curricula in graduate medical education.

Collectively, our data illustrate how learner-perceived training needs and actual imaging order patterns only partially overlapped. Figure 1 shows that trainees identified the greatest skill gaps in evaluation of hemothorax and pleural effusions, whereas Figure 3 demonstrates that the most frequent POCUS-related clinical questions in the community involved hepatobiliary, obstetrics and gynecology, and vascular concerns. This study represents, to our knowledge, the first POCUS published needs assessment within a family medicine residency, and the first POCUS needs assessment to incorporate both learner perspectives and patient-oriented data.

This study highlights how established learner survey methodologies^11,12 differ from measure of patient needs in POCUS training. Updated ACGME requirements in family medicine emphasize a need for training that meets community needs, with our study suggesting that these needs may be at odds with traditional curriculum development strategies. Though confirmation of this trend in other settings is outside the scope of this study, both needs assessment strategies are practical methodologies that other residency programs may borrow in crafting their own curricula. Our study had limitations that will guide future study. First, learner self-assessment of importance and confidence may not correlate with more objective measures of learner performance.^14,15 Longitudinal surveys after further training, coupled with observed clinical competence, are necessary to ensure reliability of our findings and demonstrate effectiveness of POCUS curricula in closing skill gaps. Second, the learner survey questions did not specify a particular care setting. Given the broad spectrum of settings (eg, inpatient, outpatient, labor and delivery) in which residents and faculty practice, importance may be skewed by which practice setting participants envisioned while completing the survey. To address this concern, future surveys will explicitly reference care settings to determine whether context influences reported priorities. Lastly, we did observe higher importance ratings for musculoskeletal topics and high skill gaps in skin and soft tissue ultrasound modalities, which were workshops many learners were exposed to prior to the survey. The Dunning-Kruger effect¹⁶ may have altered our results; however other studies have called this effect into question.¹⁷

Given the benefits and underutilization of POCUS in the primary care setting, we suggest that family medicine residencies use multiple perspectives to prioritize POCUS curriculum design to satisfy new ACGME and ABFM requirements and implement POCUS in a way that immediately meets community needs for more effective and efficient care. Our triangulated approach (integrating objective imaging order analysis with learner-centered surveys) offers a unique and transferable methodology for balancing educational and community perspectives. As POCUS curricula continue to evolve, this dual approach will help educators decide how to prioritize training to more effectively implement POCUS into the practice of family physicians.

North American Primary Care Research Group Annual Meeting, Quebec City, Canada, November 20–24, 2024.