Evaluation of Point-of-Care Ultrasound Training for Family Physicians Using Teleultrasound

Point-of-care ultrasound (POCUS) is a noninvasive tool used at the bedside to aid with diagnosis and is associated with improved patient outcomes and higher patient satisfaction scores. ^1-7 More recently POCUS has expanded into the practice of family medicine with the American Academy of Family Physicians (AAFP) publishing guidelines for incorporating POCUS into family medicine residency curricula. ⁸ Implementation of such a curriculum requires time, resources, and POCUS-trained faculty. ^9-10 In addition, the instructor and learner need to be in the same physical location. Teleultrasound, which is POCUS education via videoconferencing technology. ^11-13 is a way to circumvent some of these barriers to curriculum implementation.

The main objective of this study was to evaluate the ability of family physicians to perform and interpret abdominal aorta POCUS after completing a curriculum fully implemented using teleultrasound. Aorta POCUS was specifically chosen for this study because office-based screening of patients for an abdominal aortic aneurysm (AAA) is associated with decreased mortality and improved access to AAA screening, which is especially important in rural settings and practices with lower-income patient populations where there are barriers to preventative care. ^14-18

We collected data for this prospective, observational study from October 2020 to April 2022. An institutional review board deemed the study exempt.

We modeled the 8-week abdominal aorta POCUS curriculum (Figure 1) after the American College of Emergency Physicians and AAFP POCUS guidelines ^{7, 8} and the objectives after the indication, acquisition, interpretation, and medical decision-making (I-AIM) model for teaching and performing POCUS. ¹⁹

Physicians individually completed a presurvey and 10-question pretest. We assessed barriers to general POCUS using a 5-point Likert scale (1=not at all confident, 3= somewhat confident, 5=completely confident). Physicians also individually completed a pre-objective structured clinical examination (OSCE) and 1-hour, hands-on training session via teleconferencing technology. Lumify (Philips International BV) ultrasound machines with Reacts software (Philips Healthcare Informatics Inc) were used for teleconferencing. Instructors were ultrasound fellowship-trained faculty. During these sessions, each physician performed three examinations using the “see one, do one, teach one” Kolb method of learning. ²⁰

After hands-on training, physicians completed an online module that included a narrated lecture on aorta ultrasound, and they were required to complete a minimum of 10 independently acquired POCUS examinations on standardized patients and/or clinic patients. We did not collect patient data, and we deidentified and wirelessly uploaded the images for quality assurance (QA) review. Two emergency medicine faculty with extensive experience performing POCUS reviewed and graded the scans, using a using a 4-point QA scale (1=no identifiable structure, 2=some structures in view but not adequate for interpretation, 3=some structures in view and adequate for interpretation, 4=all structures viewed and diagnosis supported). This scale has been used in previous studies.^21,22 To meet criteria for approval, each examination required a minimum quality score of 3 and an accurate interpretation. A third reviewer assessed a randomized subset (25%) of examinations.

Postcurriculum, participants completed a postsurvey, test, and OSCE. These assessments were completed 8 weeks after the pretest. We evaluated curriculum effectiveness by assessing the images submitted for QA as well as pre- to postcurriculum change in knowledge (test scores), change in confidence and barriers, and change in psychomotor skill (OSCEs).

We analyzed differences between pre- and postassessments using Fisher exact, Wilcoxon signed rank, and χ² tests with a 5% significance level. We performed a participant-level, clustered linear regression to test whether the frequency of scans taken affects image quality. We used intraclass correlation (ICC) to determine agreement between experts about image quality. We performed all statistical analyses using SAS statistical software, version 9.4 (SAS Institute Inc).

Thirteen out of 16 (81%) completed the full curriculum; 6 of 13 (46%) were faculty with a median of 9 years in practice and a range of 3 to 24 years (Table 1). Twelve (91%) participants had little to no prior experience using ultrasound, and only one physician had performed one aorta ultrasound examination in the preceding 6 months.

Comparing pre- to postcurriculum responses, we found a significant reduction in self-perceived barriers to performing an abdominal aorta POCUS and improved confidence (Table 2, Table 3 ). All physicians (n=13) felt that learning how to perform an aorta POCUS exam was useful to their clinical practice both before and after the curriculum. After the curriculum, all physicians (n=13) felt confident with using the ultrasound machine and acquiring images, and 7 out of 13 (54%) identified no barriers to performing abdominal aorta POCUS. All participants rated the curriculum as good (n=4) to excellent (n=9), and 11 out of 13 (85%) felt confident in their ability to teach abdominal aorta POCUS to students.

Knowledge scores significantly improved from a median of 70% (range, 10%-90%) precurriculum to 90% (range, 70%-100%) postcurriculum (P<0.0001). OSCE scores improved from a median of 8 (range, 2-10) to a median of 10 (range, 7-10) out of 10 possible points (P=0.012). Overall, 211 examinations were performed with physicians completing a median of 13 examinations (range, 10-28). Median image quality was four (range, 2-4) out of four (Table 4 ). Image quality improved by 0.005 for each subsequent image acquired. Six examinations had an AAA present, and all were accurately identified. ICC between experts for image quality was 0.68.

Implementation of a POCUS curriculum requires a large amount of time and resources. ^{9, 10} Traditionally, POCUS training programs require the instructor and learner to be in the same physical geographical location. Utilization of teleultrasound is a way to circumvent some of these limitations, and, in this study, we found it to be feasible, with 81% of participants completing the full curriculum. Postcurriculum, 54% of physicians identified no barriers to performing abdominal aorta POCUS and 85% felt confident in their ability to teach aorta POCUS to students. Our study is similar to previous work on teleultrasound, which had found it to be a feasible means to teach ultrasound. ^21-23

There are several limitations that may affect the generalizability of this study, including a small sample size and likely selection bias, as those interested in learning POCUS self-elected to be involved. Participation was not required. Additionally, it is unknown whether the physicians performing the exams knew prior to examination whether patients with AAAs had AAAs. Thus, we did not determine diagnostic accuracy. However, the knowledge assessment and OSCE evaluated the learner’s ability to recognize and describe aorta pathology. Future studies will focus on diagnostic accuracy and how implementation of an aorta POCUS curriculum impacts office-based AAA screening and patient care. Lastly, the teleultrasound curriculum we implemented required a significant amount of time for the instructors to train and evaluate each participant. Future implementation of a teleultrasound curriculum could train multiple learners at once with one instructor and significantly decrease the time required by the instructor.

In this small study, family physicians showed improved knowledge and skill, with decreased barriers to performing and interpreting abdominal aorta POCUS. Importantly, family physicians were able to perform scans accurately and independently, which is crucial as physicians had to remember the skill over time and outside of the initial training period. The teleultrasound curriculum described in this study may be a viable alternative method to in-person instruction.