A Comparison of Family Medicine Resident Documentation Following Implementation of an Asynchronous vs Synchronous Telemedicine Curriculum

Prior to the 2020 COVID pandemic, the use of telemedicine among family physicians was limited; telemedicine curriculum in graduate medical education (GME) was even more scarce.^1-8 As practices transitioned to exclusively virtual care, the ACGME rapidly issued support for telemedicine in GME.⁹ Abruptly, mastering communication in a remote setting, virtual physical exam skills, and appropriate documentation of telemedicine encounters became essential.^10-13 In 2019, about 80 medical schools included telemedicine as a curricular topic; there was a lack of data for how many residency programs required telemedicine training.^14-16

In our residency program, the overarching need for physician training in telemedicine was noted as a common theme within a process improvement project.¹⁷ We created a curriculum focused on practicing communication and physical exam skills in the telephone only setting, and emphasized the correct components for documentation. Due to limitations within our health care system, virtual video visits are not available.

The need for telemedicine-unique training is acknowledged in the literature, and a recent publication of a needs assessment survey highlighted continued gaps in teaching telemedicine.¹⁸ Likewise, there is limited evaluation of effective curriculum.^19-24 Curricular assessment is vital to ensure learners are achieving the desired objectives. Assessment of the curriculum can occur at four levels as defined by Kirkpatrick: reaction, learning, behavior, and results.^25-28 Since the pandemic, curricular assessments, including one objective, structured clinical exam-based assessemnt, have focused on Kirkpatrick level one (reaction) results.^29-31

This study describes a rural, academic family medicine residency’s documentation of telemedicine visits and evaluates a just-in-time telemedicine curriculum delivered in two distinct (synchronous/asynchronous) forms in summer 2020. As part of curricular assessment, we examined the difference in behaviors between the initial asynchronous and subsequent, synchronous telemedicine didactics. To our knowledge, this is the first report of how training impacted documentation of a telemedicine visit, a Kirkpatrick level 3 (behavior) evaluation. Our hypothesis was that the behaviors of telemedicine care would be documented more often by learners who participated in synchronous didactics than learners who participated in asynchronous didactics, due to the feedback and deliberate practice components.

Upon the initial transition to telemedicine, a mandatory but unaccountable, asynchronous curriculum was rapidly created and disseminated to senior residents in April 2020. By July 2020, COVID-19 safety measures had been  instituted, and telemedicine curriculum was delivered synchronously to incoming interns. The goals and objectives as well as curricular components and educational strategies are outlined in Tables 1 and 2. Strategies including feedback and deliberate practice were utilized to strengthen curricular topics. Table 3 outlines some of the specific considerations for a telephone physical exam. 

We compared the documentation practice of asynchronous, self-directed learners to those of synchronous, faculty-directed learners during 1 month (September 8 through October 8, 2020) by randomly selecting three telemedicine visits for each physician. The data abstractors were three individuals who were involved in the creation of the curriculum and/or had directly participated in the curriculum. Each abstractor evaluated a subset of charts from each cohort. 

Abstractors reviewed each clinical note for five documentation behaviors taught in the curriculum. Data abstractors coded the clinical notes for presence or absence of five documentation factors: (1) consent for telemedicine (as a marker of communication), (2) time on the phone, (3) physical examination, (4) procedure code for telemedicine care, and (5) correct billing code. These documentation behaviors were evidence of the learner internalizing and implementing the objectives of telemedicine care. To assess physical exam practice, we coded on a three-level variable: complete absence, presence, or presence of complaint-focused exam. Table 4 provides examples. 

The Joint Base San Antonio-Lackland Institutional Review Board determined this project to constitute a quality improvement activity, not human subjects research. 

The sample included clinical notes from 11 interns (synchronous, faculty-directed version) and 22 senior residents (asynchronous, self-directed version) totaling 99 unique encounters. Two residents were excluded from the data set due to not having completed three virtual visits within this time frame.

In documentation by interns, 73% of encounter notes included consent to complete virtual care compared to 38% for seniors (χ² [1, N=99]=10.69, P<.001). Encounter notes written by interns included a focused physical exam 58% of the time compared to 13% for senior residents (χ² [2, N=99]=24.57, P< .001). The results of physical exam documentation are included in Table 4. Interestingly, the reverse was true for documenting length of encounter. Senior residents were significantly more likely to include documentation of length of the virtual encounter (21% compared to 0%, χ² [1, N=99]=8.15, P<.01). We detected no significant differences for documenting the billing or procedure code. Table 5 provides a summary of results.

In this study of documentation behaviors of telemedicine encounters, differences are notable between the two curricular versions. In the asynchronous version, there was no accountability for completion of the role plays, nor feedback provided for the deliberate practice unlike the synchronous version. This timely feedback was likely instrumental in honing the skill of performing behaviors that required communication with the patient (ie, physical exam and consent for care). Additionally, the accountability of the synchronous version ensured deliberate practice of telemedicine skills.   

Components of time documentation and coding guidance were disseminated in other emails outside the curriculum, likely influencing this behavior. The residents who received asynchronous training did so at the same time the coding guidance was emerging, thus an emphasis was made on including the time in the note. Furthermore, these were the senior residents who likely already established coding behaviors as part of their documentation patterns. By the time the interns arrived and received synchronous training, this coding emphasis had waned. For non-communication-based behaviors, emphasis outside of formal training (via email, in huddles, etc) may be necessary to lead to behavior change. 

As previously mentioned, this curricular project was implemented within an ongoing process improvement.¹⁷ Changes for our practice that came out of this curricular assessment included creation of templates for telemedicine visits (ie, consent, physical exam), annual synchronous training during intern orientation, faculty development, and direct observation of visits.

Of note, telemedicine curricula, including at a national level,³² have been developed in the past 2 years at a rapid pace; this method of sampled chart review provides a mechanism for evaluating other curriculum on a Kirkpatrick 3 (behavior) change.

Limitations of this study include the size of the chart sample, which was collected in one residency program across 1 month. Additionally, the charts reflected encounters of learners with different levels of clinical experience; although none of the learners had been conducting scheduled telemedicine clinic appointments (opposed to a short telephone call in response to a patient inquiry) prior to the pandemic. For this reason, the skill of telemedicine, especially physical examination and documentation, was new to all learners, and the interns (synchronous curriculum) showed more consistent documentation of telemedicine skills. Further limitation of the asynchronous curriculum was lack of accountability for completion. The limitations of a retrospective chart review such as the possibility of incomplete documentation by the physician are noted.

Future work should include faculty physicians in the curriculum delivery and evaluation to ensure they have the knowledge and skills to provide feedback to the learners. Further training and evaluation should include development of communication skills and associated documentation for discussing assessment and plan on the phone. Future research should investigate how telemedicine documentation affects patient outcomes.