Relationships Among Physician Vendor-Derived Proficiency Score, Gender, and Time in the Electronic Health Record

The electronic health record (EHR) has been widely implemented to improve quality of care. Other proposed benefits of the EHR include improved productivity, efficiency, and work-life balance ^{1, 2}; however, an understanding that the EHR also can contribute to physician burnout is growing. EHR modifications and training have been shown to alleviate EHR-related burnout. ^{3, 4}

Women physicians interact with the EHR differently than men, and they are also more likely to experience symptoms of burnout. ⁵ Women physicians spend more time in the EHR both during and outside of scheduled hours, document longer notes, and receive more messages than men. ^6-8 At one academic center, women physicians spent a mean of 33.4 more minutes in the EHR per day, which was equivalent to more than 3 additional 40-hour work weeks per year. ⁹

One strategy to mitigate EHR-related physician burnout is to improve efficiency through physician training—oftentimes through EHR customization and making the functionality more efficient. The modified tools aim to ease the burden of chart review, documentation, diagnosis selection, electronic inbox management, and order placement. Customization of EHR layouts, review filters, note templates, and order preferences is associated with improved satisfaction and efficiency with the EHR. ¹, ¹⁰ Personalized training sessions on such tools have been associated with improved efficiency and reduced feelings of burnout; ^{11, 12} however, usage of such tools at the physician level is variable. ^13-17

To provide a general assessment of EHR behavior, EHR vendors provide metrics that calculate how often providers use customization and efficiency tools. Epic Systems Corporation has one such vendor-derived score, termed a proficiency score (PS), that allows health care systems to understand how physicians use certain EHR functions and to target interventions. ^{18, 19} PS appears to correlate best with the extent of provider training and perceived EHR efficiency. Posttraining, a prospective randomized crossover study of tailored EHR coaching showed improvements in PS as well as in perceived EHR usability. ¹² PS has been shown to improve with more clinician training, which in turn is associated with greater appointment volumes and efficiency. ²⁰ The evidence is mixed as to how PS affects time in the EHR. ²¹ One study showed that pediatric generalists with higher PS had decreased EHR time while specialists had the reverse association. ²² Another study showed that increased PS among primary care clinicians was associated with more active minutes in the EHR on unscheduled days and after hours. ²³

Evidence shows that women have higher EHR efficiency, EHR satisfaction, and perceived EHR usability. ⁸ A study by Khairat et al documented a higher pretraining PS in women compared with men in a recruited sample of 34 pediatricians. ¹² In contrast, a later study of 133 surgeons from a single health care system that sought to determine provider characteristics associated with EHR proficiency found no significant gender differences in PS. ²⁴ Accordingly, gender differences in PS are not consistently replicated, and less is known about whether any potential differences are associated with differential time spent in the EHR. The purpose of this study was to examine gender differences between vendor-derived PS, as a surrogate for customization and efficiency tool use, in adult primary care physicians and medical subspecialists, and to examine the relationship between PS and time spent in the EHR.

We conducted a retrospective observational study of ambulatory EHR use for adult primary care and medical subspecialty physicians at an academic safety-net health care system. We excluded physician trainees from the analysis. EHR usage data for each physician was provided by Epic Systems Corporation from September 26, 2021, to September 24, 2022. Epic Systems defines usage data as mouse movements, mouse clicks, keyboard actions, and time spent on certain activities. We obtained physician demographic data from the MetroHealth System. These data originated from the physician onboarding process. Gender was self-reported by the physician during credentialing. This study was reviewed and approved by the MetroHealth Institutional Review Board (IRB #00000223).

A total of 228 physicians were included in the study. We excluded physicians with an average of less than four ambulatory visits per scheduled day during the reporting period. We also excluded physicians without a listed gender. For the purposes of this study, primary outcomes were defined as physician time in system per day, time in system per appointment, and PS. Time in system per day was a measure of average minutes logged into the EHR each 24-hour day and was calculated by the number of minutes spent in the system divided by the number of days logged into the system. Time in system per appointment was calculated by the number of minutes spent in the system during the reporting period divided by the number of completed appointments during the reporting period. PS was calculated by the sum of monthly vendor-derived proficiency scores divided by the number of active monthly reporting periods per provider. A vendor-derived proficiency score ranged from 0 to 10 and was based on usage of various EHR customization and efficiency tools (Appendix A). We elected to use the vendor-derived score as-is based on its evident ubiquity and use in several published studies. ^{12, 20, 22, 24-26}

Possible confounders for time in the EHR included physician characteristics, patient complexity, physician panel size, and system characteristics such as use of scribes and staff turnover. Specific confounders included in this analysis based on data availability were specialty, length of employment, proportion of days with appointments, and number of appointments per day. Specialty was defined as primary care (internal medicine, family medicine, medicine/pediatrics) and medical subspecialty (all other specialty types). Length of employment may affect a physician’s comfort with an institution’s EHR instance and was categorized as greater than or equal to 2 years, or less than 2 years since start date. The proportion of days with appointments was calculated by the total number of days with at least one appointment divided by the total number of days during the reporting period. The number of appointments per day was calculated by the number of appointments divided by the days with scheduled appointments during the reporting period.

Descriptive statistics were used to summarize the study cohort. We computed outcomes stratified by gender. Differences among demographics, time in system per day, time in system per appointment, and PS were assessed using the ꭓ² test for categorical and Wilcoxon rank sum test for continuous variables. Multiple linear regression was used to adjust any impact of gender on the primary outcomes for other confounders, including length of employment, proportion of days with appointments, number of appointments per day, and specialty type. To understand the possible association between PS and time in system, we included PS as an independent variable for EHR time in system per day and time in system per appointment regression models. Gender and specialty were included as an interaction type because researchers have hypothesized that gender differences in EHR usage may be impacted by specialty type. ²² Similarly, gender and PS were included as an interaction type to understand whether the association between PS and time in system varied by gender. Multicollinearity was assessed through calculation of variance inflation factors for all three models, and linearity assessments were conducted visually by way of partial regression plots for each predictor. Multivariable model coefficients are reported as adjusted regression coefficients in the context of multiple predictors in each. Secondary analyses were performed on time in orders, time in notes, time in chart review, and time in in-basket. All analyses were performed using RStudio version 2022.2.3.492 (Posit Software). P values were two-sided, and significance was defined as P<.05.

Descriptive statistics by gender (Table 1) show that a total of 228 physicians were included in the analysis; 122 physicians (53.5%) were women, and 106 physicians (46.5%) were men. Of the total, 141 (61.8%) were primary care physicians while 87 (38.2%) were medical subspecialists. Of the primary care physicians, 58.2% were women (82 of 141 physicians). Of the medical subspeciality physicians, 46.0% were women (30 of 87 physicians). Men were employed at this hospital system longer than women physicians, but this difference was not statistically significant (97.0 vs 77.5 months, P=.334). Women physicians had a higher median number of appointments per scheduled day, but this difference was not statistically significant (9.5 [IQR 6.6–13.5] vs 7.9 [IQR 6.1–12.2], P=.064). The median proportion of days with appointments during the reporting period was the same for women and men physicians (0.4 [IQR 0.2–0.5] vs 0.4 [IQR 0.2–0.5] respectively, P=.808).

Median PS and time in system by gender (Table 2) demonstrate that women had a higher median PS compared to men physicians (7.6 [IQR 5.8–8.6] vs 6.6 [IQR 5.4–8.1], P=.021). Women physicians’ median time per day was 30.6 minutes more compared to men physicians (150.5 minutes [IQR 102.5–182.2] vs 119.9 minutes [IQR 73.8–172.4], P=.013). Comparisons between time in specific EHR functions were calculated and reported in Appendix B. The difference in median time per appointment was not statistically significant for women physicians compared to men (24.7 minutes [IQR 18.4–35.6] vs 26.1 [IQR 19.3–34.7], P=.665).

Multivariable linear regressions (Table 3, Table 4, Table 5 ) were generated to model PS, time in system per day, and time in system per appointment. Predictors included gender, length of employment, proportion of days with appointments, number of appointments per day, and specialty type as possible confounders. We found no evidence of multicollinearity; all variance inflation factors were less than 2. Linearity assumptions were assessed and satisfied through inspection of partial regression plots. After adjustment, men had lower PS than women (–0.88, P=.022; Table 3). The remaining adjusted models showed no impact of gender on time in system per day or time in system per appointment (Table 4, Table 5). In these models, PS was a significant predictor of time in system per appointment (adjusted coefficient 1.46, 95% CI 0.85–2.07, P=.017), but not in time in system per day (adjusted coefficient 4.66, 95% CI 2.26–7.06, P=.053).

The purpose of this study was to further understand gender disparities in EHR use. We examined gender differences between EHR PS and time spent in the EHR system for primary care physicians and medical subspecialists. PS is an EHR vendor-derived score and reflects how a physician utilizes EHR tools in their workflow. In our cohort, women physicians had statistically significant higher PS than their men colleagues (7.6 vs 6.7, P=.021), indicating they used efficiency and customization tools at a higher rate. These tools were used to address electronic inbox messages, place orders, write notes, insert level of service, select diagnoses, and search the medical chart. Efficiency and customization tools are meant to improve EHR usability. Our finding that women physicians use these tools at a higher rate is consistent with prior work that has shown that women intensive care unit physicians had higher self-perceived usability of the EHR. ⁸

While women physicians have a higher PS, they spent 30.6 minutes more per day in the EHR compared to men physicians. This overall finding is consistent with similar conclusions in other studies. ⁹ Others studies also have shown that this disparity persists despite women physicians caring for slightly fewer patients on average than men. ²⁷ Proposed explanations include differences in communication and practice style, as well as gendered expectations for how patients interact with women physicians. ²⁸ Alternatively, men physicians may be less likely to rely on EHR customization tools, because they are more likely to use scribes or support staff to ease the burden of some EHR tasks such as note writing. ²⁹

Importantly, in our analysis, gender differences in PS persisted after adjustment for additional confounders, but gender differences in time in system per day and per appointment did not; however, our analysis may not have been adequately powered. Adjusting for other practice and provider-based variables reduced the impact of gender in both instances so that it was no longer statistically significant (Table 4, Table 5). This finding is at odds with prior literature. While our finding could suggest that gender differences in EHR time may be more attributable to other provider and practice factors rather than gender in isolation, we suspect that our analysis was not powered enough to be conclusive in that regard. Future studies with pooled data from multiple sites would be more definitive.

In our adjusted analyses, a higher PS was significantly associated with greater EHR time per appointment. While PS was not significantly associated with time in the EHR per day, the 95% confidence interval of the odds ratio was positive, and the P value marginal. In the context of the positive association with time per appointment, this signal is at least consistent with the former finding and raises concern for an underpowered analysis. The cause of these findings could not be ascertained in our study. While use of tools possibly may cause physicians to spend more time in the EHR, we believe this is unlikely. Instead, taking time to create and use these tools possibly could be an indicator of clinical burden. Increased use of proficiency and efficiency tools may be a response to unequal levels of burden in that physicians with additional patient care responsibilities are more likely to use these tools to close charts and respond to electronic messages. Additionally, use of these tools may be related to general time management strategies, patient panel characteristics, physician-patient relationships, and engagement with staff members or learners. Understanding this relationship is important as physicians’ time in the EHR continues to grow. ³⁰

While additional EHR time outside of clinic hours is associated with increased burnout, ²³, ³¹ considering whether additional time is a mark of quality care and clinical diligence is important. One study showed that increased time per day was related to improved ambulatory quality metrics, including hemoglobin A1C control, hypertension control, and breast cancer screening rates. ³² Another study showed that patients cared for by women internal medicine specialists had lower patient mortality and readmission rates compared to men. ³³ One possibility is that potential gender differences in time in system could reflect and contribute to differences in quality of care. The relationships among time in system, PS, and patient satisfaction or quality measures were not addressed in this study; however, this would be an important area of further inquiry.

Multivariable analysis did not show significant differences in findings associated with physicians who were primary care versus medical subspecialists. Given the variety of clinical workflow among different specialties and previous literature, we expected to see some interaction. ²¹, ²², ²⁴ Possibly, however, our analysis was underpowered.

Overall, our findings extend prior work by highlighting important gender-based differences in EHR proficiency assessments and overall utilization. Our results also support others’ that call into question the practical utility of the vendor-derived PS in assessing EHR proficiency. While PS appears to be a good surrogate for training and self-perceived proficiency, its impact on overall efficiency is less certain. ¹², ²¹, ²³ Importantly, our study adds more nuanced value to other findings by accounting for gender as an important confounder in EHR time and proficiency practices.

Interventions to increase physician EHR efficiency through customization training have been shown to improve perceptions of proficiency, EHR workload, and EHR usability, ¹² but they may not be sufficient to address EHR gender disparities and burnout. Instead, additional system policies and workflow changes to alleviate physician EHR burden should be studied and considered. With regard to interventions, equitable support resources, team-based care, EHR usability design, and artificial intelligence-based support are all additional possible avenues of exploration.

Our study is strengthened by a large cohort in a multiple practice health care system. This institution has used the same ambulatory EHR for more than 2 decades and has a mature training and optimization structure. Many health care systems use the same EHR vendor tools such as the PS to assess provider efficiency and customization, allowing this study to be generalizable. Limitations include that only a single health care system was studied, and we were unable to completely normalize the outcomes by physician load, such as overall panel size and patient complexity. The study was also limited by reliance on a vendor-derived metric (proficiency score) for the primary outcome. While the measure has been assessed in several studies, its construct validity as a marker of true EHR proficiency has been challenged. ²¹ Nonetheless, the measure’s consistent positive associations with user training programs and ubiquity by virtue of Epic’s relatively high EHR market penetration in the United States allows for some external generalizability and the ability for different sites to assess findings such as ours in their own settings.

Gender differences in PS highlight how women and men physicians may have different ambulatory EHR practices. Factors other than gender might better explain gender disparities in EHR time per day. The finding that higher PS did not correlate with less EHR time per day or appointment is discordant with the expected relationship between use of EHR optimization tools and time spent in the EHR. Multicenter studies will be needed to better understand the relationship between gender, EHR time in system, and EHR practice styles. Better-informed measures of proficiency with greater attention to construct validity are also paramount. Further areas of study include qualitative work on the impact of system policy and workflow changes geared toward equitably alleviating physician EHR burden. In addition to provider burnout, areas of study should continue to focus on how EHR usage impacts health care quality metrics and patient engagement.

This study was presented at the American Medical Informatics Association Clinical Informatics Conference, May 21–23, 2024, in Minneapolis, Minnesota.