Development of Clinical Judgment in Family Medicine Residents Using Digital Behavioral Health Tools: A Mixed-Methods Study

Digital behavioral health tools include mobile health (or mHealth) apps and digital cognitive behavioral therapy programs.^1,2 These tools can extend care and support self-management, but quality, evidence, privacy, and clinical fit vary.^2-6 Family medicine residents are expected to discuss these tools with patients despite wide variation in quality, evidence, privacy protections, and clinical fit. Residents therefore need skills to judge whether a tool fits a patient, condition, goal, and care context without replacing clinical relationships.

The Technology Acceptance Model (TAM) examines perceived usefulness, ease of use, attitudes, and intention to use.^7-9 Modified TAM frameworks can assess educational changes that precede adoption. Experiential learning and reflection may move residents from abstract familiarity toward applied judgment.¹⁰

Few studies have examined resident judgment about digital behavioral health tools during required training. I examined how a longitudinal curriculum influenced family medicine residents’ attitudes and evaluative judgment.

This convergent mixed-methods study occurred in one integrated family medicine residency during from 2023 to 2025. It examined resident perceptions and evaluative reasoning about appropriateness, evidence, ethics, usability, and patient-centered fit, rather than adoption or patient outcomes.

Participants were postgraduate years 1 and 2 residents completing required behavioral health rotations. Of 24 eligible residents, 19 completed the presurvey. Ten matched surveys contributed first-year app use or second-year implementation reflections. Some contributed both. Residents, rather than reflections, were the unit for theme counts.

The longitudinal curriculum was embedded in required rotations and didactics. A 1-hour faculty session covered evidence, privacy, ethics, patient selection, and the American Psychiatric Association App Evaluation Model domains: background, privacy and safety, clinical foundation, usability, and therapeutic goals or data integration.¹¹

During first-year training, residents selected one mHealth app for 14 day personal use. Selection reflected personal goals, clinical interests, prior experience, app store information, or an mHealth database. Apps addressed mindfulness, breathing, sleep, stress, journaling, or cognitive behavioral therapy. Residents reflected on selection, access, privacy, evidence, usability, therapeutic goals, limitations, and patient fit.

During second-year training, residents reviewed emerging evidence and implementation issues, then reflected on appropriate primary care use. Activities emphasized critical evaluation rather than routine adoption. Figure 1 summarizes the curriculum and TAM domains.

Resident attitudes were assessed using a modified TAM survey administered before and after the educational experience. Domains included perceived usefulness, perceived ease of use, attitudes toward use, facilitators, and intention to use digital tools. Items were scored on a 7-point Likert scale, with lower scores indicating more favorable responses.

Qualitative data consisted of written reflective learning assignments designed to elicit clinical reasoning, ethical considerations, and perceptions of appropriate use.

Quantitative analyses included 10 matched surveys. I calculated mean changes and within-person effect sizes (Cohen’s dz), interpreted descriptively because of the small exploratory sample. I inductively coded all reflections, compared residents, and drafted themes. Because I developed the curriculum and taught residents, confirmation bias was possible. A doctoral behavioral health trainee without teaching, supervisory, or evaluative roles reviewed codes, themes, the resident-level matrix, and excerpts. The trainee and I examined discrepant evidence, discussed how roles and expectations could shape interpretation, and refined themes by consensus. Each resident was counted once per theme. Within the results, we integrated findings through a joint display linking qualitative themes, TAM domains, quantitative patterns, and interpretations to examine convergence and contextual explanation across methods.

This study involved secondary analysis of de-identified educational data and was reviewed and determined exempt by the Atrium Health Wake Forest Baptist Institutional Review Board.

Following the educational intervention, I observed small increases in perceived usefulness and intention to use digital behavioral health tools. Other TAM domains remained largely stable (Table 1). These patterns suggest early attitudinal development without evidence of widespread endorsement or routine adoption of digital tools.

Five themes described how residents evaluated digital behavioral health tools. Experiential learning increased perceived clinical relevance for nine residents. Eight residents described how perceived value depended on the targeted condition or supporting evidence. All 10 residents identified privacy, cost, or access as relevant to app evaluation. All 10 also discussed how usability or engagement affected continued use. Six residents explicitly described digital tools as adjuncts to clinical care. The applications represented mindfulness, breathing, sleep, stress management, journaling, and cognitive behavioral therapy–informed approaches. Together, these themes show how residents weighed clinical relevance, evidence, ethics, usability, and care context when considering digital behavioral health tools. Table 2 presents resident-level theme representation and integrated findings.

The joint display contextualized survey changes. Qualitative findings showed broad attention to privacy, cost, access, usability, and engagement. However, fewer residents explicitly framed digital tools as adjuncts to care. This pattern suggests selective readiness, not broad enthusiasm or routine adoption. I defined selective readiness as willingness to consider digital tools when they fit the patient, condition, evidence base, and care context. The small changes in perceived usefulness and intention to use were consistent with this selective approach. The study did not measure residents’ actual clinical adoption of digital tools.

This report examined resident attitudes and evaluative judgment after a longitudinal digital health curriculum. Quantitative findings showed small changes in perceived usefulness and intention to use. Qualitative findings showed that most residents considered clinical relevance and evidence, while all residents discussed practical or usability concerns. Six residents explicitly framed digital tools as adjuncts to care. These patterns suggest increased attention to selective and contextual evaluation rather than generalized enthusiasm. This approach aligns with family medicine principles because it centers patient context, continuity, therapeutic relationships, and shared decision-making rather than treating technology as a stand-alone intervention.^10,12

Personal app use allowed residents to evaluate tools from user and clinician perspectives. Structured reflection prompted attention to evidence, privacy, cost, accessibility, usability, and patient fit. These findings align with prior digital health education research.^10,13,14 However, the findings represent residents’ written perceptions. They do not establish changes in clinical competence or actual technology use. Digital health curricula should therefore prioritize critical appraisal and contextual judgment while future studies evaluate behavioral and patient-level outcomes.

This study had several limitations. It involved one residency program and a small matched sample. Findings relied on self-report measures and required educational reflections. The study did not assess objective clinical adoption or patient outcomes. Attrition between surveys may have introduced selection bias because residents completing both surveys may have been more engaged with the curriculum or topic.

My involvement in curriculum development, instruction, and resident education also may have influenced coding and interpretation. Other reviewers had no direct teaching or evaluative relationships with residents, but their professional backgrounds and expectations about digital health could still have shaped interpretation. Residents completed reflections within a required educational context. Their responses may therefore have reflected social desirability or perceived faculty expectations. These limitations support cautious interpretation of the themes as residents’ reported perspectives rather than objective evidence of skill development or clinical behavior.

These findings suggest that family medicine residency curricula can incorporate structured app evaluation, personal use, and reflection when teaching digital behavioral health. Residents commonly considered evidence, privacy, access, usability, and clinical fit. Fewer explicitly described digital tools as adjuncts to care. Larger multisite studies should examine whether these educational approaches improve evaluative skills, clinical behavior, or patient outcomes.

The views expressed in this article are those of the author and do not necessarily reflect the views of Wake Forest University School of Medicine or Atrium Health.