Early Intervention and Remediation in Family Medicine Residencies: Behavioral Science Faculty Ratings of the Integration of Consensus-Based Recommendations

While remediation in undergraduate medical education has been studied extensively, a significant gap remains in understanding remediation during residency.¹ Remediation is challenging due to increasing awareness of learning difficulties,² growing prevalence of disabilities and accommodations in medical education,^3,4 and efforts to support historically underrepresented learners.⁵ The rising prevalence of mental health concerns among residents further complicates the learning environment.^6-9 In this study, we use the term remediation to refer to structured, programmatic responses to identified gaps in resident performance, functioning, or professional development that require additional support beyond routine educational processes. Early intervention refers to proactive efforts to address emerging concerns before they require formal remediation. Although universal criteria across programs are absent, residents needing remediation are commonly identified through clinical performance data, direct observation, professionalism concerns, or faculty feedback. The ethical challenges, lack of standardization, and privacy considerations surrounding remediation in family medicine residency training have contributed to a lack of outcome data, hampering the development of evidence-based process and intervention strategies.^1,10

Supporting struggling residents effectively remains a challenge for faculty and learners alike. The remediation process requires considerable faculty time and resources and can contribute to frustration and stigma among residents.^11,12 Transparent early intervention and competency based assessment may be one way to improve remediation for programs and for residents.^2,11,13,14 The Society of Teachers of Family Medicine (STFM) task force on Competency Based Medical Education (CBME) recently published its recommendations, which align with early assessment and intervention. Concordant recommendations included (a) foster a culture of reflective feedback conversations, (b) engage residents in self-regulated learning, and (c) create individualized learning plans with specific goals in the first 6 months of residency.^15,16

Building on prior work, our early intervention and remediation work group conducted a Delphi study to identify consensus-based best practices (CBBP) for early intervention and remediation among family medicine residency programs.¹³ That work established a conceptual framework grounded in expert agreement but did not assess the extent to which these practices were implemented across programs.

Because behavioral science faculty (BSF) are routinely involved in the identification, assessment, and longitudinal monitoring of residents with performance concerns in many family medicine residencies, their perspectives provide a pragmatic source of data for translating Delphi-derived best practices into measurable program-level processes. The present study was designed to assess early intervention and remediation related processes and infrastructure, using BSF observations as an initial source of data.¹³ Our objective was to develop and evaluate a questionnaire assessing BSF perceptions of early intervention and remediation processes with the ultimate goal of creating a tool for programs to evaluate and refine their early intervention and remediation methods.

We recruited active BSF to complete our survey (see Procedures section for details about recruitment). Seventy anonymous BSF respondents reported an average of 8.4 years of experience in residency education (SD = 4.2), with a range of 1 to 25 years. Fifty-four of the respondents were female, with no participants endorsing gender nonconforming. Sixty-four respondents indicated they were white, seven identified as Latino/Hispanic, and three identified as Asian. Respondents were from university-based(15), university-affiliated (15), community-based (38), and other (2) types of programs.

We used the final set of CBBPs identified through our prior Delphi study¹³ to create our current questionnaire. Each of the 11 final themes became questions using a Likert frequency to measure the degree to which the BSF observed each CBBP in their own program (Table 1).

We recruited BSF practicing in family medicine residency programs from a convenience sample of those attending two national STFM meetings in 2024. We used a QR code linking to an anonymous REDCap (REDCap Consortium) survey. The link to our survey also was distributed to the STFM family and behavioral health collaborative list serves monthly between May and October 2024.

Responses to individual questionnaire items revealed variability in early intervention and remediation approaches (Table 1). Based on the ratings of BSF, in most cases, approximately half of programs are engaging in CBBP consistently (often or always). Our results suggest that some CBBP are very common; most programs with BSF are reporting utilization of measurable steps. In contrast, some CBBP recommendations appear to be less commonly implemented (eg, identification of health resources, collegial conversations, and utilization of a broad array of resources).

We used χ² analysis to examine associations between program characteristics and CBBP items. We found one statistically significant association between program type and CBBP implementation: large programs with greater than 31 residents are more likely to report consistent documentation practices as compared to programs with fewer than 31 residents (P<0.02).

We conducted an analysis of the association between items using Fisher’s exact test (Table 2). Many of the items were statistically significantly associated with one another. Two items (measurable steps and normalizing challenges) were meaningfully associated with all other items on the questionnaire, suggesting potential for two factors that could identify elements of CBBP across programs.

We conducted a factor analysis to explore the structure of CBBP in early intervention and remediation strategies used in residency programs with BSF. We analyzed 11 items using a Promax rotation method, which assumes some correlation between variables. The analysis identified two distinct factors, accounting for a combined total variance of 5.21 (Factor 1 = 3.65; Factor 2 = 3.07). Because each of the 11 items contributes one unit of variance, a factor with an eigenvalue greater than 1.0 explains more variance than a single item. In this case, the two factors together explain approximately 47% of the total variance, indicating that nearly half of the shared response patterns cluster around two related underlying dimensions. All items met or exceeded our factor loading cutoff of 0.40, indicating acceptable alignment with their respective factor. Factor loadings represent the strength of the relationship (range 0.0 to 1.0) between each item and the underlying factor. Factor loadings are reported for each item (Table 3).

Factor 1 (organization and follow-up) included items related to structured remediation strategies with clear documentation, measurable goals, and predictable scheduling. Items loading strongly onto this factor included (factor loading in parentheses):

• Measurable steps (0.828): Throughout the early intervention and remediation process, we identify specific measurable steps for meaningful improvement (eg, SMART goals, milestone-linked thresholds) when creating intervention/remediation plans.

• Predictable schedule (0.805): When resident performance is a concern, we keep a predictable schedule of performance review and advising meetings, and create more opportunities for struggling learners to elicit feedback, seek support, and develop skills.

• Clearly document plans (0.775): For any resident on the early intervention and remediation continuum, we document plans clearly, including specific strategies, time-limited evaluation periods, and next steps if performance does not improve (ie, escalation).

• Elicit feedback (0.680): Once an early intervention or remediation plan is in place, we elicit performance feedback from faculty regularly to gain insight on progress and need for further intervention and escalation.

• Collegial conversation (0.417): When performance concerns are first identified, we begin with a collegial conversation between the resident and a familiar faculty member (eg, advisor) to elicit resident reactions to concerns and assess resident needs.

• Broad array, assessment (0.461): We use a broad array of sources to evaluate resident skills and performance so that we can identify struggling learners as early as possible.

Taken together, items that load onto organization and follow-up emphasize development of transparent, well-structured remediation plans that align with competency-based medical education principles. This factor highlights practices that ensure residents receive consistent performance feedback and actionable improvement strategies.

Factor 2 (people and development) captured elements related to fostering psychological safety, encouraging help-seeking behaviors, and creating a supportive learning environment. Items loading significantly onto this factor included.

• Normalizing challenges (0.852): We destigmatize help-seeking behavior by normalizing challenges during residency, modeling vulnerability, and creating a transparent, accessible process for accessing help.

• Identify key early experiences (0.754): We explicitly identify key early experiences in residency (eg, inpatient service) that provide data about essential skills.

• Faculty development, feedback (0.573): We incorporate faculty development in giving feedback and evaluating performance regularly.

• Train faculty, documentation (0.577): We train faculty to document concerns about resident performance.

• Health resources (0.439): We ensure that mental health and disability resources are in place.

In summary, items that load onto the people and development factor underscore the importance of culture, intentional training, and an understanding of human factors that contribute to both resident performance problems and effective faculty teaching.

The purpose of this study was to provide an initial step in developing a questionnaire assessing program-level processes and infrastructure for early intervention and remediation. We intentionally focused on the perspectives of BSF because they are frequently involved in the identification, assessment, and longitudinal monitoring of residents with performance concerns in many family medicine residencies. Our factor analysis revealed two distinct dimensions of program infrastructure related to CBBP for early intervention and remediation that provide a framework for evaluating process and infrastructure at the program level.

We found significant variability in the implementation of CBBP for early intervention and remediation. For example, while 82% of respondents always or often used measurable steps when creating plans, only 29% consistently provided health resource information. Providing health resource information possibly was not as consistently implemented because it might not be relevant in all situations.

Although we found what appeared to be differences between programs in their emphasis on documentation (ie, larger programs appeared to document more often than smaller programs), we found no other significant associations between program type and implementation of early intervention and remediation CBBP. The absence of other program-type associations suggests implementation gaps exist across all settings.

Our factor analysis identified organization and follow-up, and people and development as complementary but distinct factors of early intervention and remediation process and infrastructure. The organization and follow-up factor emphasizes systematic processes: measurable goals, predictable scheduling, and clear documentation. This factor reflects competency-based medical education infrastructure that prioritizes transparent, predictable process, including beginning with a conversation with the resident to engage them effectively (ie, collegial conversation).

The people and development factor focuses on human factors in early intervention and remediation, including faculty development, feedback, and identification of essential training experiences. We suspect that these programmatic conditions may be especially relevant when residents require early intervention or remediation, although they likely improve the learning environment for everyone.

This study represents an initial step toward a validated remediation assessment tool. The questionnaire demonstrated meaningful factor structure and item associations, suggesting utility for program evaluation and quality improvement. The two-factor model supports theoretical grounding for understanding early intervention and remediation. This approach dovetails with CBME and adult learning principles, pointing toward essential program infrastructure, faculty development, and supportive learning communities. We believe that residency programs could implement this tool for self-assessment, identifying specific areas requiring attention. Regular assessment could track improvement over time and guide targeted interventions. Longitudinal studies could establish relationships between practices and outcomes, providing evidence for specific intervention recommendations. Development of benchmarking data would help programs understand their performance relative to peers and further add to recommended tools and strategies for early intervention.¹⁴

As a preliminary study, these findings should be interpreted in light of several important limitations. BSF were intentionally selected as our focus given their longitudinal involvement in early intervention and remediation processes across many family medicine residency programs. As such, these findings do not yet reflect the perspectives of program directors, residents, or clinical faculty—key partners who should be involved in further psychometric validation of this questionnaire. Because participants were recruited through national conference meetings using a convenience sampling approach, the findings may overrepresent programs with strong behavioral science engagement and may not reflect practices in all family medicine residencies. The relatively small sample size (N = 70) limits generalizability. Self-reported data from single individuals may not accurately reflect practices in a program. This study did not evaluate the effectiveness of remediation strategies nor recommend specific interventions; rather, it described and measured program-level processes and infrastructure that support early intervention and remediation.

Effective remediation requires structured processes and infrastructure that recognize the human factors involved in residency education. This study presented a preliminary questionnaire to evaluate these practices from the BSF perspective. Assessing remediation outcomes across programs is methodologically and ethically challenging due to variation in definitions, documentation, and thresholds for formal remediation, and due to the sensitivity of remediation for residents and programs. For this reason, we focused on measuring processes and infrastructure as a necessary precursor to outcome-based research. Future studies should examine how these structures relate to program-level outcomes and evidence of successful early intervention and remediation connected to CBBP. The perspectives of program directors and core faculty should be included as well. Finally, we believe that future studies should endeavor to collect outcome data on evidence-based early intervention and remediation tools that are effective with residents, overcoming the ethical and methodological challenges inherent in this work.