Association Between Family Medicine Residents’ Mindsets and In-Training Exam Scores

An important goal of family medicine (FM) residency training is to foster a mindset of mastery for engaging in lifelong learning. A mastery mindset is a collection of beliefs and attitudes that result in a self-directed, persistent, and intrinsically motivated drive for knowledge.^1,2 Such a mindset is critical for keeping up to date with medical advances and responding to changing needs of patients.

In the competency-based medical education (CBME) approach that has recently gained traction worldwide,^3,4 learners are encouraged to take an active, self-directed role in their education. CBME emphasizes mastery learning and development of key competencies to improve patient care and safety.^3,4 In the context of FM residency training, residents receive ongoing, specific feedback on their progress toward competence through direct observation and individual coaching by clinical preceptors,⁵ thus supporting the development of a mastery mindset.

Despite the global implementation of CBME, research is needed to assess impact of competency-based learning on graduating residents.⁶ Empirical evidence, although scarce, indicates that CBME has a potential for identifying and remediating failing learners and shortening residency training.^6-8 Residents trained in an FM competency-based curriculum appear to be mastery-oriented.⁹ They also have lesser motivation to demonstrate competence to others for purposes of impression management (performance approach mindset) or being concerned with ego-protection (performance avoidance mindset).⁹ In light of this evidence and considering the mastery approach of CBME, the question arises as to whether there is an association between residents’ mindsets and their performance during residency.

Thus, the objective of this study was to investigate the relationship between FM residents’ mindsets and their performance on the American Board of Family Medicine In-training Examination (ITE).¹⁰ The ITE is a timed, computer-based assessment (240 multiple-choice questions) with good predictive validity.¹¹ The ITE assesses residents’ knowledge and skills needed to be a family physician and reports on areas requiring improvement. At the time of the study, our 2-year competency-based residency program used the ITE as a formative assessment of strengths and gaps in residents’ knowledge and clinical reasoning to help them prepare for certification examinations.¹² Although no or small positive associations between mastery mindset scores and scores on summative assessments have been reported in studies with university and high-school student populations,¹³ we expected a positive relationship between residents’ mastery mindset scores and their scores on the ITE, given its use as a formative assessment in our residency program.

This was a secondary data analysis of an FM resident cohort (N=85) at the University of Alberta, Canada. Figure 1 illustrates the study design. We collected data as part of two independent studies.^9,12 The ITE data came from the resident cohort in year 1, and then in year 2 as part of one study.¹² The mindsets data came from the same cohort of residents at the midpoint of residency training (before starting year 2) as part of another study.⁹ In that study, the Baranik and colleagues’ instrument¹⁴ was used to measure three types of mindsets in the resident cohort. Sample mindset items were: “I am willing to take on challenging work assignments that I can learn from” (mastery mindset; 4 items; α=0.72); “I prefer to work on tasks where I can prove my competence to others” (performance approach mindset; 4 items; α=0.82); and “I prefer to avoid situations at work where I might perform poorly” (performance avoidance mindset; 4 items; α=0.83).⁹ Higher average mindset scores were indicative of endorsing a respective mindset to a greater extent.

Following institutional ethics approval, we linked residents’ ITE scores in both years with their mindsets scores. Of 85 residents, 46 (54%) had complete data for the three data collection points, of whom 25 (54%) were females. Using SPSS 26.0, we performed descriptive and multivariate regression analyses. In the regression analysis, we entered residents’ ITE scores in year 1 and the three mindsets scores as independent (predictor) variables and residents’ ITE scores in year 2 as the dependent (outcome) variable.

Descriptive statistics and regression analysis results are shown in Table 1. Of the four independent variables, the largest predictor of residents’ ITE performance in year 2 was their ITE performance in year 1. Mastery mindset had a significant negative effect on ITE performance in year 2. Performance mindsets had no significant effects on residents’ ITE performance in year 2. Together the four independent variables explained 66.1% of the total variance in residents’ ITE scores in year 2.

The key study finding is that, although FM residents score high on a mastery-oriented psychological test,⁹ the mastery mindset appears to be negatively associated with their ITE performance. This observed negative relationship may be disconcerting at first sight; however, it is not surprising. In our residency program, which transitioned to CBME in 2009,⁵ clinical preceptors directly observe residents in their performance of specific competencies and provide actionable, improvement-oriented feedback to residents on a daily basis, thus supporting the development of the mastery mindset. However, the mastery mindset does not appear to transfer readily to residents’ ITE performance. We speculate that, although our residents were encouraged to use their ITE results in preparing for certification examinations,¹² residents likely would require explicit training on how to interpret ITE results and use them to develop an educational plan for self-directed learning.

It is also plausible that, despite the ITE being used as a formative assessment in our program, residents may perceive it as another hoop to jump through, rather than supporting their competency acquisition needs and long-term professional growth. Interviews or focus groups with residents are warranted to test these speculations and gain a deeper understanding of residents’ perceptions of the ITE and its practical implications for residency training to support residents’ mastery learning. 

Past research has also shown that mindsets are task and domain specific.¹ That is, one might possess a growth or mastery mindset in one domain or with respect to a particular task and a fixed or performance mindset in another domain or with respect to another task. The mindset questionnaire used in the study was general in nature in the context of workplace, whereas the ITE was task-oriented. As such, it is possible that residents may apply different mindsets to different tasks in different settings. Further qualitative studies testing this speculation are warranted.

With respect to study limitations, complete data had to be available for the three time points considered in the study (ie, ITE year 1 scores, mindsets data at the midpoint of residency, and ITE year 2 scores). Given that only 54% of residents in the resident cohort had complete data, it is unknown whether the observed relationships among the study variables hold for the entire resident cohort. Additionally, only one resident cohort was considered in this study due to the data availability. As such, we were unable to examine possible cohort effects. A longitudinal focused study is needed. A larger study of residents from different programs would also enable analysis by resident age, gender, and location of undergraduate medical education.