Evaluating the Career Impact of Faculty Development Using Matched Controls

Faculty development (FD) is required for medical student and resident faculty.1,2 While the design and evaluation of FD programs has progressed,3-5 significant design and outcome gaps remain. Steinert, et al, argue that qualitative, mixed-method and control group designs would help understand program impact (eg, career impacts, building community).4,6 Therefore, we compared career development as perceived by graduates of our established, longitudinal FD program with matched nonenrolled controls.

Building on a 20-year history of primary care FD at the Medical College of Wisconsin (MCW),7-9 we sharpened our focus on clinician-educator career development with 80 hours per year of interactive classroom sessions between 2011 and 2016. Active participation and academic project completion were graduation criteria (Table 1).

Seeking comparison group evaluation design, a matched control approach was selected10,11 using structured interviews consistent with constructivist, grounded theory.12 The MCW’s Institutional Review Board reviewed the study protocol and found it to be exempt.

During 2011-2016, 33 physician faculty graduated from the FD program. Graduates were stratified by year (early=2011-2013; late=2014-2016) and specialty (family medicine, internal medicine, pediatrics). Ten graduates were randomly selected from these strata and matched to physician faculty controls according to gender, specialty, and years in academics. As the pool of non-FD participants was limited, we used three strategies to identify controls: (1) FD graduates’ referrals; (2) FD coleaders’ suggestions; and (3) departmental leadership recommendations.

Our evaluation team consisted of our five FD program faculty and a coordinator. All were experienced in FD evaluation using both qualitative and quantitative methods (Table 1).

Six interview questions were based on a clinician educator career development model13 that framed key elements associated with success: alignment of interests with workplace needs, acceptance of responsibility for career growth, and participation in academic communities. The interview protocol was piloted with five experienced nonstudy eligible medical educators, informing minor protocol revisions and interview length (Table 2).

After informed consent, the 10 FD graduates and controls were randomly assigned to an evaluation team member for a scheduled 30-minute telephone or in-person interview. Matched controls received up to two email invitations, with nonrespondents (more than 10 days) replaced according to the strategies above.

Interviews began with a short script reiterating the evaluation’s purpose and processes (eg, data deidentification, freedom to ignore a question), followed by the interview protocol questions (Table 2). Based on study team preferences and to limit recording/transcription costs that exceeded team resources, interviewers transcribed subject responses into a form that paralleled the protocol questions. The study coordinator labeled completed forms with unique identifiers and assured all responses were deidentified.

The interviewer, along with a randomly assigned second team member, independently analyzed each completed response form to identify passages associated with key career development elements.13 These study member pairs then met to resolve any disagreements.12,14 The full evaluation team then discussed and agreed on overarching macro themes and associated micro subthemes, identifying representative narrative passages.

FD graduate and matched control characteristics were similar (Table 3). Consistent with the career-development model-guided interview questions,13 four career development macro themes were common across groups, with notable differences between graduates and controls identified as micro themes (Table 4).

On the macro level, all defined success as training high-quality, competent physicians. At the micro level, indicators of academic success differed, with graduates’ responses emphasizing scholarship and academic advancement. Controls noted disinterest in scholarship and promotion, focusing on success as teachers.

At a macro level, all respondents noted shared departmental and personal interests in improving teaching and clinical goals. At the micro level, graduates reported greater alignment between personal and department goals related to medical education compared to controls.

All reported cultivating relationships and forming networks. However, at the micro level, graduates cultivated and sustained clinical and education colleagues through professional societies and academic projects, whereas controls’ colleagues grew mainly from “the bedrock (of) patient care.”

At the macro level, all respondents had growth strategies for teaching and leadership. They started or joined resident teaching initiatives, (rounds debrief, improving community preceptor retention) or through workshops. Both groups stressed the importance of politely saying no. “Learning my limits” improved after 2 years of faculty appointment. On the micro level, graduates aligned their education-oriented learning, conferences, and colleague networks with leadership roles.

Graduates and controls valued their clinical teaching roles with agreement across the career development macro themes. However, narratives differed at the micro level, with controls noting career paths aligned with clinical and/or important clinical teacher roles and achievements. Graduates reported greater success and growth as medical educators, placing high value on scholarly products, academic promotion through medical education, and enhanced academic colleague networks. These findings extend prior FD evaluation findings,6-9,15 and support recently articulated views that FD, scholarship, and communities of learners have advanced the evolution of clinician-educator careers.3,4,16

This study has several methodological limits. Graduates’ FD enrollment demonstrates interest in medical education, indicating possible selection bias. However, many controls reported participation in FD, typically external to our institution and of brief duration. Although the 10 graduates were randomly selected, our institution’s long-standing Primary Care FD program limited our precision in matching, due to the number of available faculty controls. Finally, while we were confident that interviewer scribing within a response template was accurate and thorough, this choice may have resulted in omissions and/or inaccuracies.

This is the first published study using matched controls to evaluate the influence of longitudinal, clinician-educator faculty development on graduates’ careers. While FD graduates and controls had some similar findings, key differences highlight FD graduates’ expanded scope and careers as clinician-educators. These evaluation results support continuing career development programming and investigation.