Methods and Sample Representativeness of the Fall 2025 CERA Program Director Survey

The Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) is an initiative to advance educational research in academic family medicine.¹ Members of CAFM organizations may submit proposals for a 10-question survey. CERA’s general methodology and program director-specific procedures have been previously described.² The purpose of this report is to describe the methodology and demographic representativeness of respondents to the fall 2025 CERA Program Director Survey.

A total of 18 proposals were submitted for consideration to include in the omnibus survey. The proposals underwent peer review by three separate reviewers using a scoring rubric followed by a final review by the survey director. Surveys were then chosen for inclusion. Authors of accepted modules were paired with CERA mentors who provided guidance on refining survey questions. Following refinement, question sets (modules) were then submitted, and a draft survey was pilot tested by a group of family medicine educators not included in the target sample. Feedback was provided to study teams that were able to further revise questions. Following revisions, the CERA administrator constructed the final survey draft in SurveyMonkey (Symphony Technology Group, Menlo Park, CA). The omnibus survey included standardized demographic questions followed by the question modules. The American Academy of Family Physicians Institutional Review Board approved the project in October 2025 (Protocol 19-366 ORG A34).

CERA delivered email invitations to participate through SurveyMonkey. The fall 2025 CERA Program Director Survey was open November 12, 2025, through December 31, 2025. Nonrespondents received up to four weekly follow-up emails and a final reminder before the survey closed. Time stamps for each survey response were accessible through SurveyMonkey. After survey administration, CERA provided each team with the deidentified data specific to their survey module.

The survey sampling frame included all program directors of Accreditation Council for Graduate Medical Education (ACGME)-accredited United States family medicine residency programs identified from a publicly available ACGME list accessed in October 2025.

The Association of Family Medicine Residency Directors (AFMRD) provided summary demographic data as of October 2025 for a sample of potential respondents to enable statistical comparison with actual survey respondents.

We calculated descriptive statistics, including frequencies and percentages, for the following demographic variables: survey response time, education/training/degrees, gender, race/ethnicity, and self-identification as underrepresented in medicine (URiM). We assessed for estimated nonresponse bias by comparing the demographic differences between early and late responders.³ We defined early respondents as those who completed the survey within the first three reminders, and late respondents as those who completed it after three reminders. This provides an estimate of nonresponse bias as late responders serve as proxies of nonresponders. We used χ² goodness-of-fit tests, and Fisher’s exact test when appropriate, to compare the potential versus actual respondents and early versus late respondents, excluding the options of “No Response” and “Choose Not to Disclose” categories. Demographic categories that were not mutually exclusive were not included in the statistical analysis. In addition, AFMRD membership data reports degrees differently from CERA survey collection, limiting potential statistical analyses. We set the significance threshold of α=0.05 for all two-sided tests. All analyses were conducted using SPSS software (Version 31.0.0.0; IBM, Armonk, NY).

The scoring and peer-review process yielded six question modules for inclusion in the omnibus survey. The survey topics are listed in Table 1. The sampling frame was 707 residency program directors identified by the ACGME, of whom six indicated they had not yet had three resident classes, an inclusion criterion for the survey. This reduced the final sampling frame size to 701 residency program directors. A total of 313 program directors responded, giving a response rate of 44.7% (313/701).

The demographic characteristics of the sample were compared with the AFMRD membership data (Table 2). There were no significant differences in the sampling based on gender or race/ethnicity. The sample had a lower percentage of program directors who self-identified as URiM than in AFMRD membership data (11.2% v 16.1%, P=.031). Weighting for the undersampled URiM populations by study teams would be 1.43.

The survey was open for 60 days. Four reminder emails and one final call email were sent to nonresponding program directors. Figure 1 shows the frequency of responses obtained with each reminder. We assessed nonresponse bias by comparing early (≤3 reminders) and late (>3 reminders) respondents to the survey. There was no difference based on education, gender, or self-identification as URiM (Table 3). Black/African American respondents had a four-times higher rate of late responses than early responses (2.1% early responses vs 8.5% late responses). Meanwhile, Hispanic/Latino/of Spanish origin respondents answered early at nearly twice the rate of late respondents (7.2% early vs 4.2% late). Because race/ethnicity is not mutually exclusive, no statistical analysis was performed.

The fall 2025 CERA Program Director Survey provides survey responses that are pertinent to family medicine education, generally representative of the population surveyed, and do not have substantial estimated nonresponse bias. The survey response trend showed a normal decline with time. Each reminder had spikes of new respondents. Between these reminder intervals, a few new respondents answered. Currently, reminders are sent weekly, but reducing the interval between reminders is a reasonable option.    

URiM underrepresentation was statistically significant. In addition to analyzing sample representativeness, we also evaluated estimated nonresponse bias. Our analysis showed no significant difference between early and late responders, including in self-identifying URiM responders. Interestingly, we found a pattern in which Hispanic/Latino/of Spanish Origin responders had a higher rate of early responders, while Black/African American responders had a higher rate of late responders. These disparate response patterns of URiM respondents may be evaluated in future CERA survey cycles.