Protocol for the 2024 CERA Department Chair Survey

CERA (an acronym for the Council of Academic Family Medicine Educational Research Alliance) is a collaboration of academic family medicine organizations with the goal of improving educational research in family medicine.¹ This unique infrastructure provides a number of benefits including a higher survey response rate, pretesting of survey questions, fewer requests for survey completion for family medicine educators, and mentorship for newer researchers. CERA conducts surveys of the following subsets of family medicine educators: department chairs, residency program directors, clerkship directors, and general membership of academic family medicine organizations.

The chairs of departments of family medicine in the United States and Canada are surveyed on an annual basis. Members of the Society of Teachers of Family Medicine (STFM), the North American Primary Care Research Group (NAPCRG), the Association of Departments of Family Medicine (ADFM), and the Association of Family Medicine Residency Directors (AFMRD) can submit proposals for these surveys, which are then peer reviewed and subject to a competitive acceptance process. Accepted proposal teams are then assigned mentors, who assist with revision of survey questions, data analysis, and/or writing up results for presentation and publication. CERA provides the infrastructure for institutional review board (IRB) approval (through the American Academy of Family Physicians’ IRB), sends the survey to potential respondents, and collects the data. To date, CERA has resulted in 192 published papers and 233 conference presentations.² The purpose of this article is to describe the methodology for the 2024 CERA Department Chair Survey, including survey construction, data collection, and comparison of responders and the full population.

The call for proposals for the 2024 CERA Department Chair Survey was open online from April 1, 2024 through April 30, 2024. Proposals were solicited from members of STFM, NAPCRG, ADFM, and AFMRD through member listservs. The CERA survye team received 10 submissions which were sent out for peer review. Proposals were scored based on the following criteria: interest of the topic, background and significance, strength of the research hypothesis, and likelihood of publication in the medical literature. Reviewers are members of CAFM organizations with experience in research who have volunteered to be part of CERA’s mentor/reviewer database. Each proposal received three reviews, and the five scoring proposal submissions with the highest mean scores were accepted for the survey. Accepted survey topics are listed in Table 1.

Accepted survey teams were then assigned mentors through CERA. The mentors assisted with revision of the survey questions, which were then compiled into a draft survey. The CERA staff then sent out a draft survey for pretesting by members of academic family medicine organizations who were not part of the target population, primarily former department chairs. The survey teams were given the opportunity to revise their question sets based on the feedback of their module reviewers, pretesters, and mentors. CERA staff sent finalized survey questions to the American Academy of Family Physicians’ IRB for approval.

Family medicine department chairs were identified from the ADFM member database and respondents to prior CERA chair surveys. The ADFM database consists of two sections: (1) membership data from the ADFM, and (2) a list of department chairs who are not ADFM members, primarily identified through medical school websites. There were 213 US department chairs and 17 Canadian department chairs identified at the time of the survey. One department chair from Canada opted out of CERA surveys. One US and one Canadian email were identified as undeliverable. Nine people in the United States indicated they were no longer a department chair, and no replacement chair was identified. The survey was delivered to 218 department chairs (203 US and 15 Canadian) and was open from August 13, 2024 through September 20, 2024. The survey was sent via email using the SurveyMonkey platform, and six reminder emails were sent to nonrespondents.

The demographics of the population are derived from CAFM organizations’ databases. Thus, only medical school location, chair gender, chair race/ethnicity, and chair age are available for comparison. Demographic information was available from the CAFM organization member databases for 208 of the 218 potential respondents.

We calculated the number and frequency of location, gender, race/ethnicity, and age for both the population and respondents. To assess the level of potential bias from missing information from nonrespondents, we used a χ² test to compare the proportions of each variable between the two groups. Only variable categories with at least one population response were included in the analysis. The survey did include additional options for gender, but none of the participants chose these response categories.

A total of 118 responses were received. Seven survey responses were abandoned after answering only the first demographic question and were removed from the results, leaving 111 responses. The overall response rate for the survey was 50.92% (111/218), which is similar to response rates for the four previous CERA Department Chair Surveys (range 48.44%-54.4%). No significant differences were found in location, gender, race/ethnicity, or age between population and survey respondents (Table 2). Following administration of the survey, the CERA staff sends deidentified survey data to each research team.

The 2024 CERA Department Chair Survey demonstrates the advantages of the CERA infrastructure, including an adequate response rate and a representative sample of respondents. The 2024 CERA Department Chair Survey explored topics that are timely and important to family medicine as a specialty.

The CERA process has limitations, including a cap on the number of accepted proposals and a restriction allowing each accepted research team to submit only 10 questions. Both of these restrictions are in place to reduce cognitive load on the department chairs and maintain an acceptable response rate, which is important for the quality and generalizability of the research. CERA pretesters have estimated that it takes 15 minutes to complete the current CERA Department Chair Survey. In the case of the Department Chair Survey, a particular challenge is to maintain the list of department chairs, as there is no centralized database. It is possible that the population is incomplete, despite attempts to reach the entire population of department chairs. Additionally, the nonrespondents might differ in ways that are not reflected in the available demographic data. To our knowledge, CERA findings from prior surveys have never been challenged as displaying nonrespondent bias. Thus, these differences are unlikely to lead to bias in the responses to the survey.

Anonymized data from the survey respondents were sent to the initial submitters of each proposal, together with standardized demographic data. The proposal teams, each with an assigned CERA mentor, will have 90 days to analyze their data and prepare for presentation and publication. Following that initial 90 day period, CERA data are available to all CAFM members on the CERA website.³ These data can be used for secondary analysis and represents an additional opportunity for scholarship within family medicine.