Protocol for the 2023 CERA Department Chair Survey

The Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) is a collaboration of the academic family medicine organizations (Association of Departments of Family Medicine; Association of Family Medicine Residency Directors; North American Primary Care Research Group; and Society of Teachers of Family Medicine) with the mission of improving and facilitating educational research in family medicine.¹ CERA functions as a centralized hub for generating rigorous and generalizable research in the area of family medicine education. CERA’s primary objectives are to facilitate medical education research, provide mentorship in research methods, and offer a clearinghouse of data for CAFM members.

CERA performs omnibus surveys of various membership groups of these academic family medicine organizations, including an annual survey of family medicine department chairs, an annual survey of clerkship directors, an annual survey of the general membership of CAFM, and twice-annual surveys of residency program directors. Members of CAFM organizations can submit proposals to be included in the survey, along with standard demographic questions. These proposals are chosen through a competitive peer-review process. The CERA methodology has previously been described in detail.²

Apart from administering surveys, CERA also provides its members with additional assistance and resources, particularly mentorship for aspiring researchers. Since the first surveys in 2012, CERA has resulted in over 207 conference presentations and 164 published articles.

In this paper we present the methods and demographic results of the 2023 Department Chairs Survey.

The call for proposals for questions to be included in the 2023 CERA Chair Survey was open from April 3, 2023 through May 9, 2023. This call resulted in 15 submissions which were each sent out to three peer reviewers. Each reviewer scored proposals on a scale of 1 to 5, with 5 as the top score, based on a scoring guide that includes relevance, significance, strength of hypothesis, quality of questions, and likelihood of being published. Scores were averaged across the three reviewers, and the top five highest-scoring proposals as rated by the peer reviewers were accepted for the survey. Table 1 lists the topics accepted for the 2023 CERA Department Chairs Survey.

Each survey team was assigned a volunteer mentor who assisted with development of the final survey questions. The questions were then sent out for pretesting for readability and clarity to former family medicine department chairs who were not part of the survey population. The survey was approved on July 28, 2023 by the Institutional Review Board of the American Academy of Family Physicians.

The survey, including questions from the survey teams and standard demographic questions, was sent to family medicine department chairs at medical schools in the United States and Canada as identified by the Association of Departments of Family Medicine (ADFM). This database contains two parts: (1) the membership data of the ADFM, and (2) a list of department chairs who are not ADFM members which is primarily identified from departmental websites. A total of 230 chairs were identified and received the survey via email through the online program SurveyMonkey (Momentive), with five reminder emails, from the period August 8, 2023 through September 15, 2023. One email was not valid, and one person each in the United States and Canada indicated that they were no longer a department chair, with no replacement chair identified. The total sample for the survey was 227 department chairs, 211 US and 16 Canadian.

Demographics of the sample pool are based on databases of the CAFM organizations and include current membership data as well as information on nonmembers

We calculated the number and frequency of location, gender, race/ethnicity, and age for the potential and actual respondent samples. We conducted the comparison of the proportions for each variable between potential and actual respondents using χ² tests. The calculation included only variables with more than zero respondents.

Overall, 119 chairs responded to the survey; however, five did not proceed after answering the initial demographic questions. These were removed from the result, bringing the number of respondents to 114. The overall survey response rate was 50.2% (114/227). There were no significant differences in the location, gender, race/ethnicity, and age between the potential vs actual survey respondents (Table 2).

The topics included in the 2023 CERA Chair Survey included a range of topics that are timely and important to the field of family medicine. The CERA framework fosters medical education scholarship by reducing obstacles to conducting original research; including the IRB approval processes, mentoring and refinement of survey questions, data collection, and data cleansing. Groups who have their survey questions accepted are provided with mentors who help with data analysis and manuscript preparation. Chairs also benefit from fewer requests for survey completion and knowing that they are contributing to well-designed research that will contribute to the specialty of family medicine.

An advantage of the CERA survey is its ability to reach a national audience of department chairs in a well-organized manner and to achieve a higher response rate than many individual surveys. The current department chairs survey's response rate of 50.2% this year aligns with previous rates, ranging from 48.4% (2022) to 54.4% (2019).

A limitation of CERA surveys is that each survey team is limited to 10 closed-ended questions. Authors must adapt the testing of their hypotheses to fit within this specific survey format. This limitation constrains the types of questions that can be posed through CERA surveys. The timeline of prospective researchers may also not fit into the annual CERA Chair Survey process timeline. Due to concerns of survey length and desire to maintain a high response rate, only the top proposals are accepted for each survey, which limits the number of researchers who can use the CERA process each year. This limitation may have a greater impact on less-experienced researchers, although the call for proposals does attempt to take experience into account.

The survey teams have sole access to the data from their accepted questions for 90 days. After this, the data are released in the online repository for all CAFM members. This is intended to incentivize survey teams to submit their projects for publication within that time frame, while also making the data available for secondary analysis, such as hypotheses that span the questions of two or more survey teams.