Survey Research, Family Medicine Style

Three papers in this issue of Family Medicine use the CERA survey methodology in their research. CERA, the Council of Academic Family Medicine (CAFM) Educational Research Alliance, was created in 2010 to build a research infrastructure for family medicine researchers. ¹ CERA conducts about five surveys per year designated for program directors, clerkship directors, department chairs, and residency faculty. ² The CERA steering committee screens potential survey topics and then collates all the accepted questions into one cohesive survey. Faculty submit proposals to have their questions included in the CERA survey. CERA mentors support faculty in the design of their questions to standardize the quality of the surveys and ensure that answers to the included questions will yield the information desired. The underlying principle is to limit and vet the surveys received by members of the family medicine educational community. Nearly 200 peer reviewed publications have come from CERA studies ² with many appearing in Family Medicine. STFM constituencies recognize and contribute to CERA and may be more likely to respond to a survey coming from a trusted source than they would surveys developed by individual researchers and shared across organizational list servs.

The three CERA studies published in this issue demonstrate the scope of topics covered in this type of research. The paper by Rowland et al ³ used the program director survey to assess the culture around evidence-based medicine (EBM) teaching in residency programs. The authors’ insightful findings highlight the positive attitudes toward EBM teaching in residency programs. Rebedew et al ⁴ examined responses to two CERA program director surveys (2017 and 2023) to compare the percentage of residents that needed remediation. The hypothesis of their study was that the pandemic led to increased rates of remediation among residents, but that was not found in their comparison. The third CERA-based paper in this issue focused on strategies and barriers to diversity, equity, inclusion, and antiracism (DEIA) work in family medicine departments. ⁵ This study used data from a CERA department chair survey and found that although most respondents believed that DEIA work was important, obtaining infrastructure funding to support it was challenging.

These three papers demonstrate the breadth of data available through the CERA survey methodology. The Family Medicine editorial team and reviewers benefit from reading papers that summarize CERA studies because they offer deep insight into important trends within the field. However, low response rates can impair the ability to make meaningful conclusions from these data and the Family Medicine editorial team notes declining CERA survey response rates in papers submitted to our journal.

Doing survey research is very popular in family medicine for several reasons.⁶ First, it is relatively inexpensive and does not require external grant funding. Several different software applications (eg, REDCap, Qualtrics) enable researchers to conduct online surveys and will collate the results. Second, when done virtually, a survey is a way to reach a large group of people nationally or even worldwide. CERA is an excellent example of a discipline coming together to create a venue for survey research to measure a wide variety of topics that are important to family medicine education.

Often the goal of survey research is to make inferences about a larger population based on data collected from a smaller sample, as collecting data from the entire population (a census) isn’t feasible. Measurement error describes the difference between observed responses and true values, and the job of survey researchers is to minimize sources of error, including :

1. Error due to the survey instrument (eg, misunderstanding the question, misunderstanding response choices, under or overestimating the phenomenon of interest, biased wording)

2. Error due to the selection of respondents (eg, sampling bias, nonresponse bias)

To minimize error due to the survey instrument, researchers conduct pretesting (informally soliciting feedback, establishing face validity) and piloting (more formally testing the survey outside the study sample) to correct issues before real-world data collection. Cronbach’s α and principal component analysis can be helpful tools for developing and validating survey scales. Each CERA survey includes a recurring set of demographic items followed by new survey items that have been pretested by content experts.

Minimizing error due to the selection of respondents is often more challenging. There are many ways to go about generating a sample, but sampling strategies generally fall under two categories: probability sampling (eg, simple random sampling, multistage sampling, cluster sampling, stratified sampling) and nonprobability sampling (eg, convenience sampling, snowball sampling, purposive sampling). Achieving higher response rates and larger sample sizes are common strategies to minimize measurement error, though these do not guarantee a sample is representative of the larger population. CERA’s sampling mechanism streamlines the data collection process for individual researchers and reduces the number of survey requests for constituents.

Once data are collected, sample statistics are generated to make inferences about the larger population. Most analytic methods assume simple random sampling, therefore more complex sampling designs require statistical adjustments before calculating population estimates. Descriptive statistics including frequency counts, means, and standard deviations are helpful to describe the sample. Cross-tabulations and other subgroup analyses can be used to identify trends among different groups of respondents. Correlations can be helpful to identify positive and negative associations among items. Statistical significance (ie, “Are results due to chance?”), effect size (ie, “What is the magnitude of the effect?”), and practical relevance (ie, “Is the effect meaningful in clinical practice?”) are all taken into consideration before interpreting results.

Using these key concepts in survey design and leveraging the research infrastructure CERA provides, CERA surveys can reach a broad family medicine community and help move the discipline forward. Please respond to the next CERA survey that comes to your inbox. If you have expertise in survey methodology, please consider getting involved as a CERA reviewer or mentor.⁷