FMAQ-S: Development of a Short Form of the Family Medicine Attitudes Questionnaire

In 2018, eight family medicine organizations pledged to work toward a goal: 25% of US medical school graduates specializing in family medicine by 2030.¹ Identifying students with positive attitudes toward family medicine is important for reaching this goal, since positive attitudes are associated with family medicine choice² and measuring attitude changes helps identify the impact of educational programs. Previously, we developed and validated the family medicine attitudes questionnaire (FMAQ).^2,3 A shorter version of this questionnaire, with similar psychometric accuracy, may be more useful.⁴

We used the Item Response Theory (IRT) Graded Response Model (GRM)⁵ to create a short-form version (FMAQ-S) of the FMAQ. We did not approach this study with a specific number of items in mind, but rather used an empiric process to shorten the instrument as much as was feasible, without deteriorating its quality. The original 14-item instrument assessed attitudes across multiple topical domains: importance, relationships, competence and expertise, research, lifestyle, and shortage.^2,3 In this analysis, favoring brevity, we approached attitude as a unidimensional construct and selected items with sound item parameters without seeking to balance the number of items by domain.

We reanalyzed data collected in 2017 from 1,188 fourth-year medical students attending 16 US medical schools. Complete details of the original data collection and student demographics were published previously.² This study was exempt from review by the Michigan State University Institutional Review Board.

We divided collected data into two groups: developmental and validation samples (594 students each). We used the developmental sample to identify items from the original FMAQ² that best differentiated attitudes toward family medicine. The validation sample was used to generate evidence supporting the validity of the new FMAQ-S. We used 𝟀² analyses to compare the two samples’ self-reported demographic characteristics (race/ethnicity, gender, age, income of family of origin, anticipated educational debt, hometown size).

IRT assumes all items within an instrument measure one single latent trait.⁶ Here, the latent trait (θ) is attitude toward family medicine. The GRM typically has two types of item parameters: discrimination (denoted as a) and difficulty (denoted as b) parameters. The item discrimination parameter indicates the degree to which a specific item differentiates individuals with different levels of a latent trait.⁷ The item difficulty (location) parameter (b_i) represents the boundary on the continuum of a latent trait for each response category i. Hence, for an item with five response categories (eg, from strongly disagree to strongly agree), there are four category boundaries and four corresponding difficulty parameters.

We first assessed dimensionality of family medicine attitudes using exploratory factor analysis (EFA), then conducted item analyses to identify items that more effectively distinguished attitudes and had higher levels of precision. Items’ discrimination parameters (a_i for item i) and item information functions (I_i[θ]) were used as selection criteria. We selected items with higher discrimination parameters, as a larger a indicates that an item better distinguishes attitude differences.⁸ We then assessed the item information function, choosing items that provided more information (precision) across the latent trait continuum (attitude). We selected items that provided the most information across the latent trait continuum to construct a questionnaire that captured a range of attitudes toward family medicine, rather than one with a maximum measurement accuracy at one specific cut score. We then compared the new and original questionnaires’ test information functions. To assess the new questionnaire’s validity, we analyzed the point-biserial correlation between average questionnaire scores and medical students’ intention to specialize in family medicine for both versions.

There were no statistically significant demographic differences between the two sample groups.

The Kaiser-Meyer-Olkin (KMO) test value for the original FMAQ was 0.83; values close to 1 indicate the interrelationship between items is adequate for EFA.⁸ The eigenvalues also supported the unidimensionality of the original FMAQ (Figure 1). The first factor’s eigenvalue was 3.50; the others were <1. The scree plot shows eigenvalues dropping steeply after one factor, meaning the first factor explained about 49% of variance. The initial FMAQ can therefore be considered unidimensional.⁹

Table 1 shows item analysis results using a GRM approach. Each item uses a 5-point scale, with responses ranging from “strongly disagree” to “strongly agree.” Thus, each item has four location parameters (b1, b2, b3, and b4) and a slope (a) parameter that indicate how well the item discriminates attitudes. Location b1 is the point where the probability of selecting “strongly disagree” or higher is 50%. Similarly, b2, b3, and b4 represent the locations where the probability to respond “disagree” or higher, “unsure” or higher, and “agree” or higher, respectively, are 50%.

The discrimination parameters for the 14 original FMAQ items ranged between 0.19 and 1.84. Items 2, 3, 6, and 12 had lower parameters (0.19, 0.96, 0.99, and 0.74, respectively) than other items, and provided less item information, (I<.3) across the latent trait’s points (Table 1, Figure 2). Thus, these four items were excluded from the FMAQ-S and the remaining 10 items were advanced to the validation phase.

Descriptive statistics and topical domains of the remaining 10 items are presented in Table 2. On the 5-point scale (strongly disagree/disagree/unsure/agree/strongly agree), means ranged from 3.38–4.36 and standard deviations ranged from 0.66–1.12. Cronbach α for the abbreviated scale in this sample was .78, indicating acceptable internal consistency, comparable to the .77 Cronbach α of the original instrument.

Figure 3a illustrates test information functions and standard errors of measurement for the new FMAQ-S. This questionnaire shows acceptable information and standard error of measurement levels (θ=-4~2), suggesting it can accurately assess attitudes toward family medicine. The FMAQ-S scale exhibits a similar test information function pattern as the original FMAQ (Figure 3b). A test’s information is the sum of the information function of all the items, so the total test information increases concurrently with the number of items. The reduced number of items in the FMAQ-S means that its slightly lower I values compared to the original FMAQ are acceptable.

We analyzed the correlation between the FMAQ’s mean score and students’ intentions to specialize in family medicine to assess the FMAQ-S’s criterion validity. We used point-biserial correlation since this correlation is between an approximation of a continuous-level and a binary variable. The correlation between the FMAQ-S mean score and students’ specialty choice is 0.378, similar to the correlation in the original FMAQ (0.383, P<.01), suggesting the FMAQ-S captures the relationship between scores and intentions well.

The 10-item FMAQ-S is a concise measure for assessing medical students’ attitudes toward family medicine. It has precision and criterion validity comparable to the original FMAQ and may be easier to use in research and practice. The FMAQ-S will be valuable for evaluating the impact of educational programs on students’ attitudes toward family medicine.   

Nevertheless, educational researchers should be aware of the instrument’s limitations. First, the instrument was developed using fourth-year students in the United States, and has not been tested among other populations. Second, the FMAQ-S was developed among students with a wide range of attitudes towards family medicine, and is most appropriate for use with students with wide-ranging attitudes.