The Family Medicine Attitudes Questionnaire: A Valid Instrument to Assess Student Attitudes Toward Family Medicine

The United States has a severe and persistent shortage of adult primary care physicians.1 In this century, however, few US medical school seniors are choosing family medicine, with more family medicine (FM) residency positions being filled by osteopathic and international medical school graduates.2 For at least the last decade, less than half of the available FM residency positions have been filled by graduating US medical school seniors.3

There is a large body of literature describing factors influencing medical students’ specialty choices. Students who prefer or choose family medicine value ongoing relationships with patients, a whole-person approach to care, a broad scope of practice, and care for underserved populations.4-7 They place less emphasis on specialty prestige and compensation.4 Increasing student debt decreases some students’ willingness to pursue a primary care career.8,9 Students who choose family medicine are also more likely to come from lower income families, rural backgrounds, and underrepresented minority groups.10-13

Curricular experiences also influence student choices. Community-based experiences and the support of family physician mentors may increase student interest in family medicine,14 and the family medicine clerkship improves students’ attitudes toward, and interest in, family medicine.15 Programs with longitudinal, comprehensive curricular tracks, such as rural training tracks, have more family medicine graduates than programs with traditional block rotation curricula.16 Institutions with an explicit social mission also produce more students who choose family medicine careers.17

Although the shortage of family physicians has perhaps been most studied in the United States, it is a global issue. Studies in multiple countries have found that students view primary care careers as less prestigious than specialty careers.18-23 International studies have also found that the work environment of primary care is viewed negatively, demonstrating the need for all health care systems to promote satisfactory and engaging working conditions for primary care physicians.19,24,25

Medical student attitudes toward family medicine influence student likelihood of choosing family medicine and ultimately affect the viability of a primary care-based physician workforce in the United States. Student attitudes and intentions often are used as proxy measures for actual career choice in educational research; however, no comprehensive, validated instrument exists for assessing US medical student attitudes toward family medicine.

We previously described the development and preliminary validation testing of an instrument designed to assess US medical student attitudes toward family medicine26; however, this instrument had limitations. Items intended to assess student perceptions of family physicians’ time off and schedule control did not correlate with family medicine specialty choice, despite evidence demonstrating these concepts are important to student perceptions of lifestyle.6 The instrument also was validated only among students at two public medical schools in the Midwest, both with a relatively high proportion of students choosing family medicine.

We therefore sought to refine the instrument and assess its validity for a broader population of US medical students. Our purpose in this study was to validate an instrument that reliably identifies student attitudes toward family medicine.

The instrument was initially developed in 2010 and underwent preliminary validity testing and refinement in 2015 and 2016. It is structured to include items about family medicine across a range of topical domains, or subscales, including relationships, competence and expertise, lifestyle, research, importance, and shortage. Students indicate agreement or disagreement with each item using a 5-point scale.26

The instrument, including 17 previously validated items and the eight new items being piloted, was offered to a large population of fourth-year medical students in the spring of 2017. Surveys were sent beginning in late February, in order to capture students who had submitted rank order lists and thus made a final specialty decision. The authors sought colleagues at other medical schools who could invite their fourth-year medical students to participate. The authors contacted colleagues at the 2017 Society of Teachers of Family Medicine (STFM) Medical Student Education Conference and via listserv postings with follow-up by phone or email as necessary. Each medical student at participating institutions was sent three email invitations from an educational leader within their own institution with links to an anonymous survey. The survey included the study instrument, demographic information including gender, age, size of hometown, family income, anticipated educational debt, and race/ethnicity, and asked about specialty choice. Students were sent three invitations (one initial invitation and two reminders).

The authors created eight new items assessing student perceptions of family physicians’ time off and control over work schedules (four for each concept). These items were evaluated for content validity using a direct observation approach. Michigan State University College of Human Medicine students were recruited to complete the instrument, inclusive of all eight new items, while being observed by a research assistant (Fitzpatrick). Students were selected for diversity of year in medical school, age, hometown size, race/ethnicity, gender, family income, expected educational debt, marital status, and career interest.

The students completed the instrument individually. Each described to the research assistant their interpretation of each new item and their reasons for their answer choice. The research assistant took notes and audio recorded the session. Two investigators (Phillips and Prunuske) analyzed the notes and recording transcriptions using a modified content analysis approach. The investigators reviewed each question systematically, one by one. The investigators looked for evidence that students understood the questions, evidence of poor understanding, and consistency of understanding across medical students. Students who completed the direct observation were given gift cards.

We evaluated construct validity of both new and old instrument items using data reduction and factor analysis. Negatively worded questions were reversed for analysis purposes. We examined validity by iteratively evaluating factor loadings (Varimax) and calculating interitem correlations of subscales. Scree plot testing was used to determine the number of factors. We treated questionnaire scale variables as continuous and used Pearson correlations in all analyses.

We evaluated criterion validity by comparing student responses to choice of family medicine career. Questionnaires were scored by summing responses to each item retained in the questionnaire after internal and content validity testing. Negative questionnaire items were reverse-scored. Independent-samples t-tests were used to compare individual item responses and whole questionnaire scores with students’ choice of a career in family medicine.

Finally, we made efforts to shorten the questionnaire. Items with high interitem correlations (0.4) were iteratively tested for removal. We tested the items that were the lowest contributors to the component matrix (ie, explained the least amount of variance). Principle components analysis and scree testing were used iteratively to ensure the questionnaire’s explanatory power was retained as each item was removed. Cronbach a was calculated for the revised scale.

Items remaining in the instrument were summed to create a total score (negatively-worded items were reverse-scored). Within each subscale, remaining items were summed (without weighting) to create a domain score. Total scores of participants were evaluated for associations with demographic factors using Pearsons’ correlations (for continuous variables) and independent-samples t-tests (for categorical variables). Analysis of variance (ANOVA) was used to determine whether demographic differences in total scoring persisted after controlling for students’ choice of family medicine or another specialty. Binary logistic regression was used to evaluate whether the total score was predictive of family medicine specialty choice after controlling for demographic variables, including gender, age, size of hometown, family income, anticipated educational debt, race/ethnicity, and medical school. Scatterplots were created of total scores to look for natural score break point(s). Sensitivity, specificity, positive predictive value, and negative predictive value of the score were calculated.

The study was exempt from review by the Michigan State University Institutional Review Board. When possible, the demographic characteristics of participating students were compared to national samples of US medical students using χ2 goodness-of-fit tests.

Figure 1 summarizes methodology and results. Six medical students participated in the direct observation assessment. Based on this content validity evaluation, one new item was removed from the instrument because students found it to be confusing. All other new items were clearly understood by all students.

Sixteen medical schools participated, including eleven public schools and five private schools, across several regions of the country. One of these (Michigan State University College of Osteopathic Medicine) was an osteopathic medical school. In total, 1,188 fourth-year medical students submitted usable survey data (response rate=41.8%). The response rate at each institution varied from 27.7% to 54.5% (median response rate=41.0%). Demographics of respondents and all US medical students are shown in Table 1. Compared to all US medical students, respondents were more likely to be female, Caucasian, and plan family medicine careers (P<.01).

All items from the previously developed questionnaire were answered more favorably by students planning family medicine careers than students planning careers in other specialties (P<.001), as were the three retained pilot items measuring control (P<.05) and one pilot item measuring time off (P=.02). Three pilot questions measuring time off were not answered differently by students intending family medicine careers and were removed.

Principal components analysis was performed multiple times in an iterative fashion as the questionnaire was modified. Sample size was adequate for factor analysis (Kaiser-Meyer-Olkin Measure of Sampling Adequacy=0.876; Bartlett’s Test of Spherity <0.001; subject-to-item ratio=57:1). After removal of items failing criterion validity testing, principal components analysis revealed the presence of four components with eigenvalues exceeding one, cumulatively explaining 24.8%, 36.5%, 43.4%, and 49.2% of the variance, respectively. The Scree plot demonstrated a clear break after four factors. All retained instrument items contributed substantially to at least one of these four components.

Within each subscale, all interitem correlations were above 0.2. Twelve items were identified with high interitem correlations with other items (>0.4). These items had topical similarity to the correlated questions. For example, the three remaining piloted controllable lifestyle items had interitem correlations between 0.43 and 0.51. Six of these 12 items could be removed without degrading the variance explained by the instrument. One additional item was removed in order to increase the Cronbach a and improve the proportion of variance explained by the instrument. The final instrument contained four components with eigenvalues exceeding one, cumulatively explaining 26.8%, 37.0%, 45.3%, and 52.7% of the variance. Cronbach α of the final 14-item questionnaire was 0.767. Correlation matrix is displayed in the appendix (https://journals.stfm.org/media/2353/phillips-appendix1-may2019.pdf).

The final questionnaire includes items in six topical domains (relationships, competence and expertise, lifestyle, research, importance, and shortage). With one exception, responses of students planning family medicine careers had smaller standard deviations, indicating less variation in responses compared to students planning other careers. The questionnaire has a hypothetical score range of 25 to 70. Among all students, scores had a normal distribution. Among students planning family medicine careers, scores were also normally distributed, but most students had a score of 56 or higher (Figure 2). Among students planning FM careers, 78% had scores of 56 or above, but among students planning other careers, only 35% of students had scores of 56 or above. Thus, 56 was chosen as an arbitrary break point for sensitivity and specificity analyses. A score of 56 or above was 78.1% sensitive for a student choosing family medicine, and in this population, a score below 56 had a negative predictive value of 0.919 and a negative likelihood ratio of 0.218. However, this cutoff was only 65.3% specific; a score of 56 or higher had a positive predictive value of only 0.372 and a positive likelihood ratio of only 2.25. Figure 3 shows score distribution by percentile. Table 2 displays individual items, average scores for each item, and average scores for each domain.

There were no significant relationships between score and students’ race and ethnicity, anticipated educational debt levels, income of family of origin, or age. Women had slightly higher scores than men (55 vs 54, P<.01). Students who described themselves as being from rural communities had higher scores than those from other types of communities (self-described large city, small city, suburban, or small-town origin), but this difference was not significant (56 vs 54, P=.13).

In ANOVA analysis, a significant interaction effect was detected between size of hometown and choice of family medicine career; similarly, a significant interaction effect was detected between gender and choice of family medicine. Thus, two separate independent-samples t-tests were conducted to evaluate whether rural and nonrural students had different scores, and two separate independent-samples t-tests were conducted to evaluate whether male and female students had different scores. There was no relationship between rural status and score when the FM and non-FM samples were analyzed separately. However, the small effect of gender on score persisted. Among students interested in FM, women had higher scores than men (60 vs 58, P=.021); similarly, among students not interested in FM, women had higher scores (54 vs 53, P<.01).

In regression analysis, student medical school (P=.003), race/ethnicity (P=.014), and questionnaire score (P<.001) were independent predictors of choosing family medicine. Gender, age, anticipated educational debt, size of hometown, and family income were not independently predictive of choosing family medicine. The odds ratio for total score was 1.289 (confidence interval 1.223-1.347). This indicates that if all other factors were held equal, an increase in questionnaire score by one point increased the odds of choosing family medicine by 29%.

The Family Medicine Attitudes Instrument presented here reliably identifies US fourth-year medical students with a positive attitude toward family medicine. The survey includes domains addressing multiple aspects of student views of the specialty, and demonstrates both content and construct validity. Importantly, the overall survey demonstrates criterion validity, which allows identification of students that have an increased likelihood of selecting family medicine as a career choice. This valid and reliable tool for measuring student attitudes toward family medicine has the potential to evaluate the impact of learning environment, educational interventions, and criterion factors on student attitudes. We anticipate that this instrument will be useful in work that aims to increase the likelihood of students selecting family medicine as a career, ultimately strengthening the family medicine workforce in the United States.

However, this study has limitations. Participating institutions were located in the Midwest, East Coast and South, which may limit generalizability to Western or Pacific Coast institutions. The instrument may not be valid in assessing attitudes toward family medicine in other countries. Only one of the participating institutions was osteopathic, which may limit generalizability to osteopathic institutions. Future research should examine the validity of this tool in these settings. Our respondents also were less racially diverse than US medical students as a whole. However, we found no relationships between scores and student race and ethnicity, which should allow for application of this survey in more diverse populations of US medical students. Finally, because participation was voluntary, students with favorable views of family medicine may have been more likely to complete the survey than those with less favorable views, introducing the potential for participation bias.

The instrument needs further testing before it can be applied or used in some contexts. Because the questions have only been tested by fourth-year medical students, we do not know whether they will predict family medicine career choices among more junior students, or those who have not yet been admitted to medical school. Our content analysis indicates that these students will understand the items, but they may not be associated with ultimate specialty choice. A positive response to the questionnaire requires some knowledge of the specialty, the nature of research and patient care, and the health care system, which early students may not have. Future research should examine the predictive power of the Family Medicine Attitudes Instrument across the continuum of medical education.

We also do not know whether attitudes measured by the instrument are modifiable over the course of medical school. Some of the instrument items reflect as much about the student as the specialty. An example is the statement, “Family medicine provides a physician with enough income to live well.” This item is imbued with the students’ values and may or may not change with curricular interventions.

In conclusion, the authors have developed a validated instrument measuring medical students’ attitudes toward family medicine. The true utility of the instrument will only become evident with additional use and evaluation. The instrument and the scoring system have been published on the Society of Teachers of Family Medicine Resource Library and are freely available to the public for use.27