Clinical Questions Addressed by First-Year Medical Students in Primary Care: A Cross-Sectional Study

The longitudinal evidence-based medicine (EBM) theme was incorporated into McGill University’s undergraduate medical curriculum in 2013. As part of this curriculum, students in their first year of medical school shadow two family doctors for seven sessions each; they then must address a clinical question of their choice, based on a patient encounter, using the PICO format (patient, intervention, population, outcome) ¹.

Clinical questions are questions that health care professionals have concerning the care of their patient. Answering physicians’ clinical questions has an impact on clinical management and encourages students to hone skills central to EBM, such as acquiring, appraising, and applying medical evidence to patient care. ² Despite this recommended approach, physicians infrequently seek out information regarding their clinical questions. ^3-5 Ely et al created a revised version of a taxonomy for classifying clinical questions based on generic question types. ⁶ This taxonomy is purported to help in setting research priorities as well as to better equip point-of-care resources to answer clinical questions. ⁶ This taxonomy also has been used to classify clinical questions addressed by practicing physicians but, to our knowledge, has not been used to classify questions addressed by medical students. More generally, we know little about the clinical questions addressed by medical students when they observe encounters in a primary care environment.

To our knowledge, only four studies have categorized clinical questions addressed by medical students. ^{2, 7- 9} In a PICO-type exercise, students rotating through dermatology at the Denver Veterans Affairs Medical Center were asked to formulate a self-generated, patient-specific question in a clinical setting and to categorize their question as belonging to one of eight categories: therapy, diagnosis, prognosis, etiology/risk factors, harm, prevention, disease management, or other. ⁷ Stanford University implemented an EBM assignment in their pediatric clerkship rotation in which students had to formulate a clinical question based on their experience. Questions were classified as belonging to one of six types: therapy, prognosis, diagnosis, harm, etiology, or other. ⁸ In the study performed at the University of Pennsylvania Perelman School of Medicine, students were asked to answer clinical questions in the context of an EBM curriculum using educational prescriptions and to categorize them by the following types: background questions concerning general information (eg, the pathophysiology of a disorder), therapy, or diagnosis. ² In the study performed by Osheroff et al, which analyzed information requests in a health care setting, the subjects of the information requests were categorized according to information about a specific patient (history, physical exam findings, lab studies), disease therapy (including drug therapy), differential diagnosis, and other. ⁹

Frameworks for question classification from the four studies just described contributed to a multilevel framework that we used to characterize clinical questions addressed by an entire class of first-year medical students at McGill University. We also explored the content of the clinical questions addressed by these students in the context of primary care. Our findings will be used to inform future iterations of the PICO project in the undergraduate EBM curriculum.

This was a cross-sectional study that assessed all clinical questions submitted in 2023 in the context of a longitudinal EBM curriculum. During their first year of preclerkship studies, medical students are required to formulate and address a clinical question based on a clinical encounter they observed in primary care (Appendix A). Students are required to search for an answer to their clinical question using either a preappraised resource or an unfiltered source such as PubMed. A preappraised source is one that allows access to medical evidence, such as systematic reviews that have been critically appraised, whereas an unfiltered source provides access to primary research in academic journals. We assessed and classified students’ clinical questions according to frameworks we identified a priori in a literature search for other studies that categorized clinical questions.

To identify classification frameworks for question categorization, one author (A.Q.), a health sciences librarian, constructed a literature search and then ran it in MEDLINE and Embase on June 12, 2023. A list of terms used in the search strategies for both databases is provided in Appendix B. Two of us (M.R. and P.M.) then extracted data and summarized it from full-text articles reporting any type of evaluation of clinical questions. A total of 1,122 records were retrieved, of which 13 full-text articles met inclusion criteria, and four ultimately were included. Among these articles, the most frequent method used to classify clinical questions was based on the type of question defined as therapy, prognosis, diagnosis, etiology and harm. ^{7, 8} Based on the comprehensibility of this framework, we decided to incorporate and modify this classification scheme for the present study by replacing therapy with treatment and prevention and adding the category of screening. Two additional classification schemes were recommended by coauthors A.Q. and R.G.

Based on our literature search, we attempted to match each clinical question at three levels: (a) type of question (screening, diagnosis, prognosis, treatment [including preventive treatment], etiology, and harm); (b) classification using Ely’s taxonomy; and (c) question topic as described in the 105 priority topics of the College of Family Physicians of Canada (CFPC).

We categorized clinical questions using terms such as screening, diagnosis, prognosis, treatment, etiology, and harm to describe the type of question addressed by the medical student.

Ely et al described a taxonomy that can reliably categorize clinical questions into 10 types of generic clinical questions. ⁶ This taxonomy was derived from 295 questions asked by Oregon primary care doctors. We attempted to classify students’ clinical questions as belonging to one of these 10 generic clinical question types.

The CFPC oversees the certification of family physicians in Canada. The 105 priority topics comprise a list of medical topics and associated key features that family physicians should be able to address and that practitioners who intend to challenge the certification examination for licensing purposes can reference. ¹⁰ To classify our student questions based on these 105 topics, we began by assessing the population (P) element of the PICO clinical question. If P did not map to one of the priority topics, intervention (I) and outcome (O) elements were then mapped. We allowed for a maximum of two priority topics to be matched to each clinical question.

A total of 180 student projects were split equally among two reviewers. The reviewers classified questions according to each of the frameworks described earlier. The review of all PICO projects was conducted independently following a pilot test involving 10 projects. For this pilot, we found an almost perfect interrater agreement for Level I classification (Cohen’s κ=0.93, 95% confidence interval [CI] 0.87, 1.00) and fair agreement for Level II classification (Cohen’s κ=0.25, 95% CI –0.56, 0.564). The pilot test revealed the feasibility of applying all three levels of classification to the PICO projects independently. Clinical questions that were ambiguous or unclear were flagged and discussed at a meeting involving both raters and arbitrated by a third author (R.G.).

The research ethics requirement for this project was waived by the McGill University Institutional Review Board.

Of 180 clinical questions, 152 (84.4%) were classified under the category of treatment/prevention (Figure 1). Seven out of 180 (3.9%) questions were categorized under etiology, six (3.3%) under prognosis, six (3.3%) under screening, and five (2.7%) under harm. Only four out of 180 (2.2%) questions were classified under diagnosis.

Most questions (105/180, 58.3%) fell under the category “How should I treat condition x (not limited to drug treatment)?” (Table 1). This was followed by “How should I manage condition x (not specifying diagnostic or therapeutic)?” with 45/180 (25%) questions. No questions were assigned to the categories of “What is the cause of symptom x?” nor “What is the cause of physical finding x?”—aligning with the few questions pertaining to diagnosis at Level I.

At this level, the most common priority topic represented in student questions was “in children” with 18 (10%) questions, followed by pain (16, 8.9%), pregnancy (12, 6.7%), depression (11, 6.1%), behavioral problems (10, 5.6%), chronic disease (9, 5%), diabetes (nine, 5%), loss of weight (9, 5%), uncategorized (9, 5%), and obesity (7, 3.9%; Figure 2). We noted a degree of overlap in question classification. For example, of the 18 questions assigned to the topic “in children,” six also were assigned to the priority topic behavioral problems, and four were assigned to the priority topic earache.

Nine questions were uncategorizable according to the CFPC priority topics (Table 2). Of these, two pertained to inflammatory bowel disease or irritable bowel syndrome, and two were concerning hair loss. Forty-one of the CFPC priority topics were not represented in any of the student questions (Table 3).

Classifying clinical questions addressed by medical students using a multilevel framework is feasible. Most student questions were on treatment and prevention. A few of the other types of questions were addressed in these PICO projects, but at a very low frequency.

Other studies also have demonstrated that classification by question type is feasible. In terms of the first level of classification according to question type, 84% of questions addressed by students in this study fell under the category of therapy/prevention. By comparison, at the Denver VA Medical Center and at Stanford University, 61% and 62% (respectively) of clinical questions were categorized under therapy. ⁷, ⁸ Note that in both of these studies, the participating students were in their clerkship years, whereas the present study involved preclerkship students in their first year of medical school. In addition, the present study combined questions about prevention with treatment, rather than using the category of therapy. This factor may help explain the increased proportion of therapy questions observed in this cross section of student responses in comparison to studies that separated therapy and prevention. The decision to combine therapy and prevention was in recognition of the fact that preventive therapies are interventions to avoid adverse health outcomes (ie, vaccination, risk-lowering medications, prophylactic treatments). Our findings suggest that in the context of a PICO project, students prefer asking questions about therapy, including preventive therapy, rather than those concerning screening or other question types. In keeping with other studies, the categories of prognosis, diagnosis, harm, and etiology were underrepresented in our cross-section of clinical questions. ^{7, 8}

The second level of classification, Ely’s taxonomy, demonstrated internal consistency with the first level of classification in that the highest scoring categories addressed treatment and prevention: “How should I treat condition x (not limited to drug treatment)?” followed by “How should I manage condition x (not specifying diagnostic or therapeutic)?” No questions were categorized under “What is the cause of symptom x?” nor “What is the cause of physical finding x?”—which can be considered background questions that involve general information about a disorder, as defined by investigators from the University of Pennsylvania Perelman School of Medicine, where only 6% of students’ questions fell under this category. ² The few questions assigned to these generic types from Ely’s taxonomy align well with our finding that students rarely address questions about diagnosis or etiology. When Ely’s taxonomy was used to classify 1,396 clinical questions addressed by Oregon primary care doctors, the most common questions were “What is the drug of choice for condition x?” (11%); “What is the cause of symptom x?” (8%); and “What test is indicated in situation x?” (8%). ⁶ Thus, in the context of a PICO project, early medical students are not yet addressing questions concerning diagnosis or the cause of symptoms, as compared to physician questions arising from practice.

We can reason that the PICO format imposes restrictions for framing diagnostic clinical questions. When using the PICO format to formulate diagnostic questions, special considerations are required, ¹¹ such as how to ask questions about test accuracy and how this is distinguished from the effect of a test strategy on patient health. Diagnostic questions that address diagnostic test accuracy are likely to be cross-sectional, while questions that evaluate the effect of a diagnostic strategy on health will likely be prospective cohort or randomized controlled trials. In light of our findings, educators may wish to provide additional teaching around these considerations.

An additional hypothesis for the underrepresentation of the diagnostic question type observed in this study is that preclerkship students critically assess some aspects of patient care more than others. In 2016, Choosing Wisely Canada published a list of six things that medical students and trainees should question, encouraging individuals not to “hesitate to ask for clarification on tests, treatments, or procedures [they] think are unnecessary.” ¹² This directive is of particular relevance in hierarchical clinical settings, where individuals may feel reluctant to raise clinical questions due to fear of potential consequences or lack of confidence in their clinical reasoning skill set. ¹³

Another possible explanation for this phenomenon of fewer questions pertaining to diagnosis is that the vast majority of these questions were derived from the setting of a family doctor’s office, wherein many patients are not presenting with a new diagnosis. Instead, patient concerns may often surround the management and treatment of symptoms or chronic disease. In these settings, the chief task of the encounter is to aid the patient in terms of their therapy and management rather than identify a new diagnosis.

Diagnostic reasoning is an essential clinical skill, yet overuse and overdiagnosis remain prevalent concerns in medicine. Overdiagnosis, defined as the detection or labeling of a condition that was never going to cause harm or the application of a diagnostic label to ordinary life experiences, ¹⁴ has been associated with suboptimal patient ¹⁵ and population health outcomes. ¹⁶ Despite its role in reducing unnecessary tests and treatments, challenges in the development of diagnostic reasoning skills in medical education have been documented. For example, teaching around diagnosis is often relegated to the clinical environment, delaying opportunities for students to hone important skills. Another challenge manifests when the act of diagnosis is framed as a simple matter of categorization rather than a dynamic clinical process. Of note, the medical curriculum in the present study included didactic teaching on the possible harms of screening, such as overdiagnosis, before the assignment of the PICO project. Additional educational approaches to enhance the teaching of diagnostic skills have been suggested and include a critical approach to EBM, an understanding of the role diagnoses play in society, and leadership skills. ¹⁷

Using the first two levels of classification was seamless, and no questions were uncategorizable. The third level of classification, the CFPC priority topics, presented more challenges in terms of application because nine (5%) student questions were unclassifiable. According to the CFPC, the priority topics constitute

Level III findings describe the clinical topics that were addressed, as shown earlier in Figure 2. These findings may be used to speculate which topics are less represented in the observed clinical encounters.

This study had limitations. Our data came from an assignment where students must formulate a PICO question and seek an answer. This structure likely created a selection bias in the clinical questions based on students’ perceptions of the feasibility of finding an answer to their question. Therefore, to correlate these questions with student interest in family medicine would be an overinterpretation. In addition, the present study used a subset of Ely’s taxonomy, with the top 10 most frequently used generic types from an original list of 63. ⁶ However, according to the authors of that taxonomy, most questions could be classified using a limited number of generic types. Furthermore, the number of student projects used to determine interrater reliability was limited. While this resulted in a wide confidence interval for determining interrater reliability for Level II categorization (Cohen’s κ=0.25, 95% CI –0.56, 0.564), we note that interrater reliability for Level I categorization was near perfect (Cohen’s κ=0.93, 95% CI 0.87, 1.00), providing strong evidence of interrater reliability for this level. Additionally, our study represents student submissions from a single institution; therefore, the generalizability of these findings is uncertain.

The strengths of our study included the analysis of clinical questions addressed by an understudied population (preclerkship medical students) using a multilevel classification system. Questions addressed by medical students represent a variety of clinical topics that are well-suited for their stage of training. Our findings highlight prevalent themes in primary care from the perspective of first-year medical students as a part of an EBM curriculum and can be used to inform future iterations of the PICO project. Research is needed to illuminate whether the classification framework we used might inform other medical educators interested in evaluating their curriculum and contribute to understanding why the vast majority of questions submitted by early medical students pertain to treatment and therapy.