Authors’ Reply

We very much appreciate the thoughtful response to our article by Drs Westfall and Westfall. We would like to respond to what we consider the two main issues they raise: the importance of assessing English as a second language (ESL) impact on exam performance and whether exam performance is an adequate measure of residents’ knowledge.

We agree with the commentators that the impact of ESL on exam performance is an important variable that should be evaluated for potential bias. Unfortunately, however, neither international medical graduate (IMG) status nor language of care could serve as an adequate proxy measure for ESL. We cannot identify ESL students using IMG status because English is the first language for many IMG students, others use English as their primary educational or professional language,1,2 and the US medical graduate (USMG) group includes ESL students.3 Similarly, although American Board of Family Medicine (ABFM) collects information on the use of language other than English for care delivery, it does not reflect first language nor does it indicate fluency in any language. While physicians who indicate that they provide patient care in languages other than English could represent ESL physicians, it is also possible that they provide care in other languages with the help of an interpreter or translation technology.4,5 More practically relevant to our article, this information is currently only collected from practicing physicians, not from residents, making it impossible to use for In-Training Examination (ITE) and initial Family Medicine Certification Exam (FMCE) analyses. ABFM advocates for the collection of more accurate data regarding physicians’ primary language, along with other data that reflect diverse backgrounds,4,6 so that more forms of bias can be identified and addressed.

The commentators furthermore suggest that performance on the ITE and FMCE is not an adequate measurement of resident knowledge. First of all, ABFM asserts that the FMCE scale (FMC-Scale) measures medical knowledge related to family medicine and clinical decision-making ability.7,8 Both ITE and FMCE are built to this common FMC-Scale, which permits the longitudinal analysis conducted in our article to detect initial differences before residency and knowledge acquisition during residency.9 This distinction helps to identify potential educational pipeline issues and promote individualized learning plans for residents who need additional help.9,10 While a holistic approach is often described in terms of medical school and residency admissions, we endorse a broader approach to assessing knowledge, combining the breadth and flexibility of the ITE with specific assessments of knowledge while precepting, on hospital rounds or in labor and delivery. This is the promise of competency-based assessment,11 and the ITE is an important component of this assessment strategy. Finally, and importantly, the exams (ITE and FMCE) are not intended as a measure of clinical skills, communication, professionalism, problem-based learning, or systems-based practices—the other ACGME core competencies, which residencies evaluate and track using the ACGME milestones.12 The ABFM relies on residency clinical competency committees and the program director to attest whether residents are ready for autonomous practice or demonstrate other skills and competencies related to clinical practice, which are evaluated using other components to make the holistic decision about ABFM certification.13

ABFM welcomes additional collaborations with other organizations to advance health equity in family medicine.14,15 Our article is an initial part of a larger systematic effort not only to describe and document the progression of medical knowledge acquisition of family medicine residents from diverse backgrounds but to address and intervene to promote health equity in multiple dimensions.9,16,17 We appreciate the response to our article and the important issues it raises, and we share the hope that equity in residency education can be achieved in the future with collective efforts.