Response to “Exploring the Applications of ChatGPT in Family Medicine Medical Education”

“…may you live in interesting times…”

This is indeed a very interesting time to be in medical education, and in light of the rapid evolution and broad availability of artificial intelligence (AI) platforms, this curse may ring especially true without thoughtful guidelines and establishment of best practices with sufficient flexibility to evolve with the technology. In his letter to the editor,¹ Dr Hanna offers several novel applications of generative AI models, such as ChatGPT, for enhancement of the learner experience as a complement to traditional pedagogical approaches. We agree that language-generative AI technology may provide unique and realistic simulation of patient encounters, and this is the most intriguing prospect suggested by Dr Hanna. However, we have great concern regarding the use of generative AI to generate differential diagnosis exercises, facilitate exam preparation, or produce patient education materials due to the frequency of false, misleading and/or “hallucinatory” fabrications of information.² These fabrications may expose the learner to unrealistic patient presentations while false and/or biased information could miseducate the learner or patient.³ Fabricated information and biased information may also contribute to the misdiagnosis or mismanagement of patients.⁴ We are also concerned with the possibility that protected health information could mistakenly be divulged in the course of dialog with an AI used to assist with clinical decision support because most, if not all, end-user agreements for AI products state that the information provided by the user is not confidential and will likely be used to train the model.⁵

As Dr Hanna rightly notes, enthusiasm for this rapidly evolving tool must be tempered by a sober consideration of the risks that such AI technologies may present. We share their enthusiasm for all of the promise that these technologies hold, but also cannot overstate the point that their use in the academy must, at a minimum, be supervised by an experienced preceptor who can review, identify, and remediate bad information from AI-generated educational materials. Furthermore, in the absence of best practices in this area of medical education, we encourage development of thoughtful and forward-looking guidelines for the use of these technologies that can promote responsible, ethical, and productive use, but also mitigate the adverse consequences of misuse—both accidental and intentional.