The Challenge of External Validity: Why Specialties Interpret Evidence Differently

We live in an era when different guideline developers examine the same evidence and make different recommendations to practicing clinicians and patients. Although we might expect political parties to interpret the text of a news report differently, the public can be dismayed and confused when such differing interpretation extends to medicine and scientific evidence. While this is frustrating, there may be valid reasons that explain different interpretations of data, and we should not reflexively jump to conclusions about misinformation or misguided intentions.

Where physicians work in the health care system influences the lens by which they focus on information gleaned from randomized controlled clinical trials. This lens may either clarify or obfuscate perceptions of scientific veracity and objectivity depending on the reader’s lived experiences. Despite advances in evidence-based medicine to standardize reporting, emphasis on relative risk reduction (RRR) continues to permeate the literature and exert significant influence on guidelines intended for primary care, where absolute risk reduction (ARR) better fits the setting. This is especially important when experts attempt to apply an intervention proven effective through randomized clinical trials in settings and across time frames not studied in the trials.

Hypertension guidelines are an example of this. The American College of Physicians (ACP) developed a primary care-focused hypertension guideline that differed substantially from one by the American College of Cardiology/American Heart Association.1,2 Both groups reviewed the same data. The ACP guideline required a higher standard of evidence to apply interventions to healthier populations, while the specialist-derived guideline recommended lower blood pressure targets and more frequent pharmaceutical treatment. The primary care community’s reticence to implement such interventions reflects different practice experiences, approaches to clinical medicine, and populations of patients. Thus, differences in the clinical context in which different physicians practice medicine and patients seek care may limit our ability to deliver the right care, to the right patients, at the right time. This contextual perspective represents an important opportunity for needed research.

Generalists and specialists interact with patients at different times in the presentation of a disease, at different locations within the health system, and may address different goals of care expected by patients as a result (Table 1). Specialists are more often located in hospitals and in hospital-based service lines rather than in the community, and are more likely to see patients at a moment of crisis when there is a singular clear goal: to reduce mortality and morbidity from a threatening disease (eg, heart disease). Patients who see cardiologists, for example are likely risk stratified and selected from the larger population of patients seen in primary care. The patient is seeing the cardiologist with the goal of reducing cardiovascular mortality and morbidity. Evidence from randomized controlled trials (RCT’s) is very relevant and applicable to this problem and outcome. When patients present with one dominant and clearly life-threatening disease, at the right time for interventions (pharmaceuticals or surgical interventions), and with a single disease-centered goal, the specialty care paradigm offers appealing technological solutions arising from supportive and consistent clinical trial evidence.

Hospital-based health systems have created service lines to enhance value, efficiency and sustainable revenue, and promote more effective, efficient specialized organ system or procedure-based care. This business strategy has had subtle yet far-reaching systemic effects that limit the ability of even our best-trained specialists to practice patient-centered, whole-person medicine. But even larger problems arise when the patient’s need extends outside the service line, increasing the risk that the wrong care is delivered to the wrong patient. Specialty physicians are more likely to focus on ruling out the disease within their own service line, or find an intervention that supports their business model. The same expectations have limited primary care physicians to short, quick visits that may not honor the complexity of goal-directed care. An unintended consequence of service lines is that they alter not only the way physicians practice, but also their professional identities, thereby skewing perceptions and ensuring that disease-centered care supersedes patient-centered, whole-person care.

Patients seek primary care with less clear agendas. While each of us would prefer our health concerns be biomedical with clean technological solutions, our illnesses often represent social and psychological challenges that require wholistic interventions to ameliorate. In the last century, family medicine emerged as a response to the limits of disease-centered, specialty-driven health care. Patients present to primary care with undifferentiated or even no concerns about their overall health. Distinguishing disease-centered (problem-oriented) care from goal-oriented care has not been studied enough. Such research will require improved understanding of patient values and preferences with respect to the impact of disease-centered interventions on the life course of patients.

In primary care, the timing of patient presentation can be decades before a disease-centered outcome, such as stroke for hypertension. Alternatively, patients may present at the end stages of disease where life concerns are complicated by competing goals of care. Specific interventions can recreate the patient’s previously experienced health, yet that might not even be the patient’s primary goal. RCTs inform neither of these patient situations because RCTs address patient populations, not unique individuals. Primary care conversations with patients may be broader than disease-centered assessments of risks and treatment of predisease and disease states. In primary care settings, patients embrace their autonomy and are more likely to express their preferences, values, and disapproval of treatment options that may seem foregone conclusions in the hospital or specialty office, where the timing is prescient and the focus is often on saving a life.

Hypertension is an example of a preventive intervention for patients at risk based on an arbitrary cut point on a continuous scale.3,4 More aggressive cut points identify more individuals at risk, thus identifying more patients for treatment. The primary care community perceives that labeling lower-risk patients with hypertension requiring daily medicines may not be justifiable, based on ARR. The same evidence produces different conclusions when experts emphasize RRR over ARR, resulting in different recommended goals. Without compelling evidence, primary care clinicians prefer to label predisease as “well” with risks, and many patients share this concept. Thus, there is a higher expectation for stronger evidence by the primary care community and they resist generalizing proven interventions derived from studies of high-risk groups to populations not studied in RCTs.

Our scientific evidence is more limited than it is wrong. RCT evidence struggles to find its footing in primary care settings and situations.5 In most cases, when different medical specialties embrace or discount different bits of data from scientific evidence, the conclusions derive from systematic differences in their experiences, perspectives, populations, and goals of care. For most evidence, clinicians ask, “How does this affect my patient?” How we answer this depends on our experiences with patients who have different goals of care, assumptions about wellness/illness/disease, and experiences with health care. Perhaps through reframing and understanding the different assumptions that different specialties bring to interpretation of the same evidence, we can transcend our differences and support improved guideline development and health care for all.