Black Clouds vs Random Variation in Hospital Admissions

Black clouds” have a long history in the superstitions of physicians. In some settings, this term refers to physicians with an apparent propensity for poor outcomes,1 and in others, a propensity for busy shifts or complex patients.2 However, investigators have had difficulty finding true black-cloud physicians whose workload or patient complexity exceeds random variation. For example, in one study, pediatric residents with a black-cloud reputation received less sleep and perceived working harder than their peers despite having the same number of admissions as their peers.2 In contrast, a study of 19 internal medicine interns found that those with a black-cloud reputation had significantly more admissions than those with a white-cloud reputation.3

Theoretically, hospital admissions might be expected to follow a Poisson process in which the number of admissions varies randomly from day to day. The Poisson process, when applied to hospital admissions, includes two assumptions: (1) any given day is no different from any other day, and (2) the number of admissions on one day has no effect on the number of admissions on the next day. However, real-world data almost never meet these assumptions. For example, Fridays may be busier than Thursdays, and light days may follow busy days if an overworked hospitalist diverts patients to other services or argues for a trial of outpatient management. And if the black-cloud phenomenon exists (for whatever reason), there will be more admissions when black-cloud physicians are on call. As a result, a statistical model more complex than one based on the Poisson distribution is often needed.

Our aim was to determine whether the existence of black-cloud physicians could be supported or refuted by analyzing a larger physician sample and a longer time period than previous studies. Understanding the black-cloud phenomenon is important because the black-cloud label can affect physician behavior and self-esteem.1,2,4 Understanding variations in workload is important because busy times may be associated with adverse patient outcomes and light times with wasted resources and insufficient educational opportunities.5

We conducted a retrospective observational study of admissions to the University of Iowa family medicine service in which individual physicians were evaluated for evidence of higher or lower admission numbers than their peers. The seven peer groups were defined by training level and call period:

• Day attending physicians,
• Day senior residents,
• Day junior residents #1,
• Day junior residents #2,
• Night attending physicians,
• Night senior residents, and
• Night junior residents.

Admission numbers were obtained from the departmental business office. Day call and night call assignments of physicians were obtained from call-schedule records. To control for possible confounding by calendar variables, the influence of month, day of week, holidays, and study year were determined during the 5-year study (July 1, 2010 to June 30, 2015).

The daytime inpatient team was composed of two junior residents (first postgraduate year), one senior resident (second or third postgraduate year), and one attending physician. The night team was composed of one junior resident, one senior resident, and one attending physician. Residents worked both night and day shifts, but attending physicians were divided between those who primarily took night call and those who primarily took day call. The residency program included eight residents for each of the 3 training years. The University of Iowa Hospital is an academic tertiary care center with 732 beds and 33,000 admissions per year.

We used descriptive statistics to characterize admission data and mixed-effects negative binomial regression to test for the phenomenon of black-cloud and white-cloud physicians.6 Negative binomial regression can be seen as an extension of Poisson regression with less restrictive distributional assumptions. In this model, a significant variance due to physicians is evidence that not all physicians have the same mean (adjusted for any covariates in the model), meaning that there may be true black- and white-cloud physicians. When such evidence is present, it is also possible with follow-up testing to identify which physicians are higher or lower than the mean (black and white clouds, respectively). We evaluated potential confounding from day of week, holidays, and study year by determining whether these covariates were significant fixed effects in the statistical models. For attending physicians, we also explored the effect of physician gender and experience. Experience was defined as the number of years between the physician’s hire date and the study year.

We stratified the analysis by peer group. For example, one regression model included all senior residents taking night call, another included all attending physicians taking day call, and so on. We excluded physicians with fewer than 10 call shifts. Alpha was set at .05.

To characterize physician beliefs about black clouds, we surveyed all residents and attending physicians in the department in 2016. A 12-item electronic questionnaire asked whether physicians believed in the black-cloud phenomenon and if so, who among their colleagues were black clouds or white clouds. The authors designed and revised the questionnaire without extensive pilot testing or formal validation procedures. The study was approved by the University of Iowa Institutional Review Board. All analyses were performed with STATA (StataCorp 2011. Stata Statistical Software: Release 15. College Station, TX).

During the 5-year study, 96 family physicians admitted 6,194 patients. The mean (SD) number of admissions per 24-hour period was 3.4 (2.0) with a range of 0 to 12. The mean (SD) during the day was 2.2 (1.6) with a range of 0 to 10. The mean (SD) at night was 1.2 (1.2) with a range of 0 to 11.

The mean number of daytime admissions was higher on weekdays than weekends (2.5 [95% CI 2.4–2.6] vs 1.3 [95% CI 1.2-1.4]). The mean number of night admissions was higher on Sunday than the other nights (1.5 [95% CI 1.3-1.6] vs 1.2 [95% CI 1.1-1.2]). Admissions did not vary significantly by month. Holidays were associated with fewer daytime admissions than nonholidays (1.3 [95% CI 0.9–1.7] vs 2.2 [95% CI 2.1–2.3]), but night admissions were not affected by holidays. The number of admissions per year increased significantly and gradually from study year 1 (n=1,016) to study year 5 (n=1,523), but daytime admissions were mostly responsible for this increase.

We fit seven regression models (one for each peer group) to identify black- and white-cloud physicians. We started with simple models that included admission numbers as the dependent variable, physician as the explanatory variable (random effect), and no calendar variables (fixed effects). All but one of these seven simple models found variability due to physician and identified specific black- or white-cloud physicians in follow-up testing. During the 5-year study, there were 16 black-cloud and 18 white-cloud physicians in the simple models that were unadjusted for calendar variables. For example, among 44 day senior residents, we found four black-cloud and five white-cloud physicians.

We then added study year, holiday vs nonholiday, and Sunday vs all other days to control for potential confounding in each of the seven models (Table 1 and Appendix at http://www.stfm.org/Portals/49/Documents/FMAppendix/AppendixA-Ely2018.pdf). In five of these adjusted models, there was no evidence of black- or white-cloud physicians. In the two remaining models (night senior residents and night attending physicians) we found significant physician-to-physician variability (Table 2). However, the physician variability was so minor that, in follow-up testing, none of the regression coefficients for individual physicians were significantly different from zero, and thus no specific black or white clouds were identified after controlling for calendar variables. For example, the physician variance component for night senior residents was 0.0142 in the simple unadjusted model, but fell 73% to 0.0039 after adjusting for calendar variables (Table 1). Among attending physicians, we found no association between admission numbers and physician gender or experience.

Despite these negative findings, physicians suspecting black clouds may have focused on time periods shorter than 5 years because the residency lasted only 3 years, and only 11 of the 30 attending physicians were represented in all 5 years. In an analysis stratified by year, we found only one night senior resident with significantly more admissions than his peers, but only in year 4 (Figure 1). Results from this stratified analysis should be viewed with caution because, with seven peer groups and 5 years, 35 comparisons were required to find this single black-cloud physician with a borderline P value of .04. In other years and other peer groups, apparent outliers likely raised suspicions of black or white clouds despite the lack of statistical significance (Figure 2).

The questionnaire on beliefs about black clouds was returned by 29 of 43 physicians (67%). A majority of respondents (n=23, 79%) said they did not believe in black or white clouds. A majority (n=23, 79%) believed that day of week, meaning the full 24-hour day, influenced admission numbers with Friday receiving most votes (n=11) for the busiest day (which was true), and Tuesday and Wednesday tied for the lightest day (which was not true). In fact, Saturday and Sunday were the lightest days. Seven attending physicians and five residents were considered black clouds by at least one respondent. Six attending physicians and two residents were considered white clouds. Of the 12 perceived black-cloud physicians, five were black clouds in the unadjusted regression models (and none were black clouds in the calendar-adjusted models). Of the eight perceived white cloud physicians, none were white clouds in the unadjusted (or adjusted) regression models.

In this study, we found no evidence of black- or white-cloud physicians after adjusting for calendar variables. However, within this random variation, we found a wide range of admission numbers that could easily be misinterpreted as variation
beyond chance.

Our findings are consistent with five previous studies which found no evidence of black- or white-cloud physicians based on various measurements of workload.2,3,7-9 However, two of these studies found positive correlations between reputations as black clouds and increased workload.3,7 None of these previous studies lasted more than 1 year, and none included family physicians. In a sixth study, investigators randomized internal medicine residents to receive a written “jinx” message stating, “You will have a great call day” vs no message.10 Contrary to the usual superstition, jinxed physicians (those receiving the optimistic message) had significantly lighter workloads than nonjinxed physicians. Most physicians in one previous study preferred not to be considered black clouds by others.2

Our findings should be interpreted in light of several limitations. We collected data on hospital admissions as the only measure of workload, whereas other investigators examined patient complexity, hours of sleep during night call, and number of emergency department visits. We did not collect or analyze data on several potential confounding variables, such as weather, influenza epidemics, or university calendars. The transition from day shift to night shift and vice versa involved fuzzy borders in which, for example, a daytime attending physician might work into the evening and receive credit for a night admission. It might seem more appropriate to count admissions per team rather than admissions per individual physician. However, the composition of night teams changed daily and day teams changed weekly. Also, in this study and others, the black-cloud label was applied to individuals rather than teams. Our analysis did not distinguish between busy days occurring in close proximity from busy days separated by weeks or months. The data were collected in a single academic institution, and the extent to which our findings can be generalized to other settings is unknown. Strengths of this study include the large physician sample and the long study period.

The term “black cloud” seems to connote a mystical or supernatural phenomenon that might go unexplained if it occurred. Alternatively, it might be explained by unmeasured variables, such as physician comfort with outpatient management in borderline cases. Our analysis was based on formal hypothesis testing and statistical significance, but others might accept less stringent criteria and include, for example, the apparent outliers in Figures 1 and 2.

We found no evidence for black- or white-cloud physicians in this study after controlling for calendar variables, such as increase in admissions over the 5-year study period. Although physicians might have some influence over their own admission numbers, most physicians surveyed in this study believed that the variation resulted from chance. On a practical level, the wide range in admissions per call period from 0 to 11 poses challenges for physicians who must deal with the extremes, as well as administrators who set staffing levels. Ideally, staffing levels would vary with workload, but in practice, this flexibility is difficult to implement. The wide variation in workload, whether or not explained by chance, should encourage measures to maintain patient safety on busy days. Unvarying physician staffing based on mean admission numbers without regard to extremes may impede optimal patient care at the high end and educational opportunities at the low end.