Predicting No-shows at a Student-Run Comprehensive Primary Care Clinic

Patient no-shows cause administrative and financial problems for outpatient clinics.1-3 For example, in 10 outpatient clinics in the US Veterans Health Administration system, there was a mean no-show rate of 18.8% over 12 years, with an average cost of $196 per no-show.3 High no-show rates also negatively impact student and resident education and productivity.4 In addition, patients’ health suffers, as they do not receive follow-up.

Previous studies demonstrate that demographics, previous appointment keeping, psychosocial factors, health beliefs, and situational factors all affect no-shows.5,6 Although we are not aware of any formal conceptual framework for no-shows, reviews generally break factors into three categories: patient demographics, appointment characteristics, and other factors.5,6 Based on these studies, we derived a theoretical conceptual framework (Figure 1).

Studies examining ways to reduce no-shows have found that clinics that managed no-shows well encouraged walk-ins and work-ins and strategically overbooked schedules.7 Although reminder systems generally improve attendance,8,9 this is not universally true.3,4,10 Multimodal projects to reduce no-shows have been the most successful.11

Shade Tree Clinic (STC) is a free, medical-student-run clinic that serves patients with limited resources.12-14 STC serves as a comprehensive primary care provider for more than 350 uninsured Middle-Tennessee residents.

In this study, we sought to develop a model that predicts no-shows to guide clinic scheduling. We hypothesized that variables available through the electronic schedule, when placed in a framework based on previous literature, could be used to predict no-shows.

Data were collected for all available appointments from scheduling software (2010 to 2015). Cancelled and rescheduled appointments were excluded. Independent variables were selected based on availability and applicability to the conceptual framework (Figure 1). The scheduling software tracked whether the patient was reminded about the appointment via an automated voice call or text message. Appointments were coded as reminded if the patient responded they would attend the appointment. These data were not collected until 2014, and appointments before this were categorized as not reminded. Show rate was calculated as the proportion of appointments completed by the patient prior to the appointment being analyzed. Data about weather for each day were taken from an online database.15 The Institutional Review Board at Vanderbilt University deemed this study exempt.

Analyses were conducted using Stata version 14.1 (StataCorp, College Station, TX). Multivariable logistic regression was used to model the probability of appointment completion. Show rate for previous appointments is not always known, so a model was created with and without this variable.

The data included 13,499 appointments (Table 1). Of these appointments, 9,347 were completed, yielding an overall show rate of 69.2%. Women accounted for nearly twice as many visits as men but had comparable show rates. Patients who spoke a language other than English accounted for 31% of visits.

An initial model was built excluding previous show rate (Table 2). The inclusion of gender in the model did not affect point estimates or significance and was therefore excluded from the presented model.

When previous show rate was included in the model, all patient factors from the first model became insignificant. This suggests that show rate serves as an aggregate measure for all patient factors, including those captured by our initial model and other unmeasured factors. We created a second model using nonpatient factors and previous show rate (Table 3). Pseudo R2 increased from 0.0212 to 0.2113, suggesting a better fit and greater proportion of variance explained. A cutoff probability of 0.5 yielded sensitivity of 89.17% and specificity of 44.10% and correctly classified 75.31% of appointments. A cutoff of the lowest quartile of probabilities (P=.548) yielded a negative predictive value of 61.08% out of 3,376 appointments classified as no-shows. A cutoff of P=.25 yielded a negative predictive value of 85.62% for 758 appointments classified as no-shows.

This model suggests that variables collected during clinic operations can be used to predict appointment completion. By using a cutoff with a high negative predictive value, we can identify patients likely to be no-shows and target them for interventions.

Our data are consistent with other studies that have found that demographics and weather can affect appointment completion.5,6 A recent systematic review found just one article finding that weather was associated with no-shows,16 and six where it was not.5 Our study supports the common conception that weather affects clinic attendance. Unlike other studies,3,4,10 we found that appointment reminders increased the likelihood of appointment completion. This may reflect that our reminder system actively determines whether the patient confirmed the appointment.

Importantly, by using all appointment data available, we have maximized the power to detect differences between groups. Odds ratios must therefore be interpreted in the context of absolute group differences. For example, although the odds ratio for an inch of snow is 0.331, there is relatively little snow in the overall dataset and the difference in snow quantity between no-shows and completed appointments is not large. Based on our analyses, the odds ratio most useful for predicting no-shows is that associated with previous show rate, but the other presented factors are useful in refining this prediction.

While our second model is likely to be generalizable to other clinics, models generated using data for a specific clinic will likely be more predictive for each particular setting. Similar models could easily be created using data at other clinics.

Our study has several limitations. First, we were limited to data present in the scheduling software. It is likely that other factors within the conceptual framework are associated but were not available. Second, our second model depends heavily on previous show rate. This will not be available for new patients, and the model that only includes demographic information performs worse. Third, our appointment reminder system was not implemented until midway through 2014. Coding all appointments prior to this date as not reminded is a conservative assumption but likely dilutes the effect of this variable. Fourth, our clinic is only open on Tuesday and Saturday. This prevented analyses of whether the difference in show rate is due to differences in specific appointment days or weekday versus weekend appointments. Finally, our model relies on historic weather data. While forecast values could be used for future appointments, the accuracy of the model will depend on the accuracy of these forecasts.

In conclusion, we created a model that identifies patients who are unlikely to complete appointments and can be targeted for intervention. The process of creating this model is generalizable to other student-run and primary-care clinics. Future research could focus on the effectiveness of various interventions at reducing no-shows.