Introduction

Emergency Department (ED) provider productivity and efficiency are two important performance measures. Productivity is viewed as the number of patient encounters per provider per unit time. It can also be measured by number of Relative Value Units (RVUs) generated per provider per unit time or encounter. Efficiency refers to the time and resources required to complete an ED patient encounter. ED length of stay (LOS), defined as the total time spent within the ED for a given patient encounter, and provider-to-disposition time (PDT), defined as the interval starting with initial provider involvement and ending with disposition selection for a given patient encounter, are among recognized efficiency metrics [1–4]. However, both are affected by multiple factors including ED crowding, patient acuity, and supervision of Residents. [5–7].

Most of the care delivered in training institutions occurs via the Attending oversight of Resident care model [8]. Mixed results are documented regarding provider productivity and efficiency of the Attending-Resident Team as a function of relative team clinical experience [9–11]. A study focused on productivity used the number of new patients per hour seen by either residents or Advanced Practice Providers (APPs) in a Fast Track area (i.e., low acuity patient care area), they found Resident productivity was less than that of the APPs’ indicating resident in training might affect their productivities [12]. Previous studies revealed prolonged LOS was experienced by patients receiving care in the Attending oversight of Resident model indicating decreased provider efficiency in the training institution setting [9,10]. However, another study determined that ED LOS was not significantly affected by the presence or total number of trainees in the ED [11]. At present, only a few studies compare provider productivity and efficiency between Attendings working solo versus Attendings working with Residents and none of them examine the differences in productivity and efficiency of individual Attendings within these two groups.

Productivity and efficiency are often consistent when used to evaluate provider performance (i.e., efficient providers are also productive or vice versa). However, it is not uncommon to observe some inconsistencies (e.g., high productivity but low efficiency), thus creating a significant challenge to understanding overall provider performance. To overcome this challenge, prior research introduced a composite index, which was calculated by combining productivity and efficiency [13]. However, a major shortcoming of said calculation is the lack of external validation.

To gain a deeper understanding as to whether working with Residents affects Attending performance within an academic environment, we aim to measure productivity, efficiency, and overall performance of Attendings and further compare the differences among Attendings working solo versus Attendings working with Residents at an individual level.

Methods

Outcome measurements

We used three measurements: 1) the number of new patients per hour seen by a given Attending measured productivity; 2) PDT of each patient measured efficiency; and 3) Attending Performance Index (API) measured overall performance (see formula). API is a composite metric that integrates both productivity and efficiency. It is modified based on an established performance index [13]. A detail explanation of API is addressed in S1.3 Appendix.

Results

During the study period, a total of 49 ED Attendings and 271 ED Residents worked at the study ED. We excluded 14 part-time ED Attendings, 4 Attendings that either consistently worked solo or worked only with Residents, and 16 Attendings without balanced schedules. A final group of 15 ED Attendings and 266 Residents that collectively managed a total of 111,145 patient encounters during the study period was enrolled. (S1 Appendix Fig). Study patient general information is shown in Table 1. Patient and clinical characteristics (age, ESI, ED crowding, and ED LOS) in the final analysis differ between Attendings working solo and Attendings working with Residents groups (Table 1, p < 0.001). It was noted that the Attending working solo group saw a slightly more high-acuity (ESI 1) patients. More patients were seen by the Attending working with Residents group than the Attending working solo group during times when the ED was overcrowded (Table 1, p < 0.001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Study patient population general characteristics.

https://doi.org/10.1371/journal.pone.0228719.t001

Analysis of productivity reveals more patients per hour were seen by Attendings working with Residents than Attendings working solo (Table 2). Total numbers of patients presenting to the ED from 2300 on a given day to 1500 the following day were calculated from Monday through Sunday revealing that patient volumes were within median range during the Attending working solo timeframe (Wednesday 2300 to Thursday 1500) (S2 Appendix). Shorter PDT (i.e., efficiency) is noted in the Attending working solo group (Table 2). Essentially, Attending productivity increased but efficiency decreased when working with Residents. Overall performance seems to be increased among Attendings working with Residents in comparison to Attendings working solo. However, the benefit of increased Attending provider performance while working with Residents appears to be diminished when propensity score matching is applied (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Provider productivity, efficiency, and performance measurements comparison between attendings working solo and attendings working with residents.

https://doi.org/10.1371/journal.pone.0228719.t002

Additionally, when Attending productivity, efficiency, and overall performance are measured at an individual Attending level, the case of increased productivity with decreased efficiency is observed for most individual Attendings when working with Residents compared to working solo (S3 Appendix and S4 Appendix). All 15 Attendings were divided into two groups in terms of their solo performance. Higher Attending performance indexes (APIs) are observed among individual Attendings when working with Residents if their solo API < 0.18. On the contrary, lower APIs were noted among individual Attendings whose solo API ≥ 0.18 when working with Residents (S5 Appendix). This pattern remained when confounding factors were adjusted using the 1:1 propensity score matching model (Table 3). Therefore, a threshold API (high API ≥ 0.18 and low API < 0.18) of individual Attending performance was determined in this study.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Attending overall performance comparisons between attendings working solo versus attendings working with residents based on attending solo performance.

https://doi.org/10.1371/journal.pone.0228719.t003

Discussion

Our study found increased productivity with decreased efficiency among Attendings working with Residents. When API was used for performance assessment, we found that individual Attendings with high solo performances rendered an overall decreased performance while working with Residents. On the contrary, working with Residents increased overall performance among Attendings with low solo API. Though the study was performed in an ED, interpretation of study findings might not be limited to the ED setting since they reflect a similar academic teaching model whereby Attending physicians work with Residents in most clinical practice settings. Therefore, our results add more evidence to summative provider performance measurements in an academic institution thereby providing valuable insight to current and future approaches to Resident training and faculty evaluation across different residency specialty programs.

Productivity and efficiency can be affected by differences in patient acuity. The study ED preferentially flows patients such that Attendings working solo and Attendings working with Residents provide high acuity care and the APP staff provide low acuity care. Though the majority of high acuity patients (ESI 1–2) were seen by Attendings regardless of whether they worked solo or with Residents, productivity and efficiency differences occurred between these two groups. The possible reasons might be: 1) the daily variety of high acuity patients present at the study ED (e.g. slightly more ESI-1 patients present during non-resident shifts); and 2) the exclusion of the APPs in this study. During the ED non-resident time (16h/week), an extra APP was scheduled to work. However, this study excluded APPs who usually only saw low-acuity patients. Under this circumstance, it is necessary to use propensity score matching to minimize potential confounder effects in this study. When propensity score matching was applied, the median number of new patients per hour seen by either Attendings working solo or Attendings working with Residents reveals no changes both at the group (Table 2) and individual (Table 3) levels, indicating that patient acuity level has no effect on provider productivity measurement. In addition, different metrics have been used to measure provider productivity in the literature, including RVUs or number of new patients per shift. However, RVUs can be significantly affected based on encounter documentation alone. Our providers usually spend extra time outside of their clinical shift to complete documentation, thus it is extremely difficult and inaccurate to add extra time for documentation when deriving the provider productivity calculation. Moreover, providers working at the study ED typically work different length shifts (e.g., 8h, 9h, 10h, or 12h shifts), serving as a potential confounder that fails to compare provider productivity on a common basis when number of new patients per shift is used. Therefore, to simplify this study, the number of new patients per hour, as opposed to RVUs per hour and/or patients per shift, was used as our Attending productivity measurement [4,16]. On the other hand, we used PDT instead of patient LOS for provider efficiency measurement because LOS is often affected by different system- and patient-associated variables, which could be potential confounders in the context of provider efficiency (e.g., ED crowding, waiting room time, etc.) [17,18]. Unlike LOS, PDT is more directly affected by a given provider during the decision-making process impacting downstream diagnostic and therapeutic resource needs turnaround intervals [4,13]. Therefore, PDT delivers better interpretive quality regarding Attending efficiency. Ideally, a higher number of new patients per hour (high productivity) along with a lower PDT per patient (high efficiency) is considered the most desirable overall provider performance outcome combination. A balanced provider performance index was therefore used in this study [13].

We refrained from including all ED Attending data in this analysis to avoid developing different biases. We believe that inclusion of ED Attendings with non-balanced shift schedules based either on working solo or working with Residents arms will produce bias thereby complicating comparison of performance changes at an individual level. Furthermore, provider productivity and efficiency could vary significantly based on provider proficiency levels, thereby potentially affecting Attending productivity and efficiency when working non-balanced shifts. Working non-balanced shifts will affect Attending proficiency when spending more time performing unusual tasks (e.g., increased documentation time in absence of Resident input to overall encounter workflow). We therefore only enrolled Attendings with balanced shift schedules and analyzed median values to stabilize potential confounders. As shown in Tables 2 and 3, no changes to median values are noted after all potential confounders are adjusted by propensity score matching. Such findings further confirm the stability of using medians (IQR) for productivity and performance measurements.

Many previous studies investigated the impact of trainees on ED Attending performance concluding diverse findings [7,11,19,20]. When patient ED LOS was measured, some studies showed that ED LOS was not affected significantly by the presence of trainees rotating in the ED [11], whereas other studies showed an increase in ED LOS when Attendings worked with Residents [19,20]. In one study, an average of 7 minutes of increased LOS was reported with each additional trainee working with a given Attending [20]. In terms of provider productivity, a previous study also revealed no significant difference in number of patients per hour seen by junior EM, senior EM, or off-service Residents [7]. The reasons that our study findings differ from previous reports include: 1) we balanced our samples of Attendings in both groups, therefore the changes in their operational performance can be compared at an individual Attending level. In contrast, other studies mainly assess the operational performance at a group level; 2) we used PDT instead of LOS for provider efficiency measurement which may help reduce the effects of other confounding factors (e.g., ED crowding). The use of propensity score matching further minimizes potential biases; 3) the study ED had a 1:2 Attending-to-Resident ratio instead of 1:1 ratio staffing. This might affect Attending performance more significantly than the 1:1 teaching model, which could indirectly result in increased Attending productivity (i.e., more providers are available to see patients during the same time interval) with decreased efficiency (i.e., more trainees requiring more supervision per Attending per unit time), which differs from the previous report [7].

Our study has its limitations. First, this is a single center retrospective study which cannot demonstrate causality due to potentially harvesting incorrect information and selection bias. Second, although two key performance metrics were used for provider productivity and efficiency analysis, other metrics (e.g., number of RVUs per hour per provider, number of patients treated per shift) were not collected and compared in this study. Additionally, productivity and efficiency could be affected multi-factorially to include patient disease severity, nursing staffing levels, efficiency levels of nursing staff, etc. Study results might be inaccurate without the measure of individual patient disease severity and nursing staffing and their efficiencies. Third, among the Attending working with Residents group, due to differences in Resident efficiency and performance, Attending productivity, efficiency, and performance might potentially be affected. We also did not analyze and interpret how different levels of resident (EM versus non-EM residents) and medical students could impact attendings’ productivity, efficiency, and performance. However, we consider such effects to be minimal due to 1) blinded scheduling of Attendings and Residents resulting in random Attending-Resident shift combinations thereby avoiding significant heterogeneity; and 2) this is a 3-year study with over 100,000 patients seen by 15 Attendings working with 266 different Residents therefore each Attending had opportunity to work with Residents performing at different efficiency levels producing minimal overall effect on performance. Fourth, under rare conditions, Attendings might see patients by themselves while also supervising Residents during the Attending working with Residents shifts. Though it rarely occurs (< 1% in this study), it could still potentially affect our study results. Last, patients in the Attending working solo group accounted for a relatively small portion of the entire study population due to the current staffing model that employs Residents 90% of the available weekly schedule (152/168 hours). Given the fact of a fixed schedule of Attending only shifts, patient volume differences and potential patient selection bias could possibly occur when compared to patients from other groups. Therefore, a multi-center prospective study is warranted for external validation.

Conclusion

Diverse Attending productivity and efficiency exists. In our study, Attending overall performance, when working with Residents, varied inversely when compared to their performance working solo.

Supporting information