Introduction

Patient experience is an important metric for measuring hospital performance. Since 2012, the Centers for Medicare and Medicaid Services (CMS) have tied patient experience to reimbursement through the Value Based Purchasing (VBP) program. In 2016, 1.75% of a hospital’s Diagnosis Related Group (DRG) base operating payment is at stake and 30% of this payment is linked to a hospital’s patient experience scores.[1] Commercial payers are also linking patient experience outcomes to value-based payments, and many physician groups include patient experience as a metric for physicians’ variable compensation.[2] This increased scrutiny is appropriate, as multiple studies have shown a strong correlation between the quality of physician communication and the quality of clinical care.[3–8] Moreover, many problems with the effective delivery of health care can be attributed to ineffective communication between patient and provider.[9] Practically speaking, good physician communication will be a requirement for hospitals to sustain reimbursement at current levels. Yet there remains a deficit of evidenced-based interventions that lead to improvement of both patient perceptions of these communication skills and overall patient experience.[10–16]

Medical simulation is a validated method of teaching and improving clinical communication abilities. [16–25] Systematic review of clinician communication courses finds an effective approach combines didactic components with simulation with skilled feedback.[24] Programs based upon models of experiential learning and deliberate practice (learning focused on repetitive performance of specified skills with specific feedback)[26] succeed in teaching communications skills and changing clinician behavior. [17,18,25,27,28]

Improvement on clinician communication skills is a prerequisite to improving hospital care. Previous studies show only 10–32% of inpatients can correctly name their physicians, fewer (11%) can explain their physicians’ role in the care that they are receiving, and few can understand the key elements of their hospital plan.[12,29,30] Given these findings, a critical realm to focus improvement on clinician communication relates to basic “etiquette-based medicine” behavior such as knocking on a patient’s door, asking to enter the patient’s room, introducing oneself, sitting down in the patients’ room, and explaining one’s role in care. One cross sectional observational study on such behaviors found that one-third of physicians (30%) performed zero of the six “etiquette-based” behaviors and a majority (56%) did explain their roles to patients, despite the fact that a positive association was found between performance of the behaviors and patient experience scores. [12]

Yet evidence is mixed as to whether clinician communication skills training has an effect on patient satisfaction,[13,14,31] with most interventions showing no effect.[10,13,15,16,32] We hypothesized that using an evidenced-based framework of clinical simulation in a deliberate practice model targeted at teaching simple, discrete physician communication behaviors with structured feedback would improve patients’ perceptions of physician communication and reported experience of care. We implemented such a model in a hospital medicine division at a single academic center and examined its impact on outcomes using two systematically administered patient experience surveys, focusing on domains relevant to physician communication. We compared trends in patient experience scores between medical patients and, where possible, control surgical patients (whose physicians did not undergo the intervention) during the same time period. The control group was included to account for hospital-wide patient experience initiatives occurring concurrent to our intervention time-period.

Methods

Results

Of 50 hospitalists, 42 (84%) participated in the physician communication coaching simulation. Of 94 resident physicians, 86 (90%) participated. Hospitalists were 47% female and 53% male, and nearly half (48%) had 0–3 years of attending experience. Only 16% had greater than 10 years experience. The surgical attending clinicians were 16% female and 84% male; we did not have information on years of experience for surgical attendings. The survey participation rate for medicine and surgical patients in the pre-intervention HCAHPS cohort was 31% and 46% and post-intervention 30% and 39%, respectively. The NHPPES physician specific survey participation rate was 22% both pre and post intervention periods.

The HCAHPS patient sample included 5020 patients surveyed (Table 1). The pre-intervention cohort (3720 patients) was surveyed between January 2011 and April 2013 and the post-intervention cohort (1300 patients) was surveyed between June 2013 and January 2014. 33.4% of patients were in the surgical control cohort and 66.7% in the hospitalist cohort. Hospitalist and surgical HCAHPS surveyed patients had similar baseline characteristics. On average, hospitalist patients were slightly older (62.5 years versus 58.5 years), had slightly lower proportion male (46.9% versus 53.2%), lower proportion white (82.8% versus 89.1%), lower proportion with a college degree (16.4% versus 26.3%), and lower proportion English spoken at home (85% versus 93%). Additionally, hospitalist patients were admitted through the ED more often (87.9% versus 27.5%) and had greater disease burden (8.8% versus 4.1% with severity of illness score of 4) but were equally likely to be discharged home or home with services. The HCAHPS hospitalist pre and post-intervention cohorts were similar in all baseline characteristics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. HCAHPS patient characteristics by provider group over time period.

https://doi.org/10.1371/journal.pone.0180294.t001

Pre and post-intervention patients in the NHPPES survey cohort were similar (Table 2). However, post-intervention patients were slightly younger (58.7 years versus 63 years), had a slightly higher proportion male (53.4% versus 48.4%), were less severely ill (37.8% versus 47.6% with severity of illness score of 3 or 4), and were more likely to have the interview conducted in Spanish (7.7% versus 2.9%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. NHPPES patient characteristics by time period.

https://doi.org/10.1371/journal.pone.0180294.t002

After the intervention, the intercept shift for percent “always” (degree of change from pre-intervention percent excellent responses) for HCAHPS questions of “treating patients with courtesy”, “explaining things in a way patients could understand”, and “overall teamwork” showed no significant differences between surgical control and hospitalist intervention patients either before or after adjustment (adjusted p = 0.899, p = 0.890, p = 0.438, respectively) (Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. HCAHPS intercept shift by provider group over time period.

https://doi.org/10.1371/journal.pone.0180294.t003

NHPPES survey adjusted and unadjusted percent excellent responses increased significantly for the questions of “keeping patient informed” (adjusted intercept shift 9.9% P = 0.019), “overall teamwork” (adjusted intercept shift 11%, P = 0.037), and “using words the patient could understand” (adjusted intercept shift 14.8%, p = 0.001) (Table 4). Post-intervention percent excellent responses increased in the questions addressing “physicians’ explanations of treatments” (adjusted intercept shift 6.9%, p = 0.210) and “treating patients with courtesy” (adjusted intercept shift 3.3%, p = 0.453) but were not statistically significant. Fig 1 shows post-intervention percent excellent responses for all questions analyzed trended towards pre-intervention levels over time (Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Non-HCAHPS physician specific patient experience survey (NHPPES) intercept shift by time period.

https://doi.org/10.1371/journal.pone.0180294.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. HCAHPS trends in “Percent Always” and NHPPES trends in “Percent Excellent”

https://doi.org/10.1371/journal.pone.0180294.g001

Discussion

A hospitalist and IM resident physician communication simulation-based coaching method focused on specific “etiquette-based” communication behaviors through a deliberate practice model with structured feedback was not associated with significantly improved HCAHPS physician communication patient experience scores.

The lack of significant improvement on HCAHPS scores is similar to results from prior physician communication training programs with most showing no effect. [10,13,15,16,32] O’Leary et al. (2013) implemented a three session physician communication skill training program based on AIDET principles for 61 hospitalist physicians in an academic medical center but found no significant improvements in HCAHPS doctor communication domains.[11] However, the O’Leary study also notably showed no significant pre-post improvement in non-HCAHPS (Press Ganey) physician communication questions, while our intervention showed significant improvement in some physician communication questions from a similar non-HCAHPS (NHPPES) patient experience tool. Similarly, a systematic review of physician communication trainings focused on shared decision making showed that only 40% of the studies included showed improve patient satisfaction when providers were trained on communication skills.[13]

Why, then, were there no significant changes in the physician communication domains of HCAHPS scores were seen compared to surgical controls, but our secondary analysis did reveal improvement in similar domains? There are several possible explanations for these findings. First, HCAHPS scores were designed to assess and evaluate patient satisfaction performance in aggregate, at the hospital or system level. [33,36] Although often erroneously used to describe the performance of individual physicians or groups of physicians, our results may illustrate that the HCAHPS survey was not designed to be an evaluation tool of individual provider patient satisfaction performance.[37] In fact, in an analysis of 420 patients admitted to a hospitalist medicine service, the discharging hospitalist accounted for only 34% of all physician encounters. [38] Notably, despite other physicians accounting for the majority of patient care encounters, most performance improvement analyses would attribute HCAHPS outcome data to the discharging hospitalist. Research also shows specialist physicians strongly influence patients overall perceptions of physicians.[39] We also do not know if, for example, hospitalist communication improved but trainees did not similarly benefit. These factors make it difficult to accurately assess the association between HCAHPS physician communication outcomes to individual physician communication behavior or physician communication interventions. A second possible explanation for the difference between HCAHPS and NHPPES results is that fact the HCAHPS analysis examined difference over time compared to control group while the secondary analysis used a weaker, pre-post design (because we lacked an adequate control group). Therefore, while our secondary analysis of NHPPES data suggests that our intervention may have had an effect, we can only conclude that further controlled studies assessing our intervention using a physician specific patient experience survey tool (not HCAHPS) are needed.

Interestingly, despite the limitation of utilizing HCAHPS to access the effectiveness of local level QI activities, a small, single center, non-randomized IM resident education program focused on improving physician related HCAHPS scores did show significant improvements. This intervention linked resident didactic education to individualized patient experience score feedback, monthly recognitions, and tangible incentives.[31] Similarly, our secondary analysis showed improvement in patient reported “excellence” post-intervention, with 10% to 15% increases in percent “excellent” for questions related to keeping patients informed, teamwork, and using understandable patient language. However, the “bump” observed was short-lived, showing physician communication reverts to its prior state after a single interventional period. Unsurprisingly, this suggests personalized physician communication coaching may have some immediate effects that diminish without deliberate reinforcement. This conclusion is supported by Banka et al.’s (2015) study finding significant improvement in HCAHPS scores during their physician communication intervention since they attributed the improvements to ongoing reinforcement activities such as patient satisfaction score feedback, monthly recognition, and incentives for high patient-satisfaction scores.[31] One possible explanation for this finding is that regression towards the mean, a common phenomenon in improvement work requiring behavioral change, occurs if concerted efforts are not made to maintain the educational gain. This leads us to conclude that no single intervention is enough to maintain behavioral improvements over time. Instead, continual reinforcement, perhaps through ongoing simulation training and revisiting the specified skills gained in meetings and through emails, is likely critical to maintain meaningful improvements. [40] However, our intervention represents a typical “real-world” operational quality improvement project where an initial strategy is rolled out but, often due to resource limitations, a similar maintenance strategy is not undertaken. Our study highlights to health systems the critical nature of the “maintenance” phase when determining operational strategies aimed at behavior change.

We believe our intervention, while offering a model of physician communication education and coaching based on evidenced based learning methods, may have also lacked critical change management techniques of real-time data feedback, tangible behavioral incentives, and reinforcement of the specified communication skills. Unsurprisingly, systematic review shows multi-faceted interventions appear to be most effective in quality improvement.[13] However, deciphering which element(s) contribute most to the effect is difficult. Understanding which elements are key in a multi-faceted intervention is critical when adapting successful approaches to health systems with different goals.[13] Therefore, we designed our study to examine the effects of a single intervention. Unfortunately, a more messy “real world” bundled approach (i.e. a “physician communication engagement bundle”) of tactics may be necessary to see significant improvement in HCAHPS scores even if it becomes difficult to decipher which element had the most return on investment. Interestingly, more is not always better. A systematic review of 43 randomized study interventions on physician communication trainings focused on shared decision making found that short-term training (less than 10 hours) is equivalent to longer training.[13] This suggests that some repetition is necessary but that “a lot” more may not be necessary in regards to enhancing physician communication skills.

Educational interventions on physician communication show limited effect on patient experience. [10,13–16,31,32] However, studies do show the performance of basic, structured communication behaviors has an association with patient experience outcomes.[12,41] Our results, combined with prior research, still do not fully quantify the utility of structured/scripted communication methodologies as a means to improve patient reported satisfaction with physician communication.[11,41–45] Evidence of scripted communication education improving patient satisfaction remains scarce with most studies being very limited in scope and methodology.[41–45] However, these basic, yet critical skills of “etiquette based” physician-patient communication are rarely done [12] and can lead to serious communication gaps between physicians and their patients. [30]

One small pilot study of 246 emergency room encounters by medical students found the students infrequently (0.4% of encounters) use all targeted communication elements (AIDET framework) but that the use of certain elements (acknowledging a patient by name, explaining that other providers would see the patient) was associated with an increase in patient satisfaction. [41] However, while the utility of scripted patient communication tools is of indeterminate significance on the patient experience, there is ample evidence supporting the educational utility of the learning framework of our intervention. The use of simulation to teach clinical communication skills is well supported in the literature [16–25], as is the utility of teaching communication skills using a model of deliberate practice with structured feedback. [17,18,25,27] We believe our intervention’s communication skills education based on the structured AIDET mnemonic combined with focused feedback mirrors a model of deliberate practice. The structure of the education is valid since a meta-analysis found simulation based medical education with deliberate practice is superior to traditional clinical medical education in achieving specific clinical skill acquisition goal.[46] Likewise, focusing the educational framework on structured “etiquette-based” clinical communication skills is necessary since these behaviors are associated with higher patient experience scores.[12,41] Further research is necessary to fully evaluate the results of the educational framework (utilizing both simulation as well as deliberate practice) on patient reported outcomes. For example, qualitative studies could examine the aspects of communication behaviors that physicians perceived were changed by the training. Controlled studies need to assess our model’s learning framework on a physician specific patient experience. Furthermore, while much research examines the effects of improved “complex” clinician communication skills (such as goals of care discussions) has on the patient experience,[16,20,47] researchers should continue to test the effects on patient experience of the most basic “etiquette-based” elements of physician communication behavior.

Our study has several limitations. We conducted the intervention at a single academic hospital, limiting its generalizability. However, our secondary findings suggest that similar (but repeated) interventions should be studied in more sites, which would allow for control and intervention sites. Most patients who completed surveys were discharged home due to survey administration methods via phone. Similarly, we were unable to assess patient-level differences between survey respondents versus non-respondents. We were unable to determine the number of surveys administered to the patients of each physician. These factors limit generalizability of the study results. However, these limitations are prevalent in most studies utilizing hospital patient satisfaction survey results. The NHPPES survey was administered to only hospitalist patients and we did not have a comparison group for these patients. This poses an avenue ripe for further study. Our study was observational, thus subject to selection bias and confounding. We controlled for identifiable confounders such as illness severity, education, health status, etc. but it is possible that additional unidentified factors could have affected our results. Additionally, we were unable to control for the fact that non-participating hospital medicine physicians did provide some level of hospital care for some of the surveyed patients. This may have minimized our intervention’s significant effect size. We are limited by our use of post-hospitalization surveys, which tend to have a low response rate. Our response rate of 31% for HCAHPS patients is similar to the 2013 HCAHPS national average of 33%.[48] However the NHPPES response rates were appeared to be slightly lower at 22%, which could theoretically introduce bias to our results. Finally, we were limited in that we did not conduct qualitative interviews and did not use other methods to determine how the intervention changed physician behavior. However, we plan to continue to assess this in future studies.

Our research shows the limited utility of a simulation-based, deliberate practice physician communication coaching method with structured feedback to improve physician-related HCAHPS scores. It also highlights the limitations of using HCAHPS to quantify the effects of physician-level patient experience improvement strategies. Secondary analysis results demonstrate potential merit for the educational method focused on “etiquette-based” communication behaviors. Further study of this educational method utilizing controls and a period of behavioral reinforcement tactics (physician communication bundle) needs to be undertaken. Additional research needs to identify validated tools to assess patient perceptions of physician communication as it relates to local physician level quality improvement activities. [37] Without easily executed and validated tools to assess patient experience outcomes at a more granular level, we will never be able to design, implement, achieve and reinforce meaningful change in this arena. Finally, our results suggest that a broader “bundled approach” of multi-faceted physician communication improvement tactics with maintenance strategies may yield more consistent and long-lasting effects.

Supporting information

Acknowledgments

We would like to acknowledge the following for their contribution to the data analysis: Alice Ehresman, RN MBA, Michael Ehresman, MBA, Richard Brzostek

We would like to acknowledge the following for their contribution to the study design: John Brooling, MD, Behdad Besharatian, MD, Eimer Kitt, MD, Prasanna Durariraj, MD, Dev Basu, MD, Wei Boon Ooi, MD