http://orcid.org/0000-0001-9576-895X
Figures
Abstract
Study objective
The aim of the study was to investigate factors that best predict patient’s satisfaction with their ED visit, as well as examine whether patients and their families perceived the factors related to satisfaction similarly.
Methods
This is a retrospective study where secondary data analysis was done on patient satisfaction data collected over three quarters for quality improvement purposes. Exploratory factor analysis (EFA) was conducted on the data from the first quarter to identify the factor structure, followed by confirmatory factor analysis (CFA) on the rest of the data to confirm the EFA factor structure. A structural equation model (SEM) was fitted where the factors predicted overall satisfaction with the ED visit. Finally, measurement invariance was conducted to examine if patients and families perceived the factors related to ED services alike.
Results
Two factors were found to be predictive of satisfaction: clinical team and system processes. The SEM showed that system process was a statistically significant predictor of overall satisfaction, while clinical team predicted overall satisfaction to a smaller extent. Multi-group CFA showed that the factor structure fitted neither family nor patient groups adequately. The instrument did not exhibit partial invariance.
Citation: Natesan P, Hadid D, Harb YA, Hitti E (2019) Comparing patients and families perceptions of satisfaction and predictors of overall satisfaction in the emergency department. PLoS ONE 14(8): e0221087. https://doi.org/10.1371/journal.pone.0221087
Editor: Saravana Kumar, University of South Australia, AUSTRALIA
Received: September 6, 2018; Accepted: July 30, 2019; Published: August 13, 2019
Copyright: © 2019 Natesan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Patient satisfaction is a key metric used to assess service quality in emergency departments[1]. It describes how patients perceive and value their care. Patients’ satisfaction with their emergency care has been shown to affect their compliance with discharge instructions including medications usage as well as follow up visits [2, 3]. It has also been found to influence future ED choice as well as the likelihood of recommending the ED to others [3, 4]. Furthermore, improving patient-physician interaction has been shown to improve job satisfaction of healthcare professionals and ED staff, thus contributing to a positive work environment[3, 5].
Existing ED studies have identified several factors associated with improved patient satisfaction. These include high triage acuity level of the patient[3], strong interpersonal skills and empathy of the clinical staff[6] as well as the education patients received about their condition[7]. On the other hand, perceived and expected long waiting times negatively correlate with patient satisfaction [8–11].
While multiple studies have explored predictors of patient satisfaction, to our knowledge, no study has looked at family satisfaction with the care their loved ones receive in the ED. This is important because patients often lean on family for extra support during their ED visit, make them integral to the decision-making process and rely on them as carriers of information during the follow up phase[12]. In addition, no study has examined whether the items that measure patient and family members’ satisfaction function identically across the patient and family member groups. Without testing for measurement invariance, comparison of statistics across the groups are invalid[13, 14]. Measurement invariance is the property that is exhibited by instruments without measurement bias, that is, given the same score on an underlying construct, respondents from different groups would be expected to answer identically. Finally, understanding what the predictors of satisfaction are within the Lebanese cultural context are important and can build on the growing literature on contextualizing health care to cultural needs[15].
Additionally, only few studies have investigated perceptions of the ED services using multivariate analyses[5, 16]. Studying a fundamentally multivariate construct such as ED services using univariate analyses oversimplifies and ignores the inherent complexity of the construct. Multiple hypothesis testing in such situations can lead to increased experiment wise type-I error rate. The few that have used multivariate analyses, have relied on statistical significance testing (SST) rather than a modeling approach which is susceptible to all drawbacks of SST such as sensitivity to large sample sizes, reliance on an arbitrary cut-off p-value, and lack of replicable results (see [17, 18]).
To this end, the present study investigated the factors that best predicted overall satisfaction with the ED visit in a Lebanese cultural context. The study also examined whether factors that measure satisfaction with the ED are perceived identically between patients and their families.
Methods
Study design and setting
This study utilized a secondary analysis of de-identified data of survey responses collected by our Patient Affairs office from January 2016 to September 2016. The study setting is one of the largest EDs in Lebanon, seeing approximately 57,000 ED visits annually at a tertiary care academic medical center. The majority of ED patients (78%) are discharged home. As part of routine care, a Patient Affairs Office handles all patient complaints and administers patient satisfaction surveys on care received at our inpatient units, emergency department, and specialty ambulatory clinics. The Institutional Review Board at the American University of Beirut (AUB) approved this study.
Survey instrument
In 2013, the ED leadership team and Patient Affairs developed a patient satisfaction survey based on existing ED surveys published in the peer-reviewed literature [5, 10, 19]. They customized these questions to the American University of Beirut Medical Center (AUBMC) setting based on patient complaints and process improvement initiatives within the ED. The survey included 30 questions that focus on five major domains including admission and discharge process, courtesy of staff, care and communication of clinical team, length of stay (LOS), and overall satisfaction (see S2 File).
Additionally, relevant patient demographic information was included.
Data collection
The administration of the survey began in 2016 with the goal of surveying 2% of discharged ED patients). Patient Affairs received a daily list of all patients who presented to the ED. Patients discharged home from the ED were randomly selected by choosing every third patient and calling them within 2 days of discharge for survey participation. Families were only surveyed if the patient was less than 18 years of age or if the patient was not able to complete the survey for medical reasons and a family member who accompanied the patient throughout their ED visit was willing to complete it. A verbal consent was obtained prior to administering the questionnaire. In addition, all respondents were assured confidentiality of their responses. The staff members administering the phone surveys were trained in surveying techniques.
Data analysis
The questions about patient satisfaction of the ED were rated by the respondents on a categorical scale with 1-very poor, 2-poor, 3-fair, 4-good, and 5-very good. Because the data were ordinally scaled, we used polychoric correlations for the analyses[20]. We screened the items for multicollinearity by examining their correlations. In item pairs that had correlations greater than 0.9, we deleted one of the items because essentially both items were providing the same information.
First, we conducted parallel analysis[21, 22] on the data from the first quarter to determine the number of factors to retain. Following this, we conducted an exploratory factor analysis (EFA) using principal axis factoring (PAF) on the first quarter and a confirmatory factor analysis (CFA) on the rest of the data. Instead of the five domains covered by the survey, parallel analysis indicated that two factors be retained as shown in Fig 1. EFA yielded two factors: system processes courtesy of staff, admissions and discharge process, the cleanliness of the ED, the respect of confidentiality and privacy, the overall noise of the ED, and LOS all indicated one factor)and clinical team (various aspects of care provided by the nurses and physicians). CFA on the second and third quarters based on the PAF was fitted to ensure that the factor structure held for the rest of the data as well. Model modification was done based on suggestions by the program to correlate errors of items but only if they made substantive sense. In order to predict how each factor predicted overall satisfaction of the ED, a SEM was fitted with the two factors predicting overall satisfaction with the ED and likelihood of recommending the ED to others.
Model fit was determined based on statistically non-significant χ2 value, CFI (> 0.95, Hu and Bentler, 1999), RMSEA and SRMR (< 0.08, Hu and Bentler, 1999), and RMSEA < 0.05 (good fit) and 0.05 < RMSEA < 0.08 (medium but acceptable fit, Browne & Cudeck, 1993). Finally, CFA and parallel analysis followed by PAF was conducted on patient and family data in order to test if the factor structure for the entire group held for each of these groups. For SEM that predicts overall satisfaction and measurement invariance, data from the entire sample of four quarters were used. Statistical software program R and package lavaan were used to conduct the statistical analyses.
Results
Demographics
Of the 1494 patients contacted, 705 agreed to complete the survey yielding a response rate of 50.6%. Demographics for the data are shown in Table 1. Sixty-six percent of the respondents were patients and the rest were family. Age and gender were only collected for the last three quarters of the administration. Twenty two percent of these patients were below 18 years of age. The median age of the patients was 30 with an interquartile range of 19 to 50. Fifty two percent of the respondents were female. For all these patients, their family was interviewed. For those who were above 18, only 17% interviewed were family. Mann-Whitney test of overall satisfaction with the ED showed statistical non-significance with p-values of 0.79 and 0.45 for gender and first visit, respectively. This indicates that overall satisfaction with the ED was not influenced by gender and first visit. The educational level of the respondents was not related to their overall satisfaction with the ED.
Exploratory factor analysis
During item screening, Explanations the registration staff gave you and Cooperation/helpfulness of the registration staff were deleted because they were similar to Courtesy of the staff in the registration area. Similarly, Explanations the cashiers gave you and Cooperation/helpfulness of the cashiers were deleted because they were similar to Courtesy of the cashiers. These indicated that the respondents did not clearly distinguish between courtesy, explanations, and cooperation of the registration staff and the cashiers. Additionally, Skill of nurse at inserting an Intravenous line (IVL) was deleted because more than 80% of the responses were missing. This is expected because patients may not feel qualified enough to judge whether or not the nurse was skilled while inserting an IVL, families may not know how the patient felt while the nurse inserted an IVL, and not all patients get an IVL inserted during their ED visit.
Parallel analysis indicated that two factors be retained. PAF factor coefficients for the two factors–clinical team and system processes items are shown in Table 2. The factor pattern coefficients for items on the factors they were hypothesized to indicate were greater than 0.38. This means that the items were good indicators of the factors and provides supportive evidence of convergent validity. Cronbach’s alpha for system process, clinical team, and the two factors together were .9 [.89, .92], .8 [.77, .83], and .87 [.95, .89], respectively indicating good support for internal consistency reliability of the factors. The correlation between the factors was 0.62 indicating that the factors were correlated but were distinct from each other. This provides some evidence of discriminant validity.
Confirmatory factor analysis
A CFA model (CFA-a) was fitted to a dataset for the rest of the data. This model had adequate fit except for SRMR (Table 3). However, the model modification suggested that there were error correlations between the following item pairs:
- Overall ED (LOS)–LOS before being seen by the ED physician;
- Nurses introduced themselves–Physicians introduced themselves;
- Waiting time at the registration area–Waiting time at the cashier
- Waiting time at the registration area–LOS before being seen by the ED physician.
It is logical that these items have correlations that can be explained by factors other than the two modeled; this is because of similarity in wording and explanations other than the underlying construct such as time and introduction by the clinical care professional. Therefore, a modified CFA model (CFA-b) was fitted to the data with errors from these items correlated. We retained this model because this model had slightly better fit than the previous model. Statistical non-significance of these models was not considered in model retention decisions because in reality, it is not always possible to obtain statistical non-significance for complex models such as those with more than 4 items per factor[23].
The factor pattern coefficients and the figure for CFA-b are shown in Table 2 and Fig 2, respectively. The factor correlation between clinical team and system process was 0.84. Next, the two factors, clinical team and system processes were regressed to predict overall satisfaction with the ED and the likelihood of recommending the ED to others as shown by the structural equation model (SEM) in Fig 3. Both system process (0.549, SE = 0.08) and clinical team (0.265, SE = 0.08) were statistically significant predictors of overall satisfaction. However, only system process (0.771, SE = 0.08) was a substantially large and statistically significant predictor of the likelihood of recommending the ED to others.
Our initial goal was to conduct measurement invariance to identify differences in patient and family perceptions of ED satisfaction. For this, the same factor structure and various parameters should be identical across groups. However, the simplest multigroup CFA model, that is, one with identical factor structure but varying parameters for both groups did not produce adequate fit. Therefore, we conducted PAF for both datasets individually.
Parallel analysis showed that 2 factors needed to be retained for both datasets. Therefore, PAF was conducted for family and patient datasets specifying retention of 2 factors to each dataset. The factor pattern coefficients revealed that the two groups define clinical team and system process differently (Table 2). Families perceive discharge as part of the clinical team while patients perceive discharge as part of the system process. In fact, family members perceive that admissions are part of the clinical team as seen by the small but not negligible factor pattern coefficients of these items on the clinical team factor (0.23–0.32). The fit indices of the configural invariance model that tests if the same factor structure holds true for both groups together did not pass the thresholds when fitted using multi-group CFA. Therefore, even partial invariance could not be established for the two groups.
Overall, these results suggest that the two groups define their perceptions of their satisfaction of ED services differently. Therefore, their perceptions cannot be compared directly. Neither can family’s satisfaction with the ED be used in place of patient’s satisfaction with the ED.
Discussion
This study is the first to explore multivariate predictors of satisfaction with ED visit for patients compared to their accompanying family members. The results suggest that patients and families experience ED visits differently and that different aspects of the care predict their overall satisfaction with the visit. In cultural contexts where families are a key stakeholder in the care, understanding these differences is important for managers and for driving improvement efforts.
Studies have shown that waiting time [9], courtesy of the staff, physicians’ services [10], and cleanliness to be the strongest predictors of ED satisfaction. Our findings are in line with the literature. System processes were the best predictors of overall satisfaction with ED visit in our study, followed by clinical team. LOS was not directly regressed on overall satisfaction unlike in other studies because we fitted latent variable models as opposed to conducting univariate analyses, which are only concerned with single measured variables rather than constructs. Furthermore, system processes were better predictors of likelihood of recommending the ED to others than satisfaction with the clinical team.
Several studies have revealed that waiting times [10] in the ED is the largest predictor of patient satisfaction. Our study shows that waiting time alone is not a major predictor of patient satisfaction, but the complex factor called system processes, which also includes the admission process, discharge process, cleanliness, and overall noise in the ED is.
Margaret et al. (2002) concluded that factors influencing the overall satisfaction were similar amongst both children and parents; patient satisfaction was associated with the adequacy of information provided, shorter waiting room times, and the quality of provider-patient interactions. While our study did not compare the family member’s perception directly with the patient who they accompanied (i.e. paired samples), it did include surveys of family members who had accompanied both adult and pediatric patients (i.e. independent samples). Contrary to Margaret et al.’s study, our results demonstrated that the perceptions of family members about their visit to the ED could not be compared to those of patients. While patients see only the aspects of physicians as indicators of clinical team care, families associate physician-related questions as well as questions related to the discharge process as indicators of clinical team care. This could be related to the existing process in our ED where the clinical team directs patients to the financial clearance process at discharge, in addition to being the main driver of patient discharge including patient education, and follow-up instructions. This highlights the need to handle family members’ perceptions separately and the inability to use their experiences of the ED as a proxy for patients’ perceptions. In sum, even simple comparisons of average scores by patients and families are not valid.
Limitations
The study was limited to one hospital in the Lebanese health system and may not be generalizable to other hospitals and other healthcare systems. In addition, our study did not compare the family’s perception with the perception of the patients they accompanied; rather it compared patients and families of other patients. Furthermore, the majority of respondents were patients and only 44% were family members thus limiting representation of the latter group. Finally, our sample size restricted us to fit only simple models.
Conclusion
Even though patients visit the hospital expecting high quality clinical service, their overall satisfaction is affected by other system aspects of clinical care indicated by the admission process, the discharge process, the overall LOS, the cleanliness of the ED, the respect of confidentiality and privacy, and the overall noise of the ED, rather than actual clinical care itself. Family’s experience of ED care is not the same as the patient’s and survey instruments need separate validity testing to be used for this group in settings where families are key stakeholders to care. Understanding the drivers of both patients’ and families’ satisfaction is an area of needed research. Future research may also focus on establishing a meaningful cut-point on the factor scores to determine if the care was satisfactory.
Supporting information
S1 File. De-identified data of survey responses collected between January 2016 and September 2016.
https://doi.org/10.1371/journal.pone.0221087.s001
(CSV)
Acknowledgments
The authors would like to acknowledge Miriam Saliba for her assistance with initial launch of this quality initiative.
References
- 1. Larrabee JH, Bolden LV. Defining patient-perceived quality of nursing care. Journal of nursing care quality. 2001;16(1):34–60. pmid:11668855
- 2. Rydman RJ, Roberts RR, Albrecht GL, Zalenski RJ, McDermott M. Patient satisfaction with an emergency department asthma observation unit. Academic Emergency Medicine. 1999;6(3):178–83. pmid:10192667
- 3. Hostutler JJ, Taft SH, Snyder C. Patient needs in the emergency department: nurses’ and patients’ perceptions. Journal of Nursing Administration. 1999;29(1):43–50. pmid:9921148
- 4. Hansagi H, Carlsson B, Brismar B. The urgency of care need and patient satisfaction at a hospital emergency department. Health care management review. 1992;17(2):71–5. pmid:1601663
- 5. Magaret ND, Clark TA, Warden CR, Magnusson AR, Hedges JR. Patient satisfaction in the emergency department—a survey of pediatric patients and their parents. Academic Emergency Medicine. 2002;9(12):1379–88. pmid:12460841
- 6. Shrivastava S, Shrivastava P, Ramasamy J. Exploring the dimensions of doctor-patient relationship in clinical practice in hospital settings. 2014.
- 7. Thompson DA, Yarnold PR, Williams DR, Adams SL. Effects of actual waiting time, perceived waiting time, information delivery, and expressive quality on patient satisfaction in the emergency department. Annals of emergency medicine. 1996;28(6):657–65. pmid:8953956
- 8. Ammo MA, Abu-Shaheen AK, Kobrosly S, Al-Tannir MA. Determinants of Patient Satisfaction at Tertiary Care Centers in Lebanon. Open Journal of Nursing. 2014;4(13):939.
- 9. Atari M, Atari M. Brief Emergency Department Patient Satisfaction Scale (BEPSS); development of a new practical instrument. Emergency. 2015;3(3):103. pmid:26495394
- 10. Soleimanpour H, Gholipouri C, Salarilak S, Raoufi P, Vahidi RG, Rouhi AJ, et al. Emergency department patient satisfaction survey in Imam Reza Hospital, Tabriz, Iran. International journal of emergency medicine. 2011;4(1):2.
- 11. Aaronson EL, Mort E, Sonis JD, Chang Y, White BA. Overall emergency department rating: identifying the factors that matter most to patient experience. Journal for Healthcare Quality. 2018;40(6):366–75.
- 12. Elmqvist C, Frank C. Patients’ strategies to deal with their situation at an emergency department. Scandinavian journal of caring sciences. 2015;29(1):145–51. pmid:24750520
- 13. Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural equation modeling. 2002;9(2):233–55.
- 14. Meredith W. Measurement invariance, factor analysis and factorial invariance. Psychometrika. 1993;58(4):525–43.
- 15. Truong M, Paradies Y, Priest N. Interventions to improve cultural competency in healthcare: a systematic review of reviews. BMC health services research. 2014;14(1):99.
- 16. Sun BC, Adams JG, Burstin HR. Validating a model of patient satisfaction with emergency care. Annals of emergency medicine. 2001;38(5):527–32. pmid:11679864
- 17. Thompson B. What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher. 2002;31(3):25–32.
- 18.
Cohen J. Statistical power analysis for the behavioral sciences. Hilsdale. NJ: Lawrence Earlbaum Associates. 1988;2.
- 19. Yarris LM, Frakes B, Magaret N, Adams AL, Brooks H, Norton RL. How accurately can emergency department providers estimate patient satisfaction? Western Journal of Emergency Medicine. 2012;13(4):351. pmid:22942936
- 20. Jöreskog KG. On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika. 1994;59(3):381–9.
- 21. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30(2):179–85.
- 22. Horn JL, Engstrom R. Cattell’s scree test in relation to Bartlett’s chi-square test and other observations on the number of factors problem. Multivariate Behavioral Research. 1979;14(3):283–300. pmid:26821851
- 23. Marsh HW, Muthén B, Asparouhov T, Lüdtke O, Robitzsch A, Morin AJ, et al. Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling: A Multidisciplinary Journal. 2009;16(3):439–76.