https://orcid.org/0000-0002-2370-2588
Figures
Abstract
Dyspnea is a common, distressing symptom of cardiopulmonary and neuromuscular diseases and is defined as “a subjective experience of breathing discomfort that consists of qualitatively distinct sensations that vary in intensity.” However, Japanese intensive care units (ICUs) do not routinely screen for dyspnea, as no validated Japanese version of the Respiratory Distress Observation Scale (RDOS) is available. Therefore, we aimed to translate the English version of this questionnaire into Japanese and assess its validity and reliability. To translate the RDOS, we conducted a prospective observational study in a 12-bed ICU of a universal hospital that included 42 healthcare professionals, 10 expert panels, and 128 ventilated patients. The English version was translated into Japanese, and several cross-sectional web-based questionnaires were administered to the healthcare professionals. After completing the translation process, a validity and reliability evaluation was performed in the ventilated patients. Inter-rater reliability was evaluated using Cohen’s weighted kappa coefficient. Criterion validity was ascertained based on the correlation between RDOS and the dyspnea visual analog scale. The area under the receiver operating characteristic curve analysis was used to evaluate the ability of the RDOS to identify patients with self-reported dyspnea. The average content validity index at the scale level was 0.95. Data from the 128 patients were collected and analyzed. Cohen’s weighted kappa coefficient and the correlation coefficient between the two scales were 0.76 and 0.443 (95% confidence intervals 0.70–0.82 and 0.23–0.62), respectively. For predicting self-reported dyspnea, the area under the receiver operating characteristic curve was 0.81 (95% confidence interval 0.67–0.97). The optimal cutoff used was 1, with a sensitivity and specificity of 0.89 and 0.61, respectively. Our findings indicated that the Japanese version of the RDOS is acceptable for face validity, understandability, criterion validity, and inter-rater reliability in lightly sedated mechanically ventilated patients, indicating its clinical utility.
Citation: Sakuramoto H, Hatozaki C, Unoki T, Aikawa G, Kobayashi S, Okamoto S, et al. (2021) Translation, reliability, and validity of Japanese version of the Respiratory Distress Observation Scale. PLoS ONE 16(8): e0255991. https://doi.org/10.1371/journal.pone.0255991
Editor: Tai-Heng Chen, Kaohsuing Medical University Hospital, TAIWAN
Received: May 25, 2021; Accepted: July 28, 2021; Published: August 11, 2021
Copyright: © 2021 Sakuramoto et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This work was supported by the Japan Society for the Promotion of Science Kakenhi (Grants-in-Aid for Scientific Research) [Grant Number 19K10859]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Dyspnea is a common, distressing symptom of cardiopulmonary and neuromuscular diseases and is defined as “a subjective experience of breathing discomfort that consists of qualitatively distinct sensations that vary in intensity” [1]. In particular, intensive care unit (ICU) patients are at risk for dyspnea at different stages of their ICU stay [2–4]. In a study by Rotondi et al. [5], 22% of ICU patients recalled "not being able to get enough air through the intubation tube," and of these, 92% remembered the experience as moderately to extremely uncomfortable. Dyspnea occurs in 47% of patients undergoing mechanical ventilation [4, 6] and is associated with adverse patient outcomes, such as prolonged ventilation, anxiety, noninvasive ventilation failure, and mortality [2–4].
Despite the essential cooperation of the patients in assessing the presence and intensity of dyspnea [1], most ICU patients are unable to self-report due to lack of consciousness owing to, for example, cognitive impairment, delirium, and use of sedatives. Indeed, it has been reported that approximately 80% of patients experience delirium during their ICU stay [7]. Until recently, all dyspnea assessment tools used in Japan have been designed for conscious patients who can express themselves. In other countries, the Respiratory Distress Observation Scale (RDOS) developed in 2008 has shown good reliability and validity in patients who are unable to self-report dyspnea, such as those in palliative care or with chronic obstructive pulmonary disease, heart failure, and pneumonia [8]. Additionally, recent studies have validated the RDOS as a surrogate measure of self-reported dyspnea in critically ill patients [9]. Therefore, the RDOS is considered to be useful for the assessment of dyspnea in the ICU setting.
However, Japanese ICUs do not routinely screen for dyspnea, as no validated Japanese version of the RDOS is available for use in critically ill patients. Without validated tools, dyspnea may be neglected and go untreated in Japanese ICU patients. Therefore, this study aimed to translate the RDOS into Japanese and evaluate its validity and reliability.
Materials and methods
Design
The original RDOS was translated into Japanese, and several cross-sectional web-based questionnaires were administered to healthcare professionals to assess their content validity. Thereafter, validation and reliability studies were conducted to evaluate the criterion-related validity and inter-rater reliability.
Translation process
After receiving written permission from the original author, Dr. Campbell [8], the RDOS translation was commenced using the back-translation method based on a translation, adaptation, and validation guideline for scales [10] to ensure linguistic and cultural equivalence. The translation team was composed of three nurse educators and four clinical nurses. The translation process was conducted in four steps, also including an assessment of content validity.
- Step 1: Forward translation of the RDOS from English to Japanese was performed independently by a nurse educator and an expert nurse. The two translations were synthesized into one preliminary version through discussion between both translators to assess the equivalence of meaning between the original and translated versions.
- Step 2: Two translators who were blinded to the original RDOS translated the forward-translated version of the RDOS from Japanese into English. One translator was knowledgeable about health terminology and the content area of the construct of the instrument in English. The other translator was knowledgeable about the cultural and linguistic nuances of English. The two back-translated versions of the RDOS and the original RDOS were compared by multidisciplinary experts to assess the accuracy of terminology and clarity of expressions in terms of consistency in meaning.
- Step 3: To evaluate the clarity of the items, a pilot test of the Japanese version of the RDOS was conducted. A total of 42 healthcare professionals (27 nurses, 10 doctors, and 3 certified critical care nurses) evaluated each item of the RDOS using a dichotomous scale ("easy to understand" vs. "difficult to understand") via a web-based questionnaire. The percentage of "unknown" for each item was calculated, and items that were evaluated as "unknown" by >20% of the evaluators were revised.
- Step 4: The expert panel consisted of eight nurses, one physical therapist, and two physicians with expertise in respiratory care, who evaluated the clarity of the RDOS using the same process as in Step 3. Thereafter, each item was evaluated for content validity by an expert panel on a four-point Likert scale (1 = not relevant; 2 = unable to assess relevance; 3 = relevant but needs minor alterations; and 4 = very relevant and succinct). The average content validity index at the scale level (S-CVI/Ave) was calculated. An S-CVI/Ave of ≥0.90 was considered acceptable [10].
Criterion-related validity and inter-rater reliability of the Japanese version of the RDOS
A validation and reliability study was conducted in a 12-bed medical-surgical ICU of a universal hospital between February 2020 and February 2021.
Patients and recruitment process
We enrolled adult ICU patients who had received mechanical ventilation for ≥24 h. Patients who were expected to die within 48 h, had already received ≥24 h of mechanical ventilation prior to ICU admission, had a history of mental illness, did not understand Japanese, received neuromuscular blocking agents, or had paralysis/neuromuscular disorders were excluded. This study was approved by the Institutional Review Board of the Study Coordinator Center (Ibaraki Christian University; approval number 2019–013) and the Ethics Board of the University of Tsukuba Hospital Research Ethics Review Committee (approval number R01-184). Written informed consent was obtained from patients or their relatives according to the Institutional Review Board recommendations after providing the participants with a detailed description of the study.
Data collection
Baseline characteristics were recorded, including age, sex, diagnosis for ICU admission, ventilation status, sedative use, opiate use, and disease severity, which was calculated using the Acute Physiology and Chronic Health Evaluation II. A pair consisting of a researcher and an untrained nurse simultaneously and independently assessed an ICU patient with regards to the RDOS, depth of sedation levels using the Richmond Agitation‒Sedation Scale, delirium using the confusion assessment tool for ICUs [11], and pain using the Behavioral Pain Observational Scale. Additional evaluation using the observational dyspnea visual analog scale (D-VAS) [9, 12] was performed objectively by a trained nurse. A correlation coefficient of <0.20 was considered as a "slight, almost negligible relationship," in the range 0.20–0.40 as a "low correlation," 0.40–0.70 as a "moderate correlation," 0.70–90 as a "high correlation," and >0.90 as a "very high correlation." The main researcher was blinded to the scores of the others, and evaluation with the D-VAS was performed before the RDOS to remove any bias.
Sample size
The required sample size was calculated based on reliability, as previously described [9]. Inter-observer agreement based on previous studies was used as an estimate of moderate correlation (r = 0.44–0.76) [9, 13]. A sample size of 17–38 patients was determined to be sufficient at a significance level (α) of 0.05 and a test power (1-β) of 0.90 [14].
Data analysis
The data are expressed as numbers and percentages or as medians and interquartile ranges for nonparametrically distributed data or as means and standard deviations for parametrically distributed data. Inter-rater reliability was evaluated using a weighted Cohen’s kappa coefficient and Spearman’s rank-order correlations. Criterion validity was ascertained using Spearman’s rank-order correlations to examine the association between the RDOS and the D-VAS. The area under the receiver operating characteristic curve analysis was used to evaluate the diagnostic ability of the RDOS in identifying patients with self-reported dyspnea. The sensitivity and specificity for various RDOS cutoff points were calculated, and the Youden index was utilized to detect the optimal RDOS score cutoff for our sample. This was then compared with the original cutoff of the RDOS of ≥3 [13]. All analyses were conducted using with SPSS Statistics 25.0.
Results
Translation
The back-translated version of the RDOS was submitted to the original author for approval and then evaluated by 42 healthcare professionals. The item of "Restlessness: movement without purpose" was revised because 31% of the participants evaluated the item as "unclear." A panel of ten experts critiqued the translated RDOS and three items were evaluated as "unclear" by >20% of the panel; the translation team thus revised these items. The S-CVI/Ave was 0.95, indicating acceptable content validity. Thus, the Japanese version of the RDOS was finalized (S1 Fig; the individual in this RDOS scale photograph has given written informed consent (as outlined in the PLOS consent form) to publish these case details).
Patient characteristics
Of the 719 patients admitted to the ICU, 591 were excluded (Fig 1). The data of 128 patients were collected, and their characteristics are presented in Table 1. Each participant was evaluated either one or two times. In total, 213 observations were made. In 123/213 (65%) of these observations, the patient was able to communicate. Self-reported dyspnea was present in 23/213 patients (10.8%), and the mean D-VAS score was 0.61 (1.9). The mean RDOS score was 0.98 (1.2), and 22/213 patients (10.3%) had dyspnea with an RDOS score of >3.
MV, mechanical ventilation; NMBA, neuromuscular blocking agent.
Inter-rater reliability
In the 213 paired observations by the researcher and nurse, both RDOS scores showed excellent inter-rater reliability with a weighted Cohen’s kappa coefficient of 0.76 (95% confidence interval [CI] 0.70–0.82). The correlation of the RDOS scores measured by the researcher and nurse was 0.82 (95% CI 0.76–0.87), representing a high correlation and therefore, good inter-rater reliability.
Criterion validity
Of the 213 paired observations, criterion validity was evaluated in 66 observations excluding those in patients with delirium and/or in patients who were unable to self-report dyspnea. The correlation coefficient between the RDOS and the D-VAS was 0.443 (95% CI 0.23–0.62). For the prediction of self-reported dyspnea, the RDOS area under the receiver operating characteristic curve was 0.81 (95% CI 0.67–0.97) (Fig 2). The optimal cutoff in this study was 1 with a sensitivity and specificity of 0.89 and 0.61, respectively. The sensitivity and specificity of the original cutoff score of 3 were 0.56 and 0.95, respectively.
AUCs for the Respiratory Distress Observation Scale as a predictor of self-reported dyspnea in patients without delirium. For the prediction of self-reported dyspnea, the RDOS AUC was 0.81 (95% CI 0.67–0.97). The optimal cutoff was 1 in this study, with sensitivity = 0.89 and specificity = 0.61. For the original cutoff score of 3, the sensitivity was 0.56 and specificity was 0.95. AUC, Areas under the receiver operating characteristic curves; 95% CI, 95% confidence interval.
Discussion
This study is the first to formally translate the RDOS from English to Japanese using a multistep back-translation method based on a guideline specific to the translation, adaptation, and validation of scales [10]. The Japanese version of the RDOS was found to be acceptable in terms of face validity, understandability, criterion validity, and inter-rater reliability in lightly sedated mechanically ventilated patients. These results indicate that the Japanese version of the RDOS is acceptable for clinical use.
Translation was performed according to the standard method, and face validity, relevant validity, and understandability were established. As pointed out in a previous study, direct translation does not guarantee sufficient equivalence [10]; therefore, we used the back-translation method to correct for content variation. Back-translation by two translators with different backgrounds was performed in consideration of the differences in medical terminology and subtle nuances. Finally, face validity, relevant validity, and understandability were evaluated and established by multidisciplinary experts.
We confirmed high inter-rater reliability. These findings are consistent with those of previous studies [8, 9] and indicate that the RDOS was used accurately by every nurse. We consider that not only the text but also the pictures on how to assess RDOS likely contributed to this accuracy. Furthermore, in this study, one evaluator was not trained for using the RDOS; however, high inter-rater reliability was obtained. Accordingly, the Japanese version of the RDOS is easy to use regardless of training and can be easily performed within 1–3 min.
The criterion validity was found to be within an acceptable range. These findings are consistent with those of previous studies [9, 13]. Thus, the Japanese version of the RDOS was translated accurately and validated. Additionally, the Japanese version of the RDOS may indicate the severity of dyspnea. Thus, clinicians may be able to quantitatively evaluate dyspnea, which is important for observing the response to changes in mechanical ventilation settings and palliative care.
The cutoff point of the Japanese version of the RDOS was 1, while the cutoff point of the original version was 3. There are two potential reasons for this difference. First, translation issues may not be valid. However, this is unlikely considering that the translation process was in accordance with guidelines and the criterion validity was acceptable. Second, differences in language, ethnicity, and culture can alter the perception and expression of the sensation of dyspnea [15]. Japanese people, in particular, exhibit reduced facial emotional expression [16]. In addition, in this study, about 70% of the participants were male. Considering that the percentage of males in past surveys was also 70%, it is unlikely that there was any influence of sex on the cutoff or other factors. Further, this study included ventilated patients, therefore the effects of sedation may have caused less expression in the patient’s face. However, it was not our objective to determine the cutoff point, and the sample size was too small for this purpose. To confirm the optimal cutoff point, further studies with larger sample sizes are needed.
Clinical and research implications
The validated Japanese version of the RDOS is available for use in critically ill patients and thus should be included in their routine care [17]. The RDOS is feasible because it is easy to use and requires no training. Moreover, we believe that the scientific procedures and processes used for language translation as presented in our study can be used to conduct similar research for other languages in future studies.
Limitations
This study had several limitations. First, the Japanese version of the RDOS was evaluated in patients who were able to self-report. Ideally, the validity of the RDOS should be evaluated in patients who cannot self-report. However, this seems to be methodologically difficult. Second, the participants were ventilated patients in a single medical and surgical ICU, leading to an inherent risk of data overfitting. Moreover, the validity was not evaluated in nonventilated patients. Therefore, the results may not be generalizable to all ICU patients. However, the original RDOS has been validated in nonventilated patients; thus, the Japanese version is likely to also be useful in such patients.
Conclusion
The Japanese version of the RDOS is acceptable for face validity, understandability, criterion validity, and inter-rater reliability in lightly sedated mechanically ventilated patients, indicating its clinical utility. Further studies with larger sample sizes are needed to confirm the cutoff point of the Japanese version of the RDOS.
Supporting information
S1 Fig. Japanese version of the Respiratory Distress Observation Scale.
https://doi.org/10.1371/journal.pone.0255991.s001
(TIF)
Acknowledgments
We are grateful to Dr. Margaret L. Campbell for granting us permission to translate the RDOS and providing guidance. We would also like to thank Dr. Ryuichi Hasegawa, Dr. Yuki Enomoto, Mr. Tsuyoshi Maruyama, Ms. Megumi Moriyasu, Ms. Masako Shirasaka, Ms. Yasuyo Yoshino, Ms. Kumi Sunaoshi, Mr. Takahiro Tsujimoto, Mr. Tamoto Mitsuhiro, Ms. Mio Kitayama, and Mr. Kohsuke Sakaki for their expert comments. We would like to thank Editage (www.editage.com) for English language editing.
References
- 1. Parshall MB, Schwartzstein RM, Adams L, Banzett RB, Manning HL, Bourbeau J, et al. An official American Thoracic Society statement: update on the mechanisms, assessment, and management of dyspnea. American journal of respiratory and critical care medicine. 2012;185(4):435–52. Epub 2012/02/18. pmid:22336677; PubMed Central PMCID: PMC5448624.
- 2. Chen YJ, Hwang SL, Li CR, Yang CC, Huang KL, Lin CY, et al. Vagal withdrawal and psychological distress during ventilator weaning and the related outcomes. J Psychosom Res. 2017;101:10–6. Epub 2017/09/05. pmid:28867413.
- 3. Dangers L, Montlahuc C, Kouatchet A, Jaber S, Meziani F, Perbet S, et al. Dyspnoea in patients receiving noninvasive ventilation for acute respiratory failure: prevalence, risk factors and prognostic impact: A prospective observational study. Eur Respir J. 2018;52(2). Epub 2018/07/07. pmid:29976650.
- 4. Schmidt M, Demoule A, Polito A, Porchet R, Aboab J, Siami S, et al. Dyspnea in mechanically ventilated critically ill patients. Critical care medicine. 2011;39(9):2059–65. Epub 2011/05/17. pmid:21572329.
- 5. Rotondi AJ, Chelluri L, Sirio C, Mendelsohn A, Schulz R, Belle S, et al. Patients’ recollections of stressful experiences while receiving prolonged mechanical ventilation in an intensive care unit. Critical care medicine. 2002;30(4):746–52. Epub 2002/04/10. pmid:11940739.
- 6. Bergbom-Engberg I, Haljamae H. Assessment of patients’ experience of discomforts during respirator therapy. Critical care medicine. 1989;17(10):1068–72. Epub 1989/10/01. pmid:2791570.
- 7. Ely EW, Shintani A, Truman B, Speroff T, Gordon SM, Harrell FE Jr., et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):1753–62. Epub 2004/04/15. pmid:15082703.
- 8. Campbell ML, Templin T, Walch J. A Respiratory Distress Observation Scale for patients unable to self-report dyspnea. J Palliat Med. 2010;13(3):285–90. Epub 2010/01/19. pmid:20078243.
- 9. Chan YH, Wu HS, Yen CC, Campbell ML. Psychometric Evaluation of the Chinese Respiratory Distress Observation Scale on Critically Ill Patients With Cardiopulmonary Diseases. J Nurs Res. 2018;26(5):340–7. Epub 2017/12/09. pmid:29219939.
- 10. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract. 2011;17(2):268–74. Epub 2010/09/30. pmid:20874835.
- 11. Ely EW, Inouye SK, Bernard GR, Gordon S, Francis J, May L, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA. 2001;286(21):2703–10. Epub 2001/12/26. pmid:11730446.
- 12. Persichini R, Gay F, Schmidt M, Mayaux J, Demoule A, Morelot-Panzini C, et al. Diagnostic Accuracy of Respiratory Distress Observation Scales as Surrogates of Dyspnea Self-report in Intensive Care Unit Patients. Anesthesiology. 2015;123(4):830–7. Epub 2015/08/11. pmid:26259140.
- 13. Campbell ML, Templin TN. Intensity cut-points for the Respiratory Distress Observation Scale. Palliat Med. 2015;29(5):436–42. Epub 2015/01/31. pmid:25634631; PubMed Central PMCID: PMC4429859.
- 14.
Hulley SB CS, Browner WS, Grady D N T. Designing clinical research: an epidemiologic approach. 4th ed ed: Lippincott Williams & Wilkins; 2013. https://doi.org/10.1542/peds.2012-3818B pmid:23547043
- 15. Hardie GE, Janson S, Gold WM, Carrieri-Kohlman V, Boushey HA. Ethnic differences: word descriptors used by African-American and white asthma patients during induced bronchoconstriction. Chest. 2000;117(4):935–43. Epub 2000/04/18. pmid:10767221.
- 16. Sato W, Hyniewska S, Minemoto K, Yoshikawa S. Facial Expressions of Basic Emotions in Japanese Laypeople. Front Psychol. 2019;10:259. Epub 2019/02/28. pmid:30809180; PubMed Central PMCID: PMC6379788.
- 17. Baker KM, Vragovic NS, Banzett RB. Intensive Care Nurses’ Perceptions of Routine Dyspnea Assessment. Am J Crit Care. 2020;29(2):132–9. Epub 2020/03/03. pmid:32114611.