https://orcid.org/0000-0002-5846-2211
Figures
Abstract
Introduction
Worldwide, the estimated prevalence of urinary incontinence is 8.7%. Urinary incontinence is more frequent in women than in men. Posing the right questions is crucial, when diagnosing urinary incontinence, but also to evaluate the need of treatment and treatment effect. Therefore, reliable and validated questionnaires within this area are needed. Even though the International Consultation on Incontinence Questionnaire–Urinary Incontinence Short Form (ICIQ-UI SF) has been used on a daily basis in the Danish Urogynaecological Database since 2006, it has not yet been validated in a Danish population of both men and women.
Objective
To test the reliability and validity of the Danish version of the ICIQ-UI SF in a Danish speaking population of men and women among municipal employees.
Methods
Content validity was evaluated with semi-structured interviews. A quantitative field test was performed, in which the questionnaire was distributed electronically to municipal workers by E-mail. Statistical methods included item characteristics (missings, kurtosis and skewness), internal consistency (Chronbach’s alfa), test-retest (ICC), construct validity (known group validation), and floor and ceiling effect.
Results
A number of 1814 Danish municipal workers completed the questionnaire. Of the total number of responders, 426 were invited to complete the questionnaire twice (for test-retest) and 215 (50.5%) of these completed the questions again two weeks later. Statistical analyses of the ICIQ-UI SF demonstrated no floor and ceiling effects, skewness was zero and kurtosis 0.00–0.49. Cronbach’s alfa was 0.87 and intraclass correlation coefficient 0.73. Two out of three hypotheses were accepted in the known-groups validation.
Citation: Grøn Jensen LC, Boie S, Axelsen S (2022) International consultation on incontinence questionnaire – Urinary incontinence short form ICIQ-UI SF: Validation of its use in a Danish speaking population of municipal employees. PLoS ONE 17(4): e0266479. https://doi.org/10.1371/journal.pone.0266479
Editor: Peter F.W.M. Rosier, University Medical Center Utrecht, NETHERLANDS
Received: October 12, 2021; Accepted: March 22, 2022; Published: April 6, 2022
Copyright: © 2022 Grøn Jensen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because it contains sensitive personal information about Danish citizens. Owing to Danish legislation, data will be available only after approval by The Danish Data Protection Agency and with a signed access agreement. Therefore, access to the data can be received following a request to forskningsprojekter@rm.dk (The Danish Data Protection Agency – Central Region of Denmark). Project number: 656336.
Funding: This project was financed by ‘Satspuljemidler’ assigned by the Danish Health Ministry to Kontinensforeningen (a Danish patient organisation). URL: https://kontinens.org The funders did not play any role in the study design, data collection and analysis decision to publish or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: UI, Urinary Incontinence; DUGS, Danish Urogynaecology Society; BMI, Body Mass Index; ICIQ-UI SF, International Consultation on Incontinence Questionnaire–Urinary Incontinence Short Form; REDCap, Research Electronic Data Capture; COSMIN, COnsensus-based Standards for the selection of health Measurement Instruments; ICC, Intraclass correlation coefficient; PROM, Patient reported outcome measures; GDPR, General Data Protection Regulation
Introduction
Worldwide, the estimated prevalence of urinary incontinence (UI) is 8.7% [1]. A meta-analysis found mean prevalence rates for men and women to be 14.5% and 23.5% respectively, as UI is more frequent in women than in men [2]. Thereby, UI is a common problem and moreover has a profoundly negative impact on health-related quality of life both physically and socially [3, 4].
When diagnosing UI, posing the right questions is crucial, and reliable and validated questionnaires are needed in clinical as well as research settings.
The Danish ICIQ-UI SF
The International Consultation on Incontinence (ICI) recommends to use questions from their Modular Questionnaire (the ICIQ), when conducting studies on UI, to achieve international unified standardization [5]. The International Consultation on Incontinence Questionnaire–Urinary Incontinence Short Form (ICIQ-UI SF) evaluates the severity of UI symptoms and their impact on health-related quality of life. When using the ICIQ-UI SF, a total ICIQ score with a range from 0–21 is achieved from the first three questions. A score of zero means no leakage of urine and no affection on quality of life [6]. Question 1 (Q1) quantifies the frequency of urinary leaking, question 2 (Q2) evaluates the amount of leaking and question 3 (Q3) how much the urinary incontinence interferes with the everyday life.
In 1999, the questionnaire was developed by ICI sponsored by the World Health Organization in order to detect UI symptoms, their impact on quality of life and treatment outcome [5].
In Denmark, two Danish translations of the ICIQ-UI SF are used. One is approved by the ICIQ and the other is developed by the Danish Urogynaecology Society (DUGS). In accordance with previously used terminology, the version developed by DUGS will be mentioned as “VersionDUGS” [7]. VersionDUGS uses the term “leakage of urine” while the original ICIQ-UI SF uses “urinary incontinence”. VersionDUGS is widely used in Denmark and has been used on a daily basis in the Danish Urogynaecological Database (containing data from women operated for UI and prolapse) since 2006 [8]. But it has not yet been validated in the general Danish population with both men and women, making it impossible for researchers and clinicians to cater the psychometric and measurement properties of the questionnaire. One validation study has been conducted on the Danish ICIQ-UI SF but this did only include women [7].
Even though validation studies of the English ICIQ-UI SF have been performed, values from Cronbach’s alfa and kappa statistics range significantly and information about ceiling effect and differential item functioning are limited [9].
Since the ICIQ-UI SF is the recommended first choice instrument, validity and reliability are still continuously relevant to evaluate, and especially when using the instrument on a new study population never tested with the ICIQ-UI SF before. Evaluation of validity and reliability of the Danish ICIQ-UI SF has never been performed in a large study sample of different sexes or in the general Danish population, even though UI affects all sexes and age groups.
The aim of this study was to test the validity and reliability of the Danish translated version of the ICIQ-UI SF, VersionDUGS, in a population reflecting the general Danish population.
UI is not only a problem among women and the Danish translated version of ICIQ-UI SF, VersionDUGS, is not only used within gynaecology but widely used within different specialities such as general practitioners, urology, neurology, and physiotherapy. Therefore, is it highly relevant with an evaluation of the validity and reliability in a big study sample with different sexes, a wide age distribution and a diverse education level. This study provides clinicians and researchers with such an evaluation.
Methods
The questionnaire
Additional questions were included to supplement the ICIQ-UI SF with background data and information about chronic diseases. Background data include information about age, sex, Body Mass Index, education level, smoking habits, highest completed education, civil status, and chronic diseases. The questionnaire is shown in S1 Table. The questionnaire was created in and distributed from REDCap—Research Electronic Data Capture, a secure web-based platform for construction and management of surveys and online databases [10].
The original ICIQ-UI SF in English and all translated versions can be found on ICIQ’s official webpage and all copyrights of these are preserved by ICIQ [11].
Approval to use VersionDUGS was provided in a written form from DUGS.
Pilot test: Content validity
VersionDUGS was pilot tested to evaluate face validity and content validity. The COSMIN panel defines face validity as ‘the degree to which a measurement instrument, indeed, looks as though it is an adequate reflection of the construct to be measured’, and the content validity as ‘the degree to which the content of a measurement instrument is an adequate reflection of the construct to be measured’ [12]. In accordance with the COSMIN checklist, content validity was evaluated with a pilot test on a small sample size from a relevant population—in this case 14 public workers (nine women and five men, from 27 to 63 years old).
Pilot-testing was performed with semi-structured interviews with the 14 public workers during and after they had answered the questionnaire. Participants to the pilot-test were recruited with informative emails to employees in a specific department with permission from the management. The ‘Three-step Test-interview’ (TSTI) was the basis for the semi-structured interviews. It combines the ‘think loud’ and ‘probing’ methods, which makes it a powerful method for evaluating the comprehension and comprehensibility [12]. The three steps in the TSTI consist of: 1) Observational data through concurrent think loud; observation of the respondent’s behaviour while filling in the questionnaire and asking the respondent to ‘think loud’, 2) Focused interview–clarifying the observed; the interviewer asks with the purpose of filling in gaps of the observational data and 3) Semi-structured interview–eliciting experiences; questions about response behaviour, wording in the questionnaire and understanding of definitions [13].
The participants were asked about the relevance of each item, the comprehensiveness of the questionnaire, the comprehensibility of the instructions, items and response options and lastly, if they missed any items or response options. Furthermore, six medical professionals within the area of focus were asked about the relevance of each item and the comprehensiveness.
Quantitative field test of the ICIQ-UI SF
The validity and reliability of VersionDUGS were investigated in a quantitative field-test. Participants were recruited through the municipal of their workplace. A formal invitation to participate including a study presentation and instructions were sent to the directors of the municipal. The directors were asked to distribute an email invitation with the open link questionnaire together with information about data storage and GDPR to their municipal employees. Several of the participating municipals distributed the invitation internally. For this reason, it is not possible to report the response rate. Inclusion criteria were age over 18 years and municipal employment. In the bottom of the questionnaire, we asked for their permission to send them the same questionnaire once again (for test-retest).
Statistical analysis
STATA was used for statistical analyses. Statistical methods include item characteristics (missings, kurtosis and skewness), internal consistency (Cronbach’s alfa), test-retest (ICC), construct validity (known group validation), and floor and ceiling effect.
Item characteristics and floor and ceiling effects.
Skewness and kurtosis are both statistical measures for distribution and reveal if answers are normal distributed variables. If skewness equals zero it reflects a normal distribution while a negative skewness reflects a left-skewed distribution where the mean is lower than the median [14]. If kurtosis equals three it reflects a normal distribution of the variable, while higher than three is called leptokurtic and below three platykurtic. A leptokurtic distribution (high kurtosis) is characterized by a certain amount of peakedness while a playtykurtic distribution (low kurtosis) is characterized by a certain amount of flatness with fewer and less extreme outliers than normal distributed variables [15].
Internal consistency.
The COSMIN panel defines internal consistency as the interrelatedness between items and is only relevant to perform on patient reported outcome measures of the reflective model and when all items form a unidimensional scale [16]. Internal consistency is calculated for Q1-Q3 in the ICIQ-UI SF with Cronbach’s alfa [17].
Reliability—test-retest.
Intraclass correlation coefficient (agreement) was calculated for Q1-Q3 in the ICIQ-UI SF [18]. An ICC under 0,5 is poor, between 0.5–0.75 moderate, over 0.75 good and over 0.9 is considered excellent [19]. Standard error of measurement-agreement (SEMagreement) and then the smallest real different (SRD) was calculated with the following equations:
Construct validity.
Three hypotheses were tested: 1) women are more likely to be urinary incontinent than men [20], 2) participants older than 50 years are more likely to be urinary incontinent than participants younger than 50 years [21], 3) participants with a BMI≥30 are more likely to be urinary incontinent than participants with BMI<30 [22, 23].
Ethics
The project was approved by the Danish Data Agency (656336) and The National Ethical Committee has approved that this project was carried out without their involvement (1-10-72-186-19).
Consent to participate in the evaluation was achieved in a written enquiry. We sent the information about the evaluation to the directors of the participating municipals and asked them if they would allow their employees to participate. If they accepted the participation, the same information about purpose, questionnaire, time consumption, GDPR rules, anonymization of the responder, data protection and -storage, ethics of the study and that participation could be interrupted at any time was sent to the employees.
Results
Interview findings: Content validity
Interview participants found the ICIQ-UI SF comprehensive and easy to complete. Therefore, the three step test interviews did not lead to any changes.
Field test
The questionnaire was electronically distributed to municipal workers in 16 Danish municipals from 20.01.2020 to 11.05.2020. A total of 1825 persons opened the questionnaire, but 11 of these did not complete any items and were excluded. Therefore, the final study sample consisted of 1814 participants. Among the responders, 426 were invited to answer the questionnaire again after two weeks and 215 of these answered and were included in test-retest (response rate for second round = 50.5%). A total of 418 of the 1814 responders (23%) reported urinary incontinence. The participants reported demographics as shown in Table 1.
ICIQ-UI SF, VersionDUGS
Item characteristics and floor and ceiling effects.
Floor effect of the ICIQ-score was 4.26% and ceiling effect was 0.24%, indicating that the ICIQ-UI SF manages to differentiate between the respondents.
Skewness was close to zero in Q1-Q3, while kurtosis was 2.79 in Q1, 13.28 in Q2 and 2.88 in Q3 as shown in Table 2.
The skewness of Q1-Q3 is positive but close to zero and thereby indicates a distribution of the answers close to a normal distribution but slightly right-skewed. The kurtosis of Q1 and Q3 indicates a platykurtic distribution with fewer and less extreme outliers than a normal distribution [15]. The kurtosis of Q2 = 13.28 is per definition a leptokurtic distribution, which produces more outliers than a normal distribution.
Reliability.
Of 426 invited respondents, 215 completed the questionnaire again after two weeks for test-retest (response rate = 50.47%). The intraclass correlation coefficient (agreement) was calculated for these participants and was 0.73 for Q1-Q3. SEMagreement was calculated:
Construct validity–known-groups validation.
Risk differences were calculated to test three hypotheses. Two out of three hypotheses were accepted: women are more likely to have incontinence than men, RD = 0.22 (95%-CI:0.17–0.25) and participants with a BMI≥30 are more likely to have incontinence than participants with BMI<30, RD = 0.11 (95%-CI: 0.06–0.16).
Discussion
This is the first validation study of the Danish translated version of ICIQ-UI SF, VersionDUGS in a population reflecting the general Danish population.
Interview-participants found the questions appropriate, comprehensive, and easy to complete.
We had 0 missing answers for Q1 and Q2, and 1.2% for Q3 which is consistent with the number of missings in other validation studies [7, 24]. Floor and ceiling effects as well as skewness and kurtosis were acceptable in our study group.
The Cronbach’s alfa was high (0.87) compared to the validation study conducted by Clausen et al. reporting a Cronbach’s’ alfa of 0.7 [7]. Cronbach’s alfa ranges from 0.71–0.78 in validation studies of the ICIQ-UI SF in languages such as Persian, Chinese, and Japanese [25–27], while higher values of Cronbach’s alfa has been shown in validation studies from Croatia and Slovenia on 0.85 and 0.81, respectively [28, 29]. Cronbach’s alfa is highly affected by the variation of the population and participants in a heterogenous population will have a higher Cronbach’s alfa than participants in a more homogenous population [30]. Our population consists of different sexes, have a wide age distribution, and represents a broad level of education, making it a very heterogenous population. This could explain why VersionDUGS has a higher Cronbach’s alfa in our validation study than in the Danish validation study by Clausen et al. [7].
The intraclass correlation coefficient (ICC) of 0.73 is moderate. An ICC of 0.73 is acceptable and indicates that responders of the Danish translated version of ICIQ-UI SF, VersionDUGS did not differ their answers in a two weeks’ time period [31]. Clausen et al. also showed stable test-retest results in a group consisting of Danish women and studies from other countries show test-retest results from acceptable to excellent [5, 7, 32]. While ICC is unitless, the SEM of 1.78 has the same unit as the measurement score. The SRD of 4.9 was calculated from the SEM and tells us that an individual’s difference on repeated testing on 4.9 or greater will reflect a real difference in 95% of the cases. This is a relatively high SRD, indicating high measurement error which clinicians may want to consider, when using VersionDUGS.
Finally, two out of three hypotheses were accepted in the known-groups validation.
Strengths and limitations
As in any qualitative study, interview findings in the pilot test of the questionnaire could be biased by the interviewer’s preunderstanding of the field. However, using a specific interview model (TSTI) and making the pilot test structured and systematic helped reducing this type of bias [12, 13]. Being conscious of our preunderstanding when evaluating the interviews decreased any potential bias as well. Finally, adopting the relevant design criteria of the COSMIN checklist in the quantitative field test of the ICIQ-UI SF reduced any idiosyncrasies [33, 34]. Nevertheless, the quantitative field test had some possible limitations. Only municipal workers were included, which should be taken into account when using VersionDUGS in other populations. This may affect the generalization of our results to an arbitrary population. However, the large study group had no other specific characteristics making them less representative. Moreover, it is by far the largest study group of different sexes in a validation study of the ICIQ-UI SF. Unfortunately, due to the open link invitation, it was not possible to estimate response rates or evaluate if the non-responders differed in terms of baseline characteristics from the responders. Nevertheless, the recruitment method is justified by the goal of achieving a large study group with a broad age distribution and the fact that no other opportunities were available when targeting municipal workers.
Conclusion
The Danish translated version of ICIQ UI SF, VersionDUGS is a valid and reliable measure of urinary incontinence in a Danish population consisting of different sexes. However, clinicians should consider the relatively high measurement error of VersionDUGS.
References
- 1. Irwin DE, Milsom I, Hunskaar S, Reilly K, Kopp Z, Herschorn S, et al. Population-based survey of urinary incontinence, overactive bladder, and other lower urinary tract symptoms in five countries: results of the EPIC study. Eur Urol. 2006;50(6):1306–14; discussion 14–5. pmid:17049716
- 2. Hampel C, Wienhold D, Benken N, Eggersmann C, Thuroff JW. Definition of overactive bladder and epidemiology of urinary incontinence. Urology. 1997;50(6A Suppl):4–14; discussion 5–7. pmid:9426746
- 3. Lukacz ES, Santiago-Lastra Y, Albo ME, Brubaker L. Urinary Incontinence in Women: A Review. JAMA. 2017;318(16):1592–604. pmid:29067433
- 4. Aoki Y, Brown HW, Brubaker L, Cornu JN, Daly JO, Cartwright R. Urinary incontinence in women. Nat Rev Dis Primers. 2017;6(3):17042.
- 5. Lim R, Liong ML, Lau YK, Yuen KH. Validity, reliability, and responsiveness of the ICIQ-UI SF and ICIQ-LUTSqol in the Malaysian population. Neurourol Urodyn. 2017;36(2):438–42. pmid:26693962
- 6. Klovning A, Avery K, Sandvik H, Hunskaar S. Comparison of two questionnaires for assessing the severity of urinary incontinence: The ICIQ-UI SF versus the incontinence severity index. Neurourol Urodyn. 2009;28(5):411–5. pmid:19214996
- 7. Clausen J, Gimbel H, Arenholt LTS, Lowenstein E. Validity and reliability of two Danish versions of the ICIQ-UI SF. Int Urogynecol J. 2021. Dec; 32(12):3223–3233. pmid:33646350
- 8. Guldberg R, Brostrom S, Hansen JK, Kaerlev L, Gradel KO, Norgard BM, et al. The Danish Urogynaecological Database: establishment, completeness and validity. Int Urogynecol J. 2013;24(6):983–90. pmid:23073539
- 9. Kurzawa Z, Sutherland JM, Crump T, Liu G. Measuring quality of life in patients with stress urinary incontinence: is the ICIQ-UI-SF adequate? Qual Life Res. 2018;27(8):2189–94. pmid:29737448
- 10. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81. pmid:18929686
- 11.
Institute BU. The International Consultation on Incontinence Questionnaire 2014–2021 [Available from: https://iciq.net].
- 12.
de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine A Practical Guide Cambridge Cambridge University Press; 2011. ISBN: 9780521118200.
- 13. Hak T, Veer Kvd, Jansen H. The Three-Step Test-Interview (TSTI): An observation-based method for pretesting self-completion questionnaires. Survey Research Methods. 2008;2(3):143–150.
- 14. Ho AD, Yu CC. Descriptive Statistics for Modern Test Score Distributions: Skewness, Kurtosis, Discreteness, and Ceiling Effects. Educ Psychol Meas. 2015;75(3):365–88. pmid:29795825
- 15. Cain MK, Zhang Z, Yuan KH. Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence and estimation. Behav Res Methods. 2017;49(5):1716–35. pmid:27752968
- 16. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–45. pmid:20494804
- 17. Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn’t matter. J Pers Assess. 2003;80(3):217–22. pmid:12763696
- 18. de Vet HC, Terwee CB, Knol DL, Bouter LM. When to use agreement versus reliability measures. J Clin Epidemiol. 2006;59(10):1033–9. pmid:16980142
- 19. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155–63. pmid:27330520
- 20. Junqueira JB, Santos V. Urinary incontinence in hospital patients: prevalence and associated factors. Rev Lat Am Enfermagem. 2018;25:e2970. pmid:29319744
- 21. Batmani S, Jalali R, Mohammadi M, Bokaee S. Prevalence and factors related to urinary incontinence in older adults women worldwide: a comprehensive systematic review and meta-analysis of observational studies. BMC Geriatr. 2021;29(1):212. pmid:33781236
- 22.
Bent Ottesen Ole Mogensen, Axel Forman. Gynækologi. Munksgaard, København 2011. 4th edition. ISBN: 9788762809253.
- 23. Elia G, Dye TD, Scariati PD. Body Mass Index and Urinary Symptoms in Women. International Urogynecology Journal 2001;12(6):366–9. pmid:11795637
- 24. Avery K, Donovan J, Peters TJ, Shaw C, Gotoh M, Abrams P. ICIQ: a brief and robust measure for evaluating the symptoms and impact of urinary incontinence. Neurourol Urodyn. 2004;23(4):322–30. pmid:15227649
- 25. Hajebrahimi S, Nourizadeh D, Hamedani R, Pezeshki MZ. Validity and reliability of the International Consultation on Incontinence Questionnaire-Urinary Incontinence Short Form and its correlation with urodynamic findings. Urol J. 2012;9(4):685–90. pmid:23235974
- 26. Huang L, Zhang SW, Wu SL, Ma L, Deng XH. The Chinese version of ICIQ: a useful tool in clinical practice and research on urinary incontinence. Neurourol Urodyn. 2008;27(6):522–4. pmid:18351586
- 27. Gotoh M, Homma Y, Funahashi Y, Matsukawa Y, Kato M. Psychometric validation of the Japanese version of the International Consultation on Incontinence Questionnaire-Short Form. Int J Urol. 2009;16(3):303–6. pmid:19207608
- 28. Mikus M, Coric M, Matak L, Skegro B, Vujic G, Banovic V. Validation of the UDI-6 and the ICIQ-UI SF—Croatian version. Int Urogynecol J. 2020;31(12):2625–30. pmid:32821964
- 29. Rotar M, Trsinar B, Kisner K, Barbic M, Sedlar A, Gruden J, et al. Correlations between the ICIQ-UI short form and urodynamic diagnosis. Neurourol Urodyn. 2009;28(6):501–5. pmid:19260080
- 30. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. International Journal of Medical Education. 2011;27(2):53–55. pmid:28029643
- 31. Perinetti G. StaTips Part IV: Selection, interpretation and reporting of the intraclass correlation coefficient. South Eur J Orthod Dentofac Res. 2018;5(1):3–5.
- 32. Tubaro A, Zattoni F, Prezioso D, Scarpa RM, Pesce F, Rizzi CA, et al. Italian validation of the International Consultation on Incontinence Questionnaires. BJU Int. 2006;97(1):101–8. pmid:16336337
- 33. Mokkink LB, Terwee CB, Gibbons E, Stratford PW, Alonso J, Patrick DL, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol. 2010;10:82. pmid:20860789
- 34. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651–7. pmid:21732199