http://orcid.org/0000-0001-9246-7721
Figures
Abstract
Objective
Immune checkpoint inhibitors (ICIs) have recently shown tremendous promise in the treatment of diverse cancers. The available data suggests that ICIs are well tolerated in terms of health-related quality of life (HRQOL) compared to other anticancer therapies. However, it appears that instruments currently used to evaluate HRQOL in this context may fail to capture important symptomatology unique to ICIs. This systematic review was designed to assess the adequacy of methods used to report HRQOL in cancer patients treated with ICIs and to identify the quality of life scales used.
Method
A systematic review was performed (systematic registration number: PROSPERO: CRD42019121427). A search of the PubMed, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences collection, and SocINDEX databases was carried out for publications in English and in French. Relevant databases were searched from the earliest records through to March 2019. Publications were selected if they reported on HRQOL in patients with cancer treated by ICIs. Risk of bias was scored using the Cochrane Collaboration bias assessment tool.
Results
Our search identified 144 publications between 2012 and 2019, of which 15 RCTs met the inclusion criteria. The results suggest that even though the overall reporting of HRQOL was deemed to be of good quality, the data available was marred by methodological aspects such as the lack of HRQOL research hypotheses and the lack of questionnaires validated for cancer patients treated with immunotherapy.
Citation: Faury S, Foucaud J (2020) Health-related quality of life in cancer patients treated with immune checkpoint inhibitors: A systematic review on reporting of methods in randomized controlled trials. PLoS ONE 15(1): e0227344. https://doi.org/10.1371/journal.pone.0227344
Editor: Ali Montazeri, Iranian Institute for Health Sciences Research, ISLAMIC REPUBLIC OF IRAN
Received: July 13, 2019; Accepted: December 15, 2019; Published: January 24, 2020
Copyright: © 2020 Faury, Foucaud. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: Financial support for this research was provided by INCa (SF) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Cancer immunotherapy has revolutionised the treatment of cancer and represents a new option for clinicians [1–3]. Immune Checkpoint Inhibitors (ICIs) have garnered attention as one of the most promising types of immunotherapy [4,5]. The U.S. Food and Drug Administration (FDA) approved seven ICIs for clinical use: Ipilimumab, Pembrolizumab, Nivolumab, Atezolizumab, Durvalumab, Avelumab and Cemiplimab [6].
Checkpoint inhibition is associated with a unique spectrum of side effects known as immune-related adverse events (irAEs), defined as toxicities caused by non-specific activation of the immune system [7, 8, 9]. Some irAEs can be serious and even life-threatening, and thus substantially affect patients’ daily lives [10]. Abdel-Rahman et al. (2018) performed a systematic review to explore time to HRQOL deterioration among cancer patients receiving PD-(L)1 inhibitors compared to those receiving traditional cytotoxic therapy. The results showed that PD-(L)1 inhibitors were associated with a consistent prolongation of the time to symptomatic deterioration [11]. Recently, Hall et al. (2019), in their systematic review, suggested that ICIs are well tolerated in terms of HRQOL compared to other anticancer therapies. However, for these authors, currently used instruments may fail to capture important symptomatology unique to ICIs [12]. For example, in the CheckMate141, trial of Nivolumab versus therapy of investigator’s choice in recurrent or metastatic head and neck, the QLQ-H&N35 was applied to evaluate HRQOL. But skin toxicities (e.g. o, L er). QOL e prominent role as important endpoints in cancer RCTs.atuc review to i …ndart therapy arm (t not be covered.contsuchrash and pruritus), the most frequent irAEs with Nivolumab [8,13], are not assessed with this HRQOL measure [13]. Results of Hall et al. (2019) should therefore be interpreted with caution.
To date, HRQOL refers to a multidimensional concept including the domains of physical, emotional and social functioning [14]. Assessing HRQOL in cancer patients is necessary to evaluate the full impact of the cancer experience [15] and to improve understanding of how different therapeutic interventions influence patients’ outcomes [16]. HRQOL is increasingly recognized as an essential end point in cancer clinical trials [17]. HRQOL outcomes from RCTs increasingly inform cost-effectiveness analyses used by policymakers to decide on the allocation of resources [18,19]. Therefore, it is imperative that findings from RCTs are robust.
HRQOL evaluations are made using standardised and validated self-assessment methods [20]. The most important methodological issue to consider in evaluating HRQOL endpoints in an oncology clinical trial is the selection of appropriate outcome measurements [21]. There is also evidence that the potentially invaluable insights that HRQOL data provide on the treatment and care of patients may not be adequately reported [19,22–24].
To our knowledge, the reporting of quality-of-life methods in trials of cancer patients treated with ICIs has not been systematically assessed. The following systematic review aims to evaluate the quality of HRQOL methodological assessment used to evaluate the effects of ICIs on HRQOL in randomised controlled trials (RCTs) to determine how improvements can be made, and to explore the value added by these measures to clinical decision-making in a trial setting. We focused on RCTs they represent the gold standard for evaluating new medical treatments, developing decision-making policies, and planning new treatment approaches [25,26].
Method
In accordance with French regulations, this study was exempted from IRB approval. The methods discussed in this review were previously published in the PROSPERO database [27], under number CRD42019121427. (http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42019121427).
Search strategy
We used standard procedures: Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [28,29], in accordance with the principles outlined in the Cochrane Handbook for Systematic Reviews of Interventions [30]. A systematic literature search was performed from 1st March 2019; we searched for all RCTs published up to 28th February 2019, irrespective of their start or completion date. We searched PubMed, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences collection, SocINDEX, limiting the search to RCTs of adults (≥ 18 years at diagnosis) published in English or French, and confined to original human studies published in peer-reviewed journals. Companion papers that focused only on HRQOL were reviewed with the original publication.
We identified relevant references using the following terms: “quality of life”, “cancer”, “neoplasms” and individual drug names. The reference lists of key papers were checked to find relevant references for inclusion. Note that, when we started our research (March 2019), the Cemiplimab had been recently approved by the FDA (September 2018). It was not specifically examined in our search strategy due to the lack of data on HRQOL available in March 2019 for this ICIs (clinicaltrials.gov).
The full search algorithm, based on Hall et al. [12], used to identify potential studies in PubMed is included in S1 Search algorithm and was adapted for the other databases.
Selection criteria
The patient population assessed included patients with any cancer randomised to treatment with an approved ICI. Predefined exclusion criteria were: RCTs that assessed treatment that did not include ICIs, RCTs on patients with other illnesses, phase I studies, and RCTs with fewer than ten patients per group. As a single domain (e.g. fatigue) is not considered as HRQOL, trials assessing only one aspect of HRQOL were excluded. Companion papers focusing on a subgroup of the total RCT sample were excluded. Unpublished reports, conference abstracts, and dissertations were excluded due to the lack of peer-review oversight. These are often subsequently published in peer-reviewed journals, risking duplication.
Data extraction
An extensive search of electronic databases was conducted by one person (SF). Literature search results were uploaded to Zotero (www.zotero.org) which facilitates bibliographic source management. Following the removal of duplicates, two reviewers (SF and JF) independently screened titles and abstracts of all references identified by the search according to eligibility criteria. Full articles were obtained for all titles that either appeared to meet the inclusion criteria or where uncertainty as to the eligibility criteria existed. A final selection pertaining to the suitability of the full-text papers to be analysed was verified by two reviewers (SF and JF). A third reviewer (CC) was available as a mediator in the event of disagreement.
Data analysis
The criteria used to evaluate quality of reporting on HRQOL were based on those proposed by Efficace et al. [25]. On the basis of good practice in reporting HRQOL [24, 31–34], Efficace et al. [25] extracted 11 basic and essential issues that a given trial should report to reach methodologically sound outcomes. The checklist items were devised to have a dichotomous answer: these can be scored as ‘yes’ (giving a score of 1) or ‘no’ (giving a score of 0), with higher scores indicating the robustness of the outcomes. The items included in the checklist are self-explanatory and a brief description is also provided in S1 Table.
Determining study quality
Risk of bias was assessed by two reviewers using the Cochrane Collaboration bias assessment tool [35]. Responses in each domain (random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data and selective outcome reporting) were assessed as having a ‘low’, ‘unclear’ or ‘high’ risk of bias.
Results
Key characteristics of identified trials
The literature search yielded 144 records. After removal of duplicates, the titles and abstracts of 138 records were screened; 109 records were identified as clearly non-relevant and consequently excluded: full-text articles were thus obtained for 29 papers. Fourteen of these publications were excluded as not meeting the selection criteria (e.g. lack of HRQOL endpoint, protocol summary, RCTs, specific subgroup of RCTs). This gave a total of 15 publications to be included in the systematic review (Fig 1). Publications covered a seven–year period (2012–2019). The key demographic characteristic results from the 15 RCTs included in this systematic review are summarised in Table 1.
HRQOL = health-related quality of life. RCTs = randomized controlled trials. ICIs = immune checkpoint inhibitors. * = some RCTs included more than one ICI.
Nine of the 15 RCTs were published in high-impact journals (> 10). For 13 of the 15 identified RCTs, the HRQOL data were published in a companion paper. Results of HRQOL data are not presented in the main papers of these 13 RCTs [36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48]. Eight of the thirteen HRQOL companion papers were published in the year after publication of the main clinical results, four within two years, and one within three years. All trials were industry-sponsored and/or affiliated with the pharmaceutical industry through one or more investigators, as noted in the financial sponsorship statements of the articles. All trials were multicentre trials and performed in more than one country. In total, 9332 patients were enrolled in the 15 RCT trials. Trial size ranged from 272 to 951 patients. The ICIs studied in these RCTs were Ipilimumab (33%), Pembrolizumab (27%), Atezolizumab (7%) and Nivolumab (47%). No relevant RCTs with HRQOL assessment focused on Durvalumab or Avelumab. The non-checkpoint inhibitor controls used in some of the studies included conventional treatment, chemotherapy, sunitinib, placebo, or everolimus, and gp100. Eight RCTs assessed ICI treatments for melanoma, three for lung cancer, one for head and neck squamous cell carcinoma, one for urothelial cancer, and two for advanced renal cell carcinoma. All trials had overall survival or progression-free survival as the primary endpoint; all reported significant differences in one of these endpoints between treatment arms. HRQOL was a secondary endpoint in nine trials [49,50,51,52,53,54,55,56,57], an exploratory endpoint in five RCTs [58,59,60,61,62] and for one trial it was unclear if the HRQOL endpoints were secondary or exploratory [63]. Seven trials discussed HRQOL in their main publication [50,59,60,52,61,56,57].
Quality of HRQOL measurements
The conceptual measurement and methodology for evaluating HRQOL outcomes from the 15 RCTs included in this systematic review are summarised in Table 2 and S2 Table. HRQOL was measured with common cancer HRQOL assessment tools in all of the studies. The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire (EORTC QLQ-C30, used in twelve of the 15 RCTs), and the European Quality of Life 5 Dimension (EQ-5D, used in eleven of the 15 trials) were the most frequently used. In two of the twelve studies, the EORTC QLQ-C30 was supplemented with the EORTC Quality of Life 13-item Lung Cancer-specific Questionnaire (EORTC QLQ-LC13), and, in another, with the EORTC module for Head & Neck cancer (EORTC QLQ-H&N35). Additional HRQOL instruments used included the Functional Assessment of Cancer Therapy (FKSI-DRS or FKSI-19, used in two trials), the Functional Assessment of Cancer Therapy-General (FACT-G, used in one trial), and the Lung Cancer Symptom Scale (LCSS, used in one trial). Twelve trials used more than one HRQOL questionnaire.
Five of the 15 RCTs did not specify the HRQOL research hypothesis due to the HRQOL endpoint being an exploratory objective [58,59,60,61,62]. For Efficace et al. [25], to satisfy the “a priori hypothesis stated” criterion, the study needed to predefine HRQOL end point and/or state expected changes because of the specific treatment. Only five trials of the 15 RCTs satisfied this criterion but none of them included an a priori hypothesis for the expected HRQOL outcomes [49,50,51,54,55].
Two of the 15 studies failed to report their reasoning for using a specific HRQOL measure and none gave a reason for choosing a particular HRQOL instrument. For 14 trials, the validity and reliability (“psychometric properties reported” criterion) of the instruments used was reported by referencing the appropriate validation studies; one trial provided no reference to validity or reliability [62]. All but one of the questionnaires (EQ-5D) were validated for the specific cancer population (“Cultural validity verified” criterion). The EQ-5D is a non-cancer-specific measure of generic health for clinical and economic appraisal. None of the questionnaires used in these 15 trials (EORTC questionnaires, EQ-5D, LCSS, FKSI, FACT-G, etc.) were validated among cancer patients treated with ICIs (“adequacy of domains covered” criterion). Checkpoint inhibition is associated with a unique spectrum of side effects that are not assessed with the HRQOL measure used. The HRQOL domains covered by the questionnaires were inadequate for several of the trials identified. Details describing how the HRQOL assessment was done were often not reported. Some details were noted for only eight of the trials (“instrument administration reported” criterion). All the RCTs reported the timing of HRQOL assessment and documented missing data.
We observed that presentation of the HRQOL results was adequate for all but two of the trials. For these two trial presentations of HRQOL, the results were defined as limited (main publication of RCTs), with incomplete HRQOL details and a discussion of the HRQOL outcomes in terms of clinical significance not reported. HRQOL data was judged as high quality if at least eight of the 11 criteria were satisfied; furthermore, three of the eight or more criteria satisfied needed to cover the three high-priority concerns identified by the experts (“baseline compliance reported,” “psychometric properties reported,” and “missing data documented”). If one or two of the 11 items on the checklist were assessed as not applicable, the cutoff was set at seven or six criteria respectively (although the three mandatory criteria still had to be met). Only for ten trials of the 15 RCTs HRQOL data was judged as high quality [49,58,50,59,51,60,52,54,55,63]. The others trials were considered to have some possible reported methodological limitations because of studies addressed (1) at least eight (or seven if one of the 11 items on the checklist were assessed as not applicable) issues but did not take into account the mandatory items [52, 61, 62]; (2) fewer than eight (or seven) issues [56,57].
Risk of bias
Fig 2 summarises the risk of bias for all of the RCTs evaluated. All the RCTs used block randomisation and/or stratified randomisation to generate the random sequence and were assessed as having a ‘low’ risk of bias for random sequence generations. For ten RCTs, participants were randomly assigned by an interactive voice response system. These trials were assessed as having a ‘low’ risk of bias for allocation concealment. The five other studies were assessed as having an ‘unclear’ risk of bias for this item. Ten RCTs were open-label (trial in which both the researchers/investigators and participants were not blinded to treatment allocation). This procedure could affect patients’ perceptions of HRQOL. For these ten open-label RCTs, the blinding of outcome assessment and blinding of participants and personnel were assessed as having a ‘high’ risk of bias. The five other RCTs were double-blind trials and were assessed as having a ‘low’ risk of bias for these two items. Four RCTs were assessed as ‘low’ risk of bias and ten as having an ‘unclear’ risk of bias for attrition criteria. For twelve RCTs, pre-specified HRQOL outcomes were reported and were assessed as having a ‘low’ risk of bias for reporting bias. The key reason for the ‘unclear’ risk of bias assessment was the lack of method details.
+ = low risk of bias. − = high risk of bias. ? = unclear risk of bias. * = bias due to inadequate generation of a randomised sequence. † = bias due to inadequate concealment of allocations before assignment. § = bias due to knowledge of the allocated interventions by participants and personnel during study. ¶ = detection bias due to knowledge of the allocated interventions by outcome assessment. # = due to amount, nature or handling of incomplete outcome data. ¥ = bias due to selective outcome reporting.
Discussion
Introduced in the past ten years, immunotherapy is starting to revolutionize the treatment of cancer. ICIs are increasingly used to treat a variety of cancers, but little is known about HRQOL. Hall et al. (2019) [10] performed a systematic review to examine HRQOL among cancer patients receiving ICIs as compared to other anticancer therapies: they did not analyse investigate the HRQOL assessment methods used in RCTs on cancer patients treated with the ICIs and it is the purpose of our work.
Fifteen RCTs meeting the inclusion criteria were identified in the literature. None of the identified RCTs studied Durvalumab or Avelumab HRQOL, probably as a result of delay in publishing data. Our analysis showed that all included RCTs were international, multicenter and supported by commercial sponsors. The primary endpoint of all of the RCTs was efficacy and safety, using progression-free survival or overall survival. HRQOL was identified as a secondary, or even exploratory, endpoint. Overall, a recent systematic review [64] about quality of patient-reported outcome (PRO) reporting across 557 cancer RCTs showed that PRO were secondary endpoints in 421 RCTs (76%). About HRQOL endpoint, RCTs on cancer patients treated with ICIs are in line with RCTs across cancer patients. For most of the RCTs, the HRQOL data were published in a companion paper in a peer-reviewed journal. Most of the companion papers were published within two years after publication of the main clinical results. The number of additional HRQOL papers is significant and indicative of interest not only for the efficacy of new drugs, but also for the patients’ experience.
Previous studies [11,12] showed that patients receiving ICIs experience similar-to-improve HRQOL compared to patients receiving other treatments for advanced cancer (such as chemotherapy, placebo, everolimus or gp100). Based on our findings, these results must be interpreted with caution in view of a number of potential weaknesses associated with the RCTs included.
First, as there are no international guidelines concerning the statistical analysis of HRQOL data in cancer RCTs [65], a clear ‘a priori hypothesis’ is needed to develop an analysis plan [20]. In Efficace et al. (2015) systematic review [64], about PRO reporting across 557 cancer RCTs, the three items most frequently omitted from reports were (1) the statement of a PRO hypothesis, (2) the description of statistical approaches for dealing with missing PRO data, and (3) the documentation of methods for PRO data collection, which were reported in only 17%, 20%, and 24% of studies, respectively. In line with the results of Efficace et al. (2015), the ‘stated a priori hypothesis’ is the most important HRQOL assessment method that needs to be implemented in RCTs about patients treated with ICIs. Indeed, none of the trials included an a priori hypothesis for expected HRQOL outcomes. As already mentioned [10,20,22,25], defining and reporting a hypothesis is an essential requirement for an appropriate study design. The choice of HRQOL questionnaire and statistical analysis in RCTs is dependent on the objective. A clear objective and specific hypotheses improve the analysis and credibility of the results [10]. The lack of hypotheses in many RCTs might partly be due to the exploratory nature of the analysis of HRQOL. Failure to pre-specify hypotheses might generate spurious HRQOL results, potentially emphasising findings that are statistically, but not clinically, relevant [25]. Consequently, we should be careful to avoid interpreting the HRQOL results of previous studies [11,12] as if based on a confirmatory analysis.
Second, as the choice of instrument affects reporting, analysis and interpretation of the trial findings deserve adequate justification [20]. In theory, both generic and disease-specific questionnaires may be used for a given condition. In practice, it is very important to choose the most suitable questionnaire to explore the domains relevant to the treatment(s) [66]. In our systematic review, while the characteristics of the HRQOL instrument used were presented for all trials but one, none of them justified their choice of a particular HRQOL instrument. As the questionnaires used in the RCTs included are considered by researchers to be the standard for use in cancer RCTs, they are often reported without a justification for their particular selection. The lack of a stated a priori hypothesis plays a part in the choice of the instrument as justification for the use of a specific method should depend mainly on the hypothesis being examined [20]. All but one of the questionnaires (EQ-5D) were validated for the specific cancer population. Nevertheless, none of these, or their clinically meaningful cut-off, were validated for cancer patients treated with immunotherapy. This is of the utmost importance as ICIs have a different toxicity profile to chemotherapy [20]. For example, skin toxicities (e.g. o, L er). QOL e prominent role as important endpoints in cancer RCTs.atuc review to i …ndart therapy arm (t not be covered.contsuchrash and pruritus) are the most frequent irAEs with anti-PD1 [8,13], but skin problems are not assessed with QLQ-H&N35. Consequently, HRQOL might be affected by ICIs in domains that are not covered by standard instruments, whereas side effects of chemotherapy (for control groups of RCTs) are covered by these standard instruments. Moreover, for the QLQ-C30, for example, although mean score changes of 10 points are widely viewed as clinically significant [67], minimally important differences as low as 4 points have been reported in other cancer trials [68,69]. When interpreting the trial results, it should be kept in mind that important HRQOL issues might not be covered [70].
Furthermore, quality of life may be defined as an individual’s self-perceived satisfaction at any stage of life [71]. Since the disease and cure are associated with symptoms and side effects, the patient learns to adapt to them [72]. This may change the patient’s internal standard of assessment. Such changes indicate the phenomenon of ‘response shift’ [73,74]. Thus, measurement of HRQOL changes may be biased [75–77]. None of the RCTs measured this bias, although different methods can be used to measure the phenomenon (e.g., Then-test method, Ideal Scale approach, Schedule for the Evaluation of Individual Quality of life).
Moreover, human behaviour is influenced by what we know or believe. The common sense model of self-regulation of health and illness (CMS) proposed by Leventhal (1980) [78] postulates that individual perceptions of health threats, and the ensuing emotional response, guide peoples' coping responses. CSM classifies illness representations into distinct dimensions: identity, cause, timeline, consequences, personal control, treatment control and illness coherence. In their systematic review and meta-analysis, Richardson et al. demonstrated that lower identity scores (referring to the description of the health threat and its symptoms) were associated with better quality of life [79]. In research, there is a risk of expectations influencing findings, most obviously when there is some subjectivity in the assessment, as in HRQOL, leading to biased results. To eliminate such bias, patients and assessors need to be blinded to treatment allocation [80]. In our systematic review, ten RCTs were open-label, and patients' perception of their quality of life could have been influenced by their knowledge of the treatment received. Most RCTs used centralised treatment allocation (interactive voice response system) and reported that the method adopted to stratify patients can help prevent potential treatment imbalances. However, instrument administration was rarely reported (in accordance with the results of Efficace et al. (2015) systematic review [64] about RCTs across cancer patients), making it hard to determine whether open-label trials could have influenced the responses to the questionnaires.
Finally, mains weaknesses associated with HRQOL in RCTs about patients treated with ICIs are the same as those described in cancer RCTs (the statement of a PRO hypothesis, the documentation of methods for PRO data collection, etc.). However, in immunotherapy RCTs, the “adequacy of domains covered” criterion is no satisfy because none of the questionnaire used in these RCTs covered, at least, the main HRQOL dimensions relevant according to the specific research question, namely irAEs. There is a need to develop standard methods for evaluating HRQOL for patients treated with ICIs. But HRQOL and irAEs are different according to cancer type and/or ICIs administered. We propose that HRQOL about patients treated with ICIs is assessed with the common cancer HRQOL assessment tools (like QLQ-C30, FACT-G), a specific module according to the type of cancer (as QLQ-LC13, QLQ-H&N35, LCSS) and a specific module for evaluate irAEs according to ICIs administered. At present, in the absence of HRQOL tool for patients treated with ICIs and for studying HRQOL more rigorously:
- we propose that HRQOL, in RCTs about patients treated with ICIs, was a primary endpoint not secondary or exploratory. Indeed, HRQOL endpoints are increasingly being used in cancer RCTs but often in secondary endpoint [64,81]. However, regardless of cancer type, quality of reporting was typically higher in RCTs where HRQOL were primary endpoints [81];
- As already mentioned, in our research, none of RCTs included an a priori hypotheses for the expected HRQOL outcomes whereas a clear hypothesis is an essential requirement for an appropriate study design, the choice of the instrument, etc. [20, 9, 22, 25]. So, in future RCTs about patients treated with ICIs a hypothesis must be included;
- In our research, HRQOL data was judged as high quality in ten RCTS. Because methodological rigor is essential to the conduct and reporting of RCTs [64], some recommendations have been established with the aim to facilitate adherence with key methodological aspects and to increase the transparency and consistency of PRO reporting in RCTs (for example, the Consolidated Standards of Reporting Trials (CONSORT) group published the CONSORT PRO extension in 2013; the Minimum Standard Checklist for evaluating HRQOL Outcomes in Cancer Clinical Trials by Efficace et al. [25]). In cancer RCTs, the overall level of reporting according to the new CONSORT PRO extension was poor. However, adherence to the CONSORT PRO extension was higher in RCTs in which PROs were included as primary endpoints versus RCTs with PROs as secondary outcomes [65]. All cancer RCTs, including RCTs with patients treated with ICIs, must implement of the CONSRT PRO extension for example;
- Patient-generated outcome measures have been developed in an effort to capture the individualistic nature of HRQOL. These measures differ from traditional HRQOL instruments in that they allow patients to individually define HRQOL domains or weights [82]. Different tools are available: Schedule for the Evaluation of Individual Quality of Life (SEIQoL), Repertory Grid, and Asthma Quality of Life Questionnaire (AQLQ) or Patient-Generated Index (PGI). For Patel et al., (2003), patient-generated outcome measures appear to be useful primarily in complementing traditional HRQOL measures, guiding individual patient treatment decisions, and assisting the design of new measures [82]. For example, the PGI is designed to both ask and document HRQOL concerns [83]. The PGI consists of three stages in which patients: i) self-identify the most important areas or activities of their lives affected by their condition; ii) score the degree to which each area is affected; and iii) allocate points among the items listed to represent the amount in which they would like each area improved [84]. A recent study provides evidence that the PGI would be a good measure for cancer patients and clinicians to use together [83]. Tang et al. (2014) systematic review aims to identify current literature in which PGI has been used as a tool to assess quality of life in cancer patients [84]. An overarching theme observed in these studies highlighted the concerns mentioned by patients that were not targeted or detected by standardized quality of life measures [84]. In HRQOL RCTs about patients treated with ICIs, patient-generated outcomes measures could be used to identify areas of cancer which are not evaluated by common cancer tools and that require attention and monitor changing needs
Study limitations
Our systematic review has several limitations. We focused only on RCTs published in French and English, but as the most important RCTs tend to be published in English journals, we believe that we have included the most significant trials. Another limitation is that we based our work on the information that the clinical community could access and only with the literature that reported publishing HRQOL. Unpublished RCTs and RCTs including HRQOL endpoint by design, but that had not published their results, protocols, or statistical analysis plans, were excluded. Our review was therefore subject to publication bias. Furthermore, though we used subjective judgments for several assessment criteria, each criterion was independently assessed by two reviewers under the supervision of a third researcher. We therefore believe that our results are reproducible and consistent. Lastly, since checkpoint inhibition is associated with a unique spectrum of side effects that are not assessed with the HRQOL measure used in the RCTs identified, the HRQOL domains covered by these questionnaires were not adequate for all the trials. As such, it is difficult to explore the value added by these measures to clinical decision-making in a trial setting.
Conclusion
To the best of our knowledge, we conducted one of the first systematic reviews aimed at investigating the HRQOL assessment methods used in RCTs for cancer patients treated with ICIs. HRQOL quantifies how a patient feels or functions, provides an economic basis for decision-making and contributes to clinical decision-making. As such, HRQOL is playing a more prominent role as an important endpoint in cancer clinical trials. The quality of the measurements and interpretation of results is crucial. Previous findings suggest that ICIs maintain HRQOL compared with standard treatments. However, these results must be interpreted with caution in view of a number of potential weaknesses associated with the RCTs included (such as open-label trials, under-reporting of HRQOL research hypotheses, instruments not validated for patients treated with immunotherapy) and insufficient guidance, as well as the lack of internationally recognised standard methods for analysing and reporting HRQOL. Hence, there is a strong need to develop standard methods for evaluating HRQOL for patients treated with ICIs. A better methodology would lead to a more efficient understanding of HRQOL outcomes.
Supporting information
S1 Table. Minimum Standard Checklist for evaluating HRQOL Outcomes in Cancer Clinical Trials.
*If a study explicitly states an exploratory HRQOL evaluation. ¶ If the HRQOL measure is validated in the same population as the one of the trial.
https://doi.org/10.1371/journal.pone.0227344.s003
(DOCX)
S2 Table. Assessment methods for HRQOL.
HRQOL = heath-related quality of life. EORTC QLQ-C30 = European Organisation for Research and Treatment of Cancer Core (EORTC) Quality of Life Questionnaire. EQ-5D = European Quality of Life 5 Dimension. EORTC QLQ-LC13 = EORTC Quality of Life 13-item Lung Cancer-specific Questionnaire. EORTC QLQ-H&N35 = EORTC module for Head & Neck cancer. LCSS = Lung Cancer Symptom Scale. FACT-G = Functional Assessment of Cancer Therapy-General. N/A = Not Available.
https://doi.org/10.1371/journal.pone.0227344.s004
(DOCX)
Acknowledgments
We would like to acknowledge Norbert Ifrah, President of the French National Cancer Institute and Christine Chomienne (CC) director of the research of the French National Cancer Institute, for their advices and for reviewing the draft.
References
- 1. Gálffy G, Puskás R. Role of pembrolizumab in the treatment of non-small cell lung cancer (NSCLC). Magy Onkol. 2017;61(1): 107–110. pmid:28273195
- 2. Emens LA, Ascierto PA, Darcy PK, Demaria S, Eggermont AMM, Redmond WL, et al. Cancer immunotherapy: opportunities and challenges in the rapidly evolving clinical landscape. Eur J Cancer 2017;81: 116–129. pmid:28623775
- 3. Morrissey KM, Yuraszeck TM, Li CC, Zhang Y, Kasichayanula S. Immunotherapy and novel combinations in Oncology: current Landscape; challenges and opportunities. Clin Transl Sci. 2016;9(2): 89–104. pmid:26924066
- 4. Dine J, Gordon R, Shames Y, Kasler MK, Barton-Burke M. Immune Checkpoint Inhibitors: An innovation on Immunotherapy for the Treatment and Management of Patients with Cancer. Asia Pac J Oncol Nurs. 2017;4(2): 127–135. pmid:28503645
- 5. Topalian SL, Hodi FS, Brahmer JR, Gettinger SN, Smith DC, McDermott DF, et al. Safety, activity, and immune correlates of anti-PD-1 antibody in cancer. N Engl J Med. 2012;366(26): 2443–2454. pmid:22658127
- 6.
Administration, U.S.F.a.D. Drugs@FDA: FDA approved drug products. 2018 february 28th; Available from: https://www.accessdata.fda.gov/scripts/cder/daf/.
- 7. Puzanov I, Diab A, Abdallah K, Bingham CO, Brogdon C, Dadu R, et al. Managing toxicities associated with immune checkpoint inhibitors: consensus recommendations from the Society for Immunotherapy of Cancer (SITC) Toxicity Management Working Group. J Immunither Cancer 2017;5(1): 95. pmid:29162153
- 8. Voudouri D, Nikolalaou V, Laschos K, Charpidou A, Soupos N, Triantafyllopoulou , et al. Anti-PD1/PDL1 induced psoriasis. Curr Probl Cancer 2017;41(6): 407–412. pmid:29096940
- 9. Brahmer J, Lacchetti C, Schneider BJ, Atkins MB, Brassil KJ, Caterino JM, et al. Management of Immune-Related Adverse Events in patients treated with immunce checkpoint inhibitor therapy: American Society of Clinical Oncology Clinical Practice Guideline. J Clin Oncol. 2018;36(17): 1714–1768. pmid:29442540
- 10. Ghislain I, Zikos E, Coens C, Quinten C, Balta V, Tryfonidis K, et al. Health-related quality of life in locally advanced and metastatic breast cancer: methodological and clinical issues in randomised controlled trials. Lancet Oncol. 2016;17(7): 294–304.
- 11. Abdel-Rahman O, Oweira H, Giryes A. Health-related quality of life in cancer patients treated with PD-(L)1 inhibitors: A systematic review. Expert Rev Anticancer 2018;18(12): 1231–1239.
- 12. Hall ET, Singhal S, Dickerson J, Gabster B, Wong HN, Aslakson RA et al. Patient-reported outcomes for cancer patients receiving checkpoint inhibitors: Opportunities for palliative care- A systematic review. JPSM 2019. https://doi.org/10.1016/j.jpainsymman.2019.03.015
- 13. Ferris RL, Blumenschein G, Fayette J, Guigay J, Colevas AD, Licitra L, et al. Nivolumab for recurrent squamous-Cell carcinoma of the head and neck. N Engl J Med. 2016;375: 1856–1867. pmid:27718784
- 14. Downing A, Morris EJ, Richards M, Corner J, Wright P, Sebag-Montefiore D, et al. Health- related quality of life after colorectal cancer in England: A patient-reported outcomes study of individuals 12 to 36 months after diagnosis. J Clin Oncol. 2015;33(6): 616–624. pmid:25559806
- 15. Ramsey SD, Andersen MR, Etzioni R, Moinpour C, Peacock S, Potosky A, et al. Quality of life in survivors of colorectal carcinoma. Cancer 2000;88(6): 1294–1303. pmid:10717609
- 16. Kopp I, Bauhofer A, Koller M. Understanding quality of life in patients with colorectal cancer: Comparison of data from a randomised controlled trial, a population based cohort study and the norm reference population. Inflamm Res. 2004;53 (Suppl 2): 130–135.
- 17. Bottomley A, Flechtner H, Efficace F, Vanvoorden V, Coens C, Therasse P, et al. Health related quality of life outcomes in cancer clinical trials. Eur J Cancer 2005;41(12): 1697–1709. pmid:16043345
- 18. Au HJ, Ringash J, Brundage M, Palmer M, Richardson H, Meyer RM, et al. Added value of health-related quality of life measurement in cancer clinical trials: the experience of the NCIC CTG. Expert Rev Pharmacoecon Outcomes Res. 2010;10(2): 119–128. pmid:20384559
- 19. Fielding S, Ogbuagu A, Sivasubramaniam S, MacLennan G, Ramsay CR. Reporting and dealing with missing quality of life data in RCTs: has the picture changed in the last decade? Qual Life Res.2016;25(12): 2977–2983. pmid:27650288
- 20. Zikos E, Ghislain I, Coens C, Ediebah DE, Sloan E, Quinten C, et al. Health-related quality of life in small-cell lung cancer: a systematic review on reporting of methods and clinical issues in randomised controlled trials. Lancet Oncol. 2014;15(2): 78–89.
- 21. Jacobsen PB, Davis K, Cella D. Assessing quality of life in research and clinical practice. Oncology 2002;16 (9 Suppl 10): 133–139.
- 22. Bottomley A, Therasse P. Quality of life in patients undergoing systemic therapy for advanced breast cancer. Lancet Oncol. 2002;3(10): 620–628. pmid:12372724
- 23. Sanders C, Egger M, Donovan J, Tallon D, Frankel S. Reporting on quality of life in randomised controlled trials: Bibliographic study. BMJ 1998;317(7167):1191–1194. pmid:9794853
- 24. Lee CW, Chi KN. The standard of reporting of health-related quality of life in clinical cancer trials. J Clin Epidemiol. 2000;53(5):451–458. pmid:10812316
- 25. Efficace F, Bottomley A, Osaba D, Gotay C, Flechtner H, D’haese S, et al. Beyond the development of Health-Related Quality-of-life (HRQOL) measures: A checklist for Evaluating HRQOL Outcomes in Cancer Clinical Trials-Does HRQOL Evaluation in Prostate Cancer Research Inform Clinical Decision Making? J Clin Oncol. 2003; 21(18): 3502–3511. pmid:12972527
- 26. Moher D, Jones A, Lepage L, CONSORT Group. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. Jama 2001;285:1992–1995. pmid:11308436
- 27.
http://www.crd.york.ac.uk/PROSPERO/
- 28. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350. pmid:25555855
- 29. Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. pmid:25554246
- 30.
Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.1.0. Cochrane Collab, 2011. Julian PT Higgins and Sally Green.
- 31. Staquet M, Berzon R, Osoba D, Machin D. Guidelines for reporting results of quality of life assessments in clinical trials. Qual Life Res. 1996;5(5): 496–502. pmid:8973129
- 32. Chassany O, Sagnier P, Marquis P, Fullerton S, Aaronson N, ERIQA. Patient-reported outcomes: The example of health related quality of life—A European guidance document for the improved integration of health related quality of life assessment in the drug regulatory process. Drug Inf J. 2002;36(1): 209–238.
- 33. de Haes J, Curran D, Young T, Bottomley A, Flechtner H, Aaronson N, et al. Quality of life evaluation in oncological clinical trials: The EORTC model—The EORTC Quality of Life Study Group. Eur J Cancer 2000;36(7): 821–825. pmid:10785585
- 34. Sprangers MA, Moinpour CM, Moynihan TJ, Patrick DL, Revicki DA, Clinical Significance Consensus Meeting Group. Assessing meaningful change in quality of life over time: A users’ guide for clinicians. Mayo Clin Proc. 2002;77(6): 561–571. pmid:12059127
- 35. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343. pmid:22008217
- 36. Rittmeyer A, Barlesi F, Waterkamp D, Park K, Ciardiello F, von Pawel J, et al. Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, pen-label, multicentre randomised controlled trial. Lancet 2017;389(10066): 255–65. pmid:27979383
- 37. Reck M, Rodríguez-Abreu D, Robinson AG, Hui R, Csöszi T, Fülöp A et al. Pembrolizumab versus chemotherapy for PD-L1-Positive Non-Small-Cell lung cancer. N Engl J Med. 2016;375: 1823–1833. pmid:27718847
- 38. Motzer RJ, Escudier B, McDermott DF, George S, Hammers HJ, Srinivas S, et al. Nivolumab versus everolimus in advanced renal-cell carcinoma. N Engl J Med. 2015;373: 1803–1813. pmid:26406148
- 39. Motzer RJ, Tannir NM, McDermott DF, Arén Frontera O, Melichar B, Choueiri TK et al. Nivolumab plus ipilimumab versus sunitinib in advanced renal-cell carcinoma. N Engl J Med. 2018;378(14): 1277–1290. pmid:29562145
- 40. Eggermont A, Chiarion-Sileni V, Grob Jean-Jacques, Dummer R, Wolchok JD, Schmidt H et al. Adjuvant ipilimumab versus placebo after complete resection of high-risk stage III melanoma (EORTC 18071): a randomised, double-blind, phase 3 trial. Lancer Oncol. 2015;16: 522–530.
- 41. Weber JS , D’Angelo SP, Minor D, Hodi FS, Gutzmer R, Neyns B et al. Nivolumab versus chemotherapy in patients with advanced melanoma who progressed after anti-CTLA-4 treatment (CheckMate 037): a randomised, controlled, open-label, phase 3 trial. Lancet Oncol. 2015;16: 375–384. pmid:25795410
- 42. Robert C, Long G, Brady B, Dutriaux C, Maio M, Mortier L et al. Nivolumab in previously untreated melanoma without BRAF mutation. N Engl J Med. 2015;372: 320–330. pmid:25399552
- 43. Robert C, Schachter J, Long G, Arance A, Grob JJ, Mortier L et al. Pembrolizumab versus Ipilimumab in advanced melanoma. N Engl J Med.2015;372: 2521–2532. pmid:25891173
- 44. Brahmer J, Reckamp K, Baas P, Crinò L, Eberhardt EE, Poddubskaya E, et al. Nivolumab versus docetaxel in advanced squamous-cell non-small-cell lung cancer. N Engl J Med.2015;373: 123–135. pmid:26028407
- 45. Hodi FS, O’Day S, McDermott DF, Weber RW, Sosman JA, Haanen JB et al. Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. 2010;363(8): 711–723. pmid:20525992
- 46. Ribas A, Puzonov I, Dummer R, Schadendorf D, Hamid O, Robert C, et al. Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial. Lancet Oncol. 2015;16(8): 908–918. pmid:26115796
- 47. Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Cowey CL, Lao CD et al. Combined nivolumab and ipilimumab or monotherapy in untreated melanoma. N Engl J Med. 2015;373: 23–34. pmid:26027431
- 48. Bellmunt J, de Wit R, Vaughn DJ, Fradet Y, Lee JL, Fong L, et al. Pembrolizumab as second-line therapy for advanced urothelial carcinoma. N Engl J Med. 2017;376(11): 1015–1026. pmid:28212060
- 49. Bordoni R, Ciardiello F, von Pawel J, Cortinovis D, Karagiannis T, Ballinger M, et al. Patient-reported outcomes in OAK: A phase III study of atezolizumab versus docetaxel in advanced non-small-cell lung cancer. Clin Lung Cancer 2018;19(5): 441–449. pmid:30017645
- 50. Cella D, Grünwald V, Nathan P, Doan J, Dastani H, Taylor F, et al. Quality of life in patients with advanced renal cell carcinoma given nivolumab versus everolimus in CheckMate 025: a randomised, open-label, phase 3 trial. Lancet Oncol. 2016;17(7): 994–1003. pmid:27283863
- 51. Coens C, Suciu S, Chiarion-Sileni V, Grob JJ, Dummer R, Wolchok JD, et al. Health-related quality of life with adjuvant ipilimumab versus placebo after complete resection of high-risk stage III melanoma (EORTC 18071): secondary outcomes of a multinational, randomised, double-blind, phase 3 trial. Lancet Oncol. 2017;18(3): 393–403. pmid:28162999
- 52. Larkin J, Minor D, D’Angelo S, Neyns B, Smylie M, Miller WH, et al. Overall survival in patients with advanced melanoma who received nivolumab versus investigator’s choice chemotherapy in CheckMate 037: a randomized, controlled, open-label phase III Trial. J Clin Oncol. 2018;36(4): 383–393. pmid:28671856
- 53. Long GV, Atkinson V, Ascierto PA, Robert C, Hassel JC, Rutkowski P, et al. Effect of nivolumab on health-related quality of life in patients with treatment-naïve advanced melanoma: results from the phase III CheckMate 066 study. Ann Oncol. 2016; 27(10): 1940¬¬—1946. pmid:27405322
- 54. Petrella T.M, Robert C, Richtig E, Miller WH, Masucci GV, Walpole E, et al. Patient-reported outcomes in KEYNOTE-006, a randomised study of pembrolizumab versus ipilimumab in patients with advanced melanoma. Eur JCancer 2017; 86: 115–124.
- 55. Reck M, Taylor F, Penrod J, DeRosa M, Morrissey L, Dastani H, et al. Impact of Nivolumab Versus Docetaxel on Health-Related Quality of Life and Symptoms in Patients With Advanced Squamous Non-Small Cell Lung Cancer: Results From the CheckMate 017 Study. J Thorac Oncol. 2018;13(2): 194–204. pmid:29129758
- 56. Schadendorf D, Larkin J, Wolchok J, Hodi FS, Chiarion-Sileni V, Gonzalez R, et al. Health-related quality of life results from the phase III CheckMate 067 study. Eur J cancer 2017;82: 80–91. pmid:28651159
- 57. Weber J, Mandala M, Del Vecchio M, Gogas HJ, Arance AM, Cowey CL, et al. Adjuvant Nivolumab versus ipilimumab in resected stage III or IV melanoma. N Engl J Med. 2017; 377: 1824–1835. pmid:28891423
- 58. Brahmer JR, Rodriguez-Abreu D, Robinson AG, Hui R, Csöszi T, Fülöp A, et al. Health-related quality-of-life results for pembrolizumab versus chemotherapy in advanced, PD-L1-positive NSCLC (KEYNOTE-024): a multicentre, international, randomised, open-label phase 3 trial. Lancet Oncol. 2017;18(12): 1600–1609. pmid:29129441
- 59. Cella D, Grünwald V, Escudier B, Hammers HJ, George S, Nathan P, et al. Patient-reported outcomes of patients with advanced renal cell carcinoma treated with nivolumab plus ipilimumab versus sunitinib (CheckMate 214): a randomised, phase 3 trial. Lancet Oncol. 2019; 20: 297–310. pmid:30658932
- 60. Harrington K, Ferris R, Blumenschein G, Colevas AD, Fayette J, Licitra L, et al. Nivolumab versus standard, single-agent therapy of investigator’s choice in recurrent or metastatic squamous cell carcinoma of the head and neck (CheckMate 141): health-related quality-of-life results from a randomised phase 3 trial. Lancet Oncol. 2017;18(8): 1104–1115. pmid:28651929
- 61. Schadendorf D, Dummer R, Hauschild A, Robert C, Hamid O, Daud A, et al. Health-related quality of life in the randomised KEYNOTE-002 study of pembrolizumab versus chemotherapy in patients with ipilimumab-refractory melanoma. Eur J cancer 2016;67: 46–54. pmid:27596353
- 62. Vaughn DJ, Bellmunt J, Fradet Y, Lee JL, Fong L, Vogelzang NJ, et al. Health-related quality of life analysis from keynote-45: A phase III study of pembrolizumab versus chemotherapy for previously treated advanced urothelial cancer. J Clin Oncol 2018; 16(1): 1579–1587.
- 63. Revicki DA, van den Eertwegh AJM, Lorigan P, Lebbe C, Linette G, Ottensmeier CH, et al. Health related quality of life outcomes for unresectable stage III or IV melanoma patients receiving ipilimumab treatment. Health Qual Life outcomes 2012;10: 66. pmid:22694829
- 64. Efficace F, Fayers P, Pusic A, Cemal Y, Yanagawa J, Jacobs M, et al. Quality of patient-reported outcome reporting across cancer randomized controlled trials according to the CONSORT patient-reported outcome extension: A pooled analysis of 557 trials. Cancer. 2015;121(18):3335–3342. pmid:26079197
- 65. Hamel JF, Saulnier P, Pe M, Zikos E, Musoro J, Coens C, et al. A systematic review of the quality of statistical methods employed for analysing quality of life data in cancer randomised controlled trials. Eur J Cancer 2017;83: 166–176. pmid:28738257
- 66.
European Medicines Agency. Committee for Medicinal Products for Human Use (CHMP). Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal productshttps://www.ispor.org/workpaper/emea-hrql-guidance.pdf (accessed mars 8, 2018).
- 67. Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol. 1998; 16: 139–144. pmid:9440735
- 68. Maringwa JT, Quinten C, King M, Ringash J, Osoba D, Coens C, et al. Minimal important differences for interpreting health related quality of life scores from the EORTC QLQC30 in lung cancer patients participating in randomized controlled trials. Support Care Cancer 2011;19(11): 1753–1760. pmid:20886240
- 69. Schadendorf D, Amonkar MM, Milhem M, Grotzinger K, Demidov LV, Rutkowski P, et al. Functional and symptom impact of trametinib versus chemotherapy in BRAF V600E advanced or metastatic melanoma: Quality-of-life analyses of the METRIC study. Ann Oncol. 2014;25: 700–706. pmid:24504441
- 70. Singer S. Quality of life after Nivolumab treatment for head and neck cancer. Lancet Oncol. 2017;18(8): 993–994. pmid:28651926
- 71. Campbell A, Converse PE, Rodgers WL. The quality of American life: Perceptions, evaluations, and satisfactions: RSF. 1976.
- 72. Hosseini B, Nedjat S, Zendehdel K, Majdzadeh R, Nourmohammadi A, Montazer A. Response shift in quality of life assessment among cancer patients: A study from Iran. Med J Islam Repub Iran 2017;31:120. pmid:29951421
- 73. Sprangers M, Schwartz C. The challenge of response shift for quality- of-life-based clinicaloncology research. Ann Oncol. 1999;10(7): 747–749. pmid:10470418
- 74. Sprangers MA, Schwartz CE. Integrating response shift into health- related quality of life research: a theoretical model. Soc. Sci. Med. 1999;48(11): 1507–1515. pmid:10400253
- 75. Breetvelt I, Van Dam F. Underreporting by cancer patients: the case of response-shift. Soc. Sci. Med. 1991;32(9): 981–987. pmid:2047902
- 76. Andrykowski MA, Hunt JW. Positive psychosocial adjustment in potential bone marrow transplant recipients: Cancer as a psychosocial transition. Psychooncology. 1993;2(4): 261–276.
- 77. Kagawa-Singer M. Redefining health: living with cancer. Soc. Sci. Med. 1993;37(3): 295–304. pmid:8356479
- 78.
Leventhal H, Meyer D, Nerenz D. The common sense representation of illness danger. In Rachman S, ed. Contributions to Medical Psychology. New York: pergamon press 1980; 7–30.
- 79. Richardson E, Schüz N, Sanderson K, Scott JL, Schüz B. Illness representations, coping and illness outcomes in people with cancer: a systematic review and meta-analysis. Psychoncology 2016;26(6): 724–737.
- 80. Day SJ, Altman DG. Blinding in clinical trials and other studies. BMJ 2000;321: 504. pmid:10948038
- 81. Sparano F, Aaronson NK, Sprangers MAG, Fayers P, Pusic A, Kieffer JM et al. Does the quality of patient-reported outcomes (PROs) assessment in randomized controlled trials (RCTs) differ across cancer types and over time? A pooled analysis of 610 RCTs published between 2004 and 2018. J. Clin. Oncol. 2019;37(15).
- 82. Patel KK, Veenstra DL, Patrick DL. A review of selected patient-generated outcome measures and their application in clinical trials. Value Health. 2003; 6(5): 595–603. pmid:14627066
- 83. Aburub AS, Gagnon B, Rodriguez AM, Mayo NE. Using a personalized measure (Patient Generated Index (PGI)) to identify what matters to people with cancer. Support Care Cancer. 2016;24(1):437–445. pmid:26099901
- 84. Tang JA, Taemin OH, Scheer JK, Parsa AT. The Current Trend of Administering a Patient-Generated Index in the Oncological Setting: A Systematic Review. Oncol Rev. 2014;8(1): 245. pmid:25992233