http://orcid.org/0000-0003-4434-748X
Figures
Abstract
Regulatory agencies around the world have been using flexible requirements for approval of new drugs, especially for cancer drugs. The US Food and Drug Administration (FDA) is mostly the first agency to approve new drugs worldwide, mainly due to the faster terms of the accelerated pathway and breakthrough therapy designation. Surrogate endpoints and preliminary data (e.g. single-arm and phase 2 studies) are used for these new approvals, however larger effect sizes are expected. We aim to compare FDA Accelerated vs Regular Pathway approvals and Breakthrough therapy designations (BTD) for lung cancer treatments between 2006 and 2018 regarding study design, sample size, outcome measures and effect size. We assessed the FDA database to collect data from studies that formed the basis of approvals of new drugs or indications for lung cancer spanning from 2006 to 2018. We found that accelerated pathway approvals are based on significantly more single-arm studies with small sample sizes and surrogate primary endpoints. However, effect size was not different between the pathways. A large proportion of studies used to support regular pathway approvals also showed these characteristics that are related to low quality and uncertain evidence. Compared to other approvals, BTD were more frequently based on single-arm studies. There was no significant difference in use of surrogate endpoints or sample size. 44% of BTD were based on studies demonstrating large effect sizes, proportionally more than approvals not receiving this designation. In conclusion, based on the indicators of evidence quality we extracted, criteria’s for granting accelerated approval and breakthrough therapy designation seen not clear. Faster approvals are in the majority full of uncertainties which should be viewed with caution and the patient have to be communicated to allow shared decision making. Post-marketing validation is essential.
Citation: Ribeiro TB, Buss L, Wayant C, Nobre MRC (2020) Comparison of FDA accelerated vs regular pathway approvals for lung cancer treatments between 2006 and 2018. PLoS ONE 15(7): e0236345. https://doi.org/10.1371/journal.pone.0236345
Editor: Erika Cecchin, Centro di Riferimento Oncologico, ITALY
Received: January 9, 2020; Accepted: July 3, 2020; Published: July 24, 2020
Copyright: © 2020 Ribeiro et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: CW is supported by the National Cancer Institute of the National Institutes of Health under award number F30CA243651-01. No additional external funding was received for this study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
For a new drug to enter the market, a local health agency must assess the efficacy and safety based on studies submitted by pharmaceutical companies. The North American agency, the US Food and Drug Administration (FDA), is the in the great majority the first agency in the world to approve new drugs [1].
Cancer is the leading cause of death in high-income countries [2]. Lung cancer incidence is increasing worldwide, with over two million new cases in 2018 [3]. The treatment arsenal for lung cancer is extensive, with an improved understanding of the genetic drivers (e.g. EGFR, ALK, BRAF) leading to a rapid growth in novel medications. Targeted and immune therapies are first-line treatments and widely recommended in current guidelines [4–6].
Considering recent FDA approvals, protein kinase inhibitors for advanced lung cancer accounted for 53% of new therapies for the four most common neoplasms and 78% were granted a special designation for faster approval [7]. Other studies have shown that these innovative trials of new FDA-approved anti-cancer drugs focus heavily on surrogate markers of clinical benefit, and trials that measure overall survival demonstrate only modest improvements [8].
In 1992, during the HIV crisis, the FDA created the "Expedited Programs for Serious Conditions". The aim was to provide faster access to medications that addressed ‘unmet medical need’ though an accelerated approval pathway and priority review designation in which studies using surrogate endpoints were considered. Subsequently other designations have been created to increase the flexibility of the approvals process. Under the “Breakthrough” designation, implemented in 2012, approvals can be based on preliminary clinical evidence–for example phase 2 or single-arm studies–that “indicates the drug may demonstrate substantial improvement over available therapy on a clinically significant endpoint(s)” [9]. While double blind randomized controlled trials (RCT) are the gold standard for intervention studies [10], these expedited programs result in more drug approvals based on single arm trials, with small sample sizes and surrogate endpoints [11–12].
Accelerated pathway differently from regular pathway allows anticancer drugs to be approved based on a surrogate endpoint, however it is unclear whether there is a difference in 1) proportion of surrogate endpoints, 2) sample size, 3) study design and, 4) effect size. Our two main comparisons were trials that received accelerated approval vs. regular approval and trials that received breakthrough designation vs. those that did not.
Methods
This study used publicly available data and did not involve individual patient information. Ethics approval was not required. This report follows the STROBE statement for observational data reporting.
Data collection
Two authors (TR and LB) independently reviewed the FDA’s online bulletin (@DrugsFDA) that includes all approvals for lung cancer. The FDA database contains approval data on new drugs and indications since 2006. We retrieved information on approvals and trials supporting these approvals from January 1, 2006 to December 31, 2018. We excluded the following approval types: agnostic therapies, treatments for neuroendocrine tumors, changes to the dosing regimen of previously approved drugs or further specification of mutations (e.g. L858R), and finally we excluded conversion from accelerated to regular in order to due to duplicate the approval for the specific condition. First approvals of new lung cancer (NSCLC and SCLC) drugs as well as novel indications were included. One approval was not included in the FDA material but identified after cross-referencing. The following data were collected about studies forming the basis of approvals: sample size, study type (open-label RCT, double-blind RCT or single-arm), the intervention and control, whether overall survival was measured as an endpoint and the primary endpoint used to justify the FDA approval (overall survival—OS, progressive-free survival—PFS, overall response rate—OvRR or objective response rate—ORR), and the measure of effect. Generally, according to RECIST, OvRR is related to partial or complete response. ORR is defined as the sum of complete and partial responses [13]. However, the definition varies among studies and the authors often did not specify which definition they used.
We recorded the approval pathway (regular vs accelerated) and special designations (breakthrough or priority review). These data were subsequently confirmed on the accelerated approval (https://www.fda.gov/drugs/nda-and-bla-approvals/accelerated-approvals) and breakthrough therapy designation (BTD) databases (https://www.fda.gov/drugs/nda-and-bla-approvals/breakthrough-therapy-approvals). We checked if the drug was a new molecular entity (NME) approved for the first time in the USA by searching the generic names on the @Drugs FDA database.
Magnitude of effect
We assessed the magnitude of effect for the specified primary outcome. According to the GRADE classification [14] a relative risk (RR) greater than or equal to two was considered “large” and those less than two were considered “not large”; and a hazard ratio (HR) based in outcomes as survival and progressive-free survival, greater than or less than .5 was considered “large” and those greater than .5 were considered “not large”. For single-arm studies, which by definition included a lack a control group, we selected a historical control (“best control available”) to estimate the magnitude of effect (Fig 1). If the drug had a biomarker (EGFR, ALK, BRAF or ROS-1) and there was a previous FDA approved drug for the same indication and treatment line, then this drug was selected as the control. However, for drugs without a biomarker or a previous FDA approved comparator we used a historical control from the “Major Milestones against Cancer Timeline for Lung cancer” [15], available at the ASCO website. Additionally, when necessary we consulted the NCCN Guidelines for Non-small lung cancer [16] and Small Cell lung cancer [17].
Statistical analysis
The main variables were grouped into the following binary categories: RCT (blinded and open-label) vs single-arm; surrogate outcomes (OvRR, ORR and PFS) vs overall survival; sample sizes of≥200 vs <200 (following Ladanie et al., 2019 [18] recommendations); and magnitude of effect as large vs not large. Study characteristics were compared between accelerated and regular approvals and this analysis was repeated for BTD for approvals from 2012 onwards, comparing BTD vs non-BTD, the absolute number and percentage was presented. We plotted effect sizes from Accelerated Approval Pathway and BTD according to its effect size measurement; one graph for studies based in hazard ratio (time to event outcome: OS and PFS) and another graph to studies based in RR estimated from single-arms studies using response rate as the primary endpoint.
The study was primarily descriptive, and we included all data available for lung cancer treatment in the period (2006–2018). The absolute difference (AD) and prevalence ratio (PR) with a confidence interval of 95% based on a Normal approximation of the sampling distribution were calculated. A 2-sided P values .05 was used to assess statistical significance by the Fisher’s exact test. Analyses were performed in R (R Project for Statistical Computing) version 3.5.3.
Results
Regulatory characteristics
From 2006 to 2018, the FDA granted 33 approvals for new drugs and supplemental indications in lung cancer (Table 1). The approvals excluded and justification are available at Supporting Information (S1 Table). 33% (11 of 33) of drugs approved via the accelerated pathway and 42% (14 of 33) received BTD. Priority review designation was granted in 42% (14 of 33) of approvals, concomitantly with other designations in some cases. 30% (10 of 33) of approvals were NEMs.
Study characteristics.
Thirty-seven studies formed the basis of the 33 approvals (Table 2). Single-arms studies and open-label RCTs accounted for 43% (16 of 37), with blinded RCTs for an additional 14% (5 of 37). OS was reported in 46% (17 of 37) of the studies and was specified as the primary endpoint in 30% (11 of 37). PFS, ORR and OvRR were the primary endpoint in 27% (10 of 37), 30% (11 of 37) and 13% (5 of 37) of studies, respectively.
Elements related to the quality of evidence in lung cancer approvals
Accelerated vs regular pathway.
The single-arm trials and small sample sized were observed 4.06 and 2.37 more times in the Accelerated pathway compared to the Regular pathway, with an absolute difference of 64% and 40%, respectively (Table 3). However, the number of trials with larger effect size was not different among the pathways.
Accelerated pathway are 1.58 (CI95% 1.09–2.30) more likely to have studies with surrogate endpoint and an absolute different between the groups of 37% was observed, however p-value was in the limit of the significance, according to Fisher’s exact test, probably due to small sample size.
Graphs 1 and 2 shows that most studies approved using the Accelerated pathway were based in relative risk calculation (from single-arm studies) and only 5 studies showed a large effect size (Graph 2).
Studies using Overall Survival were represented with the triangle and Studies using Progressive-Free Survival were represented with the Circle. Color RED referred to accelerated approval pathway and the BLACK referred to regular pathway.
All Studies (circle) were single arms, response rate was the primary endpoint and we calculated the risk ratio estimated by using historical control. Color RED referred to accelerated approval pathway and the BLACK referred to regular pathway.
Breakthrough therapy designation.
Thirty-two approvals were made between 2012 and 2018, of which 16 were granted BTD (Table 3). Single-arms trials were 2.5 more frequent in the group which received the BTD, with an absolute difference of 37%, however this difference was not significant according to Fisher’s exact test.
Large effect sizes, surrogate endpoints and small sample sizes were not more common in BTD, with absolute difference of 31%, 19% and 0%, respectively.
Graph 3 and 4 showed that several studies since 2012 received BTD. The majority of the studies which received this designation were based on relative risk calculation (from single-arm studies) (Graph 4), as expected, larger effect sizes were noted in these studies. Details on the historical controls and RR calculations are presented in the Supporting Information—S2 Table.
Studies using Overall Survival were represented with the Triangle and Studies using Progressive-Free Survival were represented with the Circle. Color BLUE referred to Breakthrough therapy designation and the BLACK referred to Non- Breakthrough therapy designation.
All Studies (circle) were single-arms, response rate was the primary endpoint and we calculated the risk ratio estimated by using historical control. Color BLUE referred to Breakthrough therapy designation and the BLACK referred to Non- Breakthrough therapy designation.
Discussion
We assessed characteristics of the evidence base for FDA lung cancer approvals between 2006 and 2018. Accelerated pathway approvals were 4.06 times more based on single-arm studies, studies with small sample sizes and surrogate primary endpoints. These results are expected given the purpose of accelerated approval is to provide the public more rapid access to new therapies. However, there was no difference in effect sizes between accelerated and regular approvals, likely because regular approvals used surrogate endpoints more often than overall survival. Such a finding is concerning because overall survival is the main patient-centered endpoint. It has previously been shown that surrogate endpoints tend to be associated with larger effect sizes compared to overall survival [19]. A recent review addressed the correlation between surrogate endpoints and overall survival with respect to lung cancer, although some studies showed a strong correlation, mostly they were weakly correlated [20].
The accelerated pathway was created in 1992 and 25 years later has been described as a “successful program” for oncology drugs with only 5% of drugs withdrawn from market due to failure to demonstrate clinical benefit. The median time for benefit to be verified and conversion from accelerated to regular approval is 3.4 years [21]. 85% of the accelerated approvals in the last 25 years were based on response rate outcomes and 72% were based on single-arm studies [21]. Considering all FDA approvals between 2005 and 2012, 448 pivotal trials were considered, of which 89.3% were RCTs and 45.3% used surrogate endpoints [12]. Similarly, in the present study, 67% of lung cancer approvals were based on surrogate endpoints and only 57% were informed by RCT evidence.
In Evidence-based medicine, core outcome measures are essential for good evidence quality [14]. Core outcomes must be patient-centered and clinically relevant. Surrogate endpoints in cancer research have been described as measurable, but not clinically meaningful [22]. It is common that a decrease in tumor size or delayed tumor growth is not related to survival gain. While not all surrogate endpoints fail to predict patient-centered outcomes, with a notable example being disease-free survival in colon cancer [23], we question the unbridled use of surrogate endpoints to grant market approval. Much has been discussed regarding value-based healthcare and the importance of a patient-centered approach and add their important healthcare results for reimbursement and payment proposals [24].
RCTs remain the gold standard for intervention studies and non-controlled studies can only be justified if a dramatic effect is seen [25]. Breakthrough approval is intended for drugs that are expected to provide “substantial improvement” in a “clinically significant endpoint over available therapy” [9]. Single-arm studies are case series and they carry many uncertainties because the study is not controlled, with uncontrolled patient selection and reporting bias. With no control group, it is difficult to assess effect size properly and impossible to draw causal conclusions [26]. The FDA allows phase 1 or phase 2 trials for breakthrough designation. It is encouraging that 37.5% (6 of 16) of BTDs granted for lung cancer were RCTs. Nonetheless, predicting clinical benefit using single-arm trials that measure surrogate endpoints has proven challenging and we recommend revised BTD criteria that align more clearly with BTD’s mission.
Patient pressure for new drugs for unmet medical need and market pressure for innovation are used to justify approval of drugs with more uncertainties. However, we found only a small proportion of drugs showing evidence of a large effect size. In general, studies supporting cancer drugs approved by the FDA demonstrate low to modest response rates [27, 28].
A study published in September 2019 assessed the effect size of all drugs and devices that received breakthrough therapy designation since 2012 and were based on nonrandomized clinical trials [29]. The authors concluded that 26% of studies in which the FDA did not request further RCTs showed a risk ratio of five or greater. For drugs, it was noted that effect size was larger among studies without RCT requirements. It is notable that the FDA did not properly assess the control group data, which is crucial for effect size estimation in single-arm studies and there is no standardized threshold for effect size [29]. Despite this, there is no consensus regarding how large a truly dramatic effect size is. Some authors consider that a risk ratio equal to or larger than 5 to 10 can be safely attributed to the intervention, as it is unlikely to be solely explained by bias or confounding [30]. In our study, we took a conservative approach, considering effect sizes of two or greater as large, this was based on the GRADE recommendation. However, despite this more conservative approach, larger effect sizes were not observed in the accelerated pathway.
The argument for expedited approval is that although these studies have significant limitations, they demonstrate large effect sizes. This was not supported in our study. Similarly, elsewhere it has been shown that approvals granted BTD between from 2012 to 2017 were approved 1.9 years earlier than those did not grant this designation. However, there was no difference in the PFS or RRs for solid tumors. In our study, we found that only 44% of the drugs granted BTD had large effect sizes, and the majority demonstrated smaller effects.
Limitations
Our sample of approvals is small and included only lung cancer treatments. However, lung cancer approvals serve as a suitable case study due to the large quantity of approvals for novel treatments in oncology (targeted drugs and immunotherapies).
We used historical controls in the estimation of effect size in single-arm studies. Although we specified transparently our process of control selection, the choice of historical controls will always be somewhat subjective and could be questioned. Historic controls likely introduce inaccuracies because: (1) the population included in the groups is different, (2) the period when the intervention was tested is different, (3) the available health care technologies and facilities may be different [31]. Moreover, we selected the historical control based on the information available to us at the time of writing. However, in reality the FDA may not have had access to this same information because the approvals processes can be concomitant.
We did not assess the bias in the included studies. This can clearly influence the certainty of the evidence. A big challenge is using a tool to assess both RCTs and single-arm studies by the same metrics.
Conclusion
Our results show that effect size was not different between accelerated and regular pathway approvals, despite accelerated approvals being based on more single-arm studies, small sample sizes and surrogate primary endpoints, all of which are related to low evidence quality.
Surrogate endpoints were also more frequent in the accelerated pathway, as expected due to the pathway criteria, however many studies in the regular approval employed surrogate endpoints. Larger effect sizes were more frequent in relative risk estimated from studies based in response rate (single-arms).
More than half the BTD approvals were based on “not large” effect sizes, despite the definition for this designation being demonstration of a dramatic effect size.
Supporting information
S1 Table. Details of approvals exclusion and justification.
https://doi.org/10.1371/journal.pone.0236345.s001
(DOCX)
S2 Table. Details of the control selection and risk ratio calculation.
https://doi.org/10.1371/journal.pone.0236345.s002
(DOCX)
References
- 1. Van Norman GA. Drugs and Devices: Comparison of European and U.S. Approval Processes. JACC Basic to Transl Sci. 2016. pmid:30167527
- 2. Dagenais GR, Leong DP, Rangarajan S, et al. Variations in common diseases, hospital admissions, and deaths in middle-aged adults in 21 countries from five continents (PURE): a prospective cohort study. Lancet. 2019.
- 3. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018. pmid:30207593
- 4. Hanna N, Johnson D, Temin S, et al. Systemic therapy for stage IV non–small-cell lung cancer: American Society of clinical oncology clinical practice guideline update. J Clin Oncol. 2017. pmid:28806116
- 5. Kalemkerian GP, Narula N, Kennedy EB, et al. Molecular testing guideline for the selection of patients with lung cancer for treatment with targeted tyrosine kinase inhibitors: American society of clinical oncology endorsement of the college of American pathologists/ international association for the study of lung cancer/ association for molecular pathology clinical practice guideline update. J Clin Oncol. 2018. pmid:29401004
- 6. Eberhardt WEE, De Ruysscher D, Weder W, et al. 2nd ESMO Consensus Conference in Lung Cancer: Locally advanced stage III non-small-cell lung cancer. Ann Oncol. 2015;26(8):1573–1588. pmid:25897013
- 7. Ribeiro T, Ribeiro A, Rodrigues L, Harada G, Nobre M. U.S. Food and Drug Administration anticancer drug approval trends from 2016 to 2018 for lung, colorectal, breast, and prostate cancer. International Journal of Technology Assessment in Health Care. 2019:1–9.
- 8. Fojo T, Mailankody S, Lo A. Unintended consequences of expensive cancer therapeutics—The pursuit of marginal indications and a me-too mentality that stifles innovation and creativity: The John Conley lecture. JAMA Otolaryngol—Head Neck Surg. 2014;140(12):1225–1236. pmid:25068501
- 9.
FDA 5. Expedited Programs for Serious Conditions–Drugs and Biologics. Draft Guid Ind. 2014. U.S. Food & Drug Administration
- 10. Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011. pmid:21208779
- 11. Yao JC, Meric-Bernstam F, Lee JJ, Eckhardt SG. Accelerated approval and breakthrough therapy designation: Oncology drug development on speed? Clin Cancer Res. 2013. pmid:23833307
- 12. Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005–2012. JAMA—J Am Med Assoc. 2014. pmid:24449315
- 13. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur J Cancer. 2009;45(2):228–247. pmid:19097774
- 14. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011. pmid:21195583
- 15.
National Cancer Institute. Milestones in cancer research and discovery. National Cancer Institute.
- 16. Ettinger DS, Wood DE, Aisner DL, et al. Non-small cell lung cancer, version 5.2017: Clinical practice guidelines in oncology. JNCCN J Natl Compr Cancer Netw. 2017. pmid:28404761
- 17. Kalemkerian GP, Akerley W, Bogner P, et al. Small cell lung cancer: Clinical practice guidelines in oncology. JNCCN J Natl Compr Cancer Netw. 2013. pmid:23307984
- 18. Ladanie A, Speich B, Briel M, et al. Single pivotal trials with few corroborating characteristics were used for FDA approval of cancer therapies. J Clin Epidemiol. 2019. pmid:31158450
- 19. Wayant C, Vassar M. A comparison of matched interim analysis publications and final analysis publications in oncology clinical trials. Ann Oncol. 2018. pmid:30307531
- 20. Haslam A, Hey SP, Gill J, Prasad V. A systematic review of trial-level meta-analyses measuring the strength of association between surrogate end-points and overall survival in oncology. Eur J Cancer. 2019. pmid:30528804
- 21. Beaver JA, Howie LJ, Pelosof L, et al. A 25-year experience of us food and drug administration accelerated approval of malignant hematology and oncology drugs and biologics a review. JAMA Oncol. 2018. pmid:29494733
- 22. Booth CM, Eisenhauer EA. Progression-free survival: Meaningful or simply measurable? J Clin Oncol. 2012. pmid:22370321
- 23. Sargent DJ, Wieand HS, Haller DG, et al. Disease-free survival versus overall survival as a primary end point for adjuvant colon cancer studies: Individual patient data from 20,898 patients on 18 randomized trials. J Clin Oncol. 2005. pmid:16260700
- 24. Porter ME. What is value in health care? N Engl J Med. 2010. pmid:21142528
- 25. Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ. 2007. pmid:17303884
- 26. Kendall JM. Designing a research project: Randomised controlled trials and their principles. Emerg Med J. 2003. pmid:12642531
- 27. Chen EY, Raghunathan V, Prasad V. An Overview of Cancer Drugs Approved by the US Food and Drug Administration Based on the Surrogate End Point of Response Rate. JAMA Intern Med. 2019. pmid:31135822
- 28. Gyawali B, Hey SP, Kesselheim AS. Assessment of the Clinical Benefit of Cancer Drugs Receiving Accelerated Approval. JAMA Intern Med. 2019. pmid:31135808
- 29. Razavi M, Glasziou P, Klocksieben FA, Ioannidis JPA, Chalmers I, Djulbegovic B. US Food and Drug Administration Approvals of Drugs and Devices Based on Nonrandomized Clinical Trials: A Systematic Review and Meta-analysis. JAMA Netw open. 2019. pmid:31509209
- 30. Djulbegovic B, Glasziou P, Klocksieben FA, et al. Larger effect sizes in nonrandomized studies are associated with higher rates of EMA licensing approval. J Clin Epidemiol. 2018. pmid:29432860
- 31. Pocock SJ. The combination of randomized and historical controls in clinical trials. J Chronic Dis. 1976.