https://orcid.org/0000-0002-6436-3164
Figures
Abstract
Background
The COVID-19 pandemic remains a significant global threat. However, despite urgent need, there remains uncertainty surrounding best practices for pharmaceutical interventions to treat COVID-19. In particular, conflicting evidence has emerged surrounding the use of hydroxychloroquine and azithromycin, alone or in combination, for COVID-19. The COVID-19 Evidence Accelerator convened by the Reagan-Udall Foundation for the FDA, in collaboration with Friends of Cancer Research, assembled experts from the health systems research, regulatory science, data science, and epidemiology to participate in a large parallel analysis of different data sets to further explore the effectiveness of these treatments.
Methods
Electronic health record (EHR) and claims data were extracted from seven separate databases. Parallel analyses were undertaken on data extracted from each source. Each analysis examined time to mortality in hospitalized patients treated with hydroxychloroquine, azithromycin, and the two in combination as compared to patients not treated with either drug. Cox proportional hazards models were used, and propensity score methods were undertaken to adjust for confounding. Frequencies of adverse events in each treatment group were also examined.
Citation: Stewart M, Rodriguez-Watson C, Albayrak A, Asubonteng J, Belli A, Brown T, et al. (2021) COVID-19 Evidence Accelerator: A parallel analysis to describe the use of Hydroxychloroquine with or without Azithromycin among hospitalized COVID-19 patients. PLoS ONE 16(3): e0248128. https://doi.org/10.1371/journal.pone.0248128
Editor: Francesco Di Gennaro, National Institute for Infectious Diseases Lazzaro Spallanzani-IRCCS, ITALY
Received: December 10, 2020; Accepted: February 20, 2021; Published: March 17, 2021
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: Health catalyst provided support for this study in the form of salary for AA, EE, AG, and JJ. Gilead Sciences provided support for this study in the form of salary for JA and RM. COTA, Inc provided support for this study in the form of salary for AB, EH, and SG. Sypase provided support for this study in the form of salary for TB, MI, AS, FW and JH. Aetion, Inc. provided support for this study in the form of salary for NG, JR, and AW. TriNetX, LLC. provided support for this study in the form of salary for SK. Dascena provided support for this study in the form of salary for RD, CL, QM, SM, AS. HealthVerity provided support for this study in the form of salary for RK. The specific roles of these authors are articulated in the ‘author contributions’ section. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have read the journal’s policy and the authors of this manuscript have the following competing interests: AA is a paid employee and stockholder at Health Catalyst. JA is a paid employee and stockholder at Gilead Sciences. AB is a paid employee of COTA, Inc with ownership interest (equity). TB is a paid employee of Sypase. NG is a paid employee and shareholder of Aetion, Inc. SG has equity ownership with COTA, Inc. EH is a paid employee by COTA, Inc. with ownership interest (equity). JH is Founder and President of Syapse with pharmaceutical company funders including Roche, Amgen, Merck & Co. (Syapse employees engaged in design, collection, analysis, interpretation, writing, and the decision to submit for publication). JH also reported being an advisor for Freenome. MI is a paid employee of Syapse. SK is a paid employee of TriNetX, LLC. RM is a paid employee and shareholder of Gilead Sciences. JR is a paid employee of and shareholder in Aetion, Inc., a company that makes software for the analysis of real-world data. AW is an employee and shareholder of Aetion, Inc., a company that makes software for the analysis of real-world data. This does not alter our adherence to PLOS One policies on sharing data and material. There are no patents, products in development, or marketed products associated with this research to declare.
Background
Despite a growing body of literature about COVID-19 and its cause, SARS-CoV-2, much remains unclear about which treatment strategies are most effective for the entire clinical course of the disease. Several therapeutic agents have been investigated, including the anti-malarial drug hydroxychloroquine. The evidence regarding its use for the treatment of COVID-19 continues to evolve [1, 2].
The use of hydroxychloroquine for COVID-19 was initially supported by in vitro studies showing anti-inflammatory and anti-SARS-CoV-2 activity [3]. Methodological weaknesses have marked subsequent in vivo studies; many studies have enrolled small numbers of patients and have not included appropriate control groups or methods to control for confounding variables, making interpretation of findings difficult [4]. While some non-randomized studies have shown a survival benefit for COVID-19 patients receiving hydroxychloroquine [5], others have found no evidence of benefit [6]. Still others have identified safety concerns, including an increased risk of prolonged QT intervals and arrhythmias for patients receiving hydroxychloroquine [7–9]. Additional studies have found that hydroxychloroquine may be more effective when given in combination with azithromycin [5] while emerging experience may indicate otherwise, leading to further uncertainty about the appropriate use of hydroxychloroquine. Differing methodologies to control for potential bias, incomplete capture of the timing of mechanical ventilation in relation to receipt of hydroxychloroquine, may have contributed to these inconsistencies.
Despite the development of treatment guidelines for COVID-19 [10], significant questions remain about best treatment practices. COVID-19 remains a significant global threat, and epidemiologic models have predicted that transmission will continue through the coming years [11]. Due to the significant morbidity and mortality associated with severe cases of COVID-19, establishing treatment guidelines is an essential step towards improving patient outcomes. It is therefore important to address methodological inconsistencies of existing studies and remaining uncertainties about the efficacy of hydroxychloroquine for the treatment of COVID-19.
Towards this end, the COVID-19 Evidence Accelerator convened by the Reagan-Udall Foundation for the FDA, in collaboration with Friends of Cancer Research, assembles experts from the health systems research, regulatory science, data science, and epidemiology to participate in parallel analyses. Analytic partners align on a common protocol and conduct analyses independently; methods and results are shared side-by-side to evaluate differences and similarities. Results are presented to a larger audience, including experts and leaders from the FDA, to provide informal discussion and review. Several groups, representing distinct populations within the U.S. to conduct parallel analyses of the effect of hydroxychloroquine, azithromycin, and the two drugs in combination on COVID-19 outcomes to compare results and better understand differences in the safety of these treatments for COVID-19.
Methods
Ethical statement
All the data partners received Institutional Review Board (IRB) approval or exemption. The use of VA data was approved by both the Department of Energy (DOE) (Oak Ridge Sitewide IRB00000547 for Protocol ORAU000718) and VA review committees and engages both VA and DOE researchers (VA-DOE Reliance Agreement under the authority of 38 U.S.C. 7303 and 38 U.S.C. 523). In addition to IRB approval, VA R&DC reviewed research proposals for final Institutional approval and ensured that all research in which the facility is engaged is consistent with the VA mission and complies with all applicable statutory and regulatory requirements. A Waiver of HIPAA Authorization and a Waiver of Informed Consent were approved for this study ORAU000718. The Aetion/HealthVerity study was approved under exemption by the New England Institutional Review Board (protocol #1-9757-1) and received a waiver of informed consent. The COTA study received approval by the Hackensack Meridian Health IRB (Pro2020-0342) and received a waiver of informed consent. The Health Catalyst dataset used for this analysis has been de-identified following the expert determination method outlined in 45 CFR 164.514(b)(1). Health Catalyst uses an external vendor to certify that the dataset is de-identified in accordance with 45 CFR 164.514(b)(1). The Dascena study received approval from the Pearl Institutional Review Board (20-DASC-120) and was granted a waiver of informed consent. The TriNetX Platform receives Protected Healthcare Information (PHI) or a Limited Data Set (LDS) from Healthcare Organizations (HCO) strictly under the constraints defined in a Business Associate Agreement (BAA) or a Data Use Agreement (DUA) under the United States (U.S.) Health Insurance Portability and Accountability Act (HIPAA). A fundamental Data Privacy principle is that TriNetX does not expose PHI or LDS to the end users of the TriNetX Platform. The data made available from the TriNetX platform is de-identified based on standard defined in Section §164.514(a) of the HIPAA Privacy Rule. The process by which Data Sets are de-identified is attested to through a formal determination by a qualified expert as defined in Section §164.514(b)(1) of the HIPAA Privacy Rule. Sypase conducted this work through a Research Collaboration Agreement with the FDA to include an IRB exemption through the Office of the Chief Scientist (OCS) Human Subject Protection (HSP) Executive Officer and all of this work involved data from secondary sources. The RCA work has been performed under an exemption from the Office of the Chief Scientist (OCS) Human Subject Protection (HSP) Executive Officer at FDA.
Data sources
The Evidence Accelerator partnered with seven groups to conduct the parallel analyses: Syapse, COTA/Hackensack Meridian Health (HMH), Dascena, TriNetX, Health Catalyst, Aetion, and Veteran’s Health Administration (VA). Each group conducting the parallel analysis collected data from their distinct sources. Syapse, COTA, Dascena, TriNetX, Health Catalyst and VA all utilized electronic health record (EHR) data, while Aetion utilized medical and pharmacy claims, and hospital billing data drawn from the HealthVerity Marketplace. Syapse utilized the EHR and molecular diagnostic lab information from two large Midwestern US health systems. COTA utilized data from the Real-world Evidence COVid RegistrY (RE-COV-RY) database collected at Hackensack Meridian Health System. Dascena utilized data from the EHRs from eight US hospitals. TriNetX drew data from the TriNetX Dataworks USA Network. Health Catalyst drew data from 17 Health Catalyst clients; the group had access to EHR data including medication administration. Aetion drew data from the HealthVerity linked medical and pharmacy claims, labs, and hospital chargemaster dataset. The VA used EHR data from the national VA Healthcare System, with COVID-19 cases adjudicated through a National Surveillance Tool [12]. A graphical description of coverage and overlap is illustrated in Fig 1 and characteristics of participating data sources and populations is described in Table 1.
The Evidence Accelerator partnered with seven groups to conduct the parallel analyses. This is a graphical description of coverage and overlap for each group conducting the parallel analysis and their distinct sources. Republished from https://www.brightcarbon.com/resources/editable-powerpoint-maps/ under a CC BY license, with permission from Bright Carbon, original copyright 2020.
Patient inclusions
Data were gathered from the 7 EHR datasets for hospital admissions in the U.S. between January 1, 2020 and June 30, 2020 (index hospitalization). Patients were eligible for inclusion if they had tested positive for COVID-19 during or prior to their visit or had an International Classification of Diseases (ICD)-10 code for COVID-19 in the 21 days leading up to admission, during admission or as a discharge diagnosis (Fig 2). All groups using ICD codes considered ICD-10 code U07.1; Syapse, Dascena and COTA additionally considered codes B97.21, B97.29, J12.81, B34.2; for the primary results Health Catalyst required a discharge diagnosis of U07.1 either primary or secondary to a related primary diagnoses (e.g., pneumonia or acute respiratory distress syndrome). To ensure accurate assessment of comorbidities, Syapse imposed a one-year minimum enrollment criteria to their study population of patients diagnosed with malignant cancers on or after January 1, 2015.
Common exclusion criteria were applied across the datasets to arrive at a final study population to assess patient characteristics and treatment outcomes for patients with COVID-19.
All patient data was maintained in compliance with the Health Insurance Portability and Accountability Act (HIPAA).
Treatment
Patients were considered to be treated with hydroxychloroquine and/or azithromycin if they received any of those medications at any point during their hospitalization, before discharge or death (the VA only considered treatments that occurred in the first 48 hours following hospitalization; Health Catalyst required treatment initiation within the first 2 days following hospitalization). Three treatment groups were compared to the population of patients that received neither hydroxychloroquine nor azithromycin. These treatment groups were hydroxychloroquine and azithromycin in combination, hydroxychloroquine alone, and azithromycin alone. Groups varied in their approach to determining date of cohort entry and index date of treatment (see S1 File for details). This study was non-interventional and treatment groups were not randomized.
Covariates
For each patient, data on potential demographic, medication and health-related covariates were extracted from structured EHR fields, prescription dates, and ICD-10 codes. In one analysis data were also extracted from structured and unstructured hospital billing data. In two networks, this data was augmented with claims data. A subset of participating groups adjusted for health-related variables and medication use prior to the index hospitalization, sociodemographic factors and comorbid conditions. The ICD codes used to identify comorbidities are presented in the S1 File. Covariates were selected independently by each group. Covariates considered by each group, and the method of their selection, are presented in Table 2.
Outcomes
Three primary outcomes were measured: use of mechanical ventilations (as a potential indicator of overall health status of patients), evidence of benefit of hydroxychloroquine (determined by hospital discharge as an indicator of recovery), and in-hospital mortality (determined by discharge disposition). Measurement of overall health status and evidence of benefit varies across groups, contingent on data availability. A summary of outcome definitions is presented in Table 3.
We also assessed the proportion of patients in each treatment group experiencing any of the following adverse events: diarrhea, hypoglycemia, cardiac arrest, abnormal electrocardiogram (ECG), arrhythmia, or prolonged QT interval. Adverse event data were not provided by Health Catalyst or the VA.
Analytic methods
Study follow-up began at slightly different time points within each group’s defined study design (S1 File). To improve baseline balance and minimize immortal time bias, in Aetion’s analyses, untreated were matched (on a number of key characteristics including calendar time and time since hospital admission) to treated patients on the day of first administration of HCQ (using risk set sampling). The index date for HCQ treated was the first day of treatment and the index date for the HCQ untreated was defined by the index date of the matched HCQ treated patient (for further detail see S1 File). For the VA, the index date was assigned to be 48 hours after hospital admission to avoid immortal time bias. For the other 4 datasets the index date was the date of admission.
Six groups used time-to-event analyses. The mortality outcome was primarily evaluated as in-hospital, for five of the groups, while the VA considered time to all-cause mortality within 30 days of the index date. Syapse examined the cumulative incidence of all-cause mortality during or after the index hospitalization. Health Catalyst conducted a primary analysis using mortality as time-to-event and sensitivity analysis treating mortality as binary.
Statistical analysis
To examine the association between potential confounders and treatment with hydroxychloroquine and azithromycin, hydroxychloroquine alone, and azithromycin alone, we compared the distributions of each covariate in each treatment group and in the group receiving no treatment. In addition, we examined the distribution of adverse events across treatment groups.
Methods to assess the association between treatment and outcomes included logistic regression, competing risk analyses, and propensity score methods. Dascena employed Fine and Gray models for the subdistribution hazard ratio (HR) were used to examine the association between treatment and each of the outcome measures. This method allows for estimation of the incidence of events, despite the presence of a competing event that precludes the observation of the event of interest. Incidence was estimated using Breslow’s estimator. All individuals who had not experienced the event were censored at the end of the study period. Additional information about statistical methods used by each group can be found in the S1 File.
To adjust for baseline confounding variables, a subset of groups employed propensity score methods. Logistic regression was used by five groups to predict the probability of treatment with hydroxychloroquine, azithromycin, or both in the study population, conditional on all measured confounders. The VA estimated propensity scores using a gradient boosting machine, implemented using the packages ‘gbm’ and ‘WeightIt’ in R [13–15]. Aetion performed all analyses in the Aetion Evidence Platform v4.5. Propensity scores were then used to adjust for confounding either through inverse probability of treatment weighting (IPTW), propensity score matching, or adjustment on the propensity score. Health Catalyst conducted sensitivity analyses using unmatched, 1:1 matched, propensity score matched, propensity score adjusted, and propensity score binned techniques. Details of the adjustment methods used by each group are presented in Table 4 and in the S1 File.
In each dataset, the association between each treatment (HCQ, HCQ+AZ) and each outcome was assessed in comparison to the non HCQ/AZ group (no treatment, “neither”). Therefore, treatment groups were not directly compared to each other in this analysis. For all analyses, a two-sided alpha of 0.05 without adjustment for multiple comparisons was used to determine statistical significance.
Results
Population
In total, 20,371 patient encounters from seven data sources were analyzed. Demographic characteristics of each dataset are laid out in Figs 3–5, with patient comorbidities presented in Table 5. Patient characteristics were similar across datasets. Patients receiving treatment with hydroxychloroquine were typically older than 45 with a larger proportion of males.
Categorical age for each treatment group shows similarities and differences across data sets. Note. Numbers represent percent values.
Sex distribution by treatment group shows similarities and differences across data sets. Note. Numbers represent percent values.
Race distribution by treatment group shows similarities and differences across data sets. Note. Numbers represent percent values.
For the adverse event with most complete reporting, any arrhythmia, analysis did not support increased arrhythmia in patients receiving hydroxychloroquine versus those not (Fisher’s Exact Test p-value 0.462) (Table 6).
Outcomes
Frequencies of each evaluated outcome are displayed in Table 7. In the Syapse study population of patients with cancer and COVID-19, crude all-cause mortality estimates were greatest among patients receiving hydroxychloroquine plus azithromycin (29.6%), followed by azithromycin alone (21.3%). The crude cumulative incidence of mechanical ventilation was also greatest in the hydroxychloroquine plus azithromycin treatment arm (35.2%), followed by hydroxychloroquine alone (22.9%).
Given constraints of the data, not all groups performed an adjusted analysis. Six groups (Dascena, Health Catalyst, TriNetX, Aetion, COTA/HMH and the VA) conducted adjusted analyses. Among the 3 groups that conducted comparative analyses between hydroxychloroquine plus azithromycin and monotherapy treatment groups (Dascena, Health Catalyst and the VA), after adjusting for confounding, hydroxychloroquine alone was not found to be associated with mortality, overall patient condition, or benefit to the patient. Interrogation of confounding mitigation demonstrated an ability to balance across treatment groups especially on key characteristics such as age and comorbid burden. Hydroxychloroquine plus azithromycin was similarly not associated with any of the outcomes assessed in this study. Adjusted results are presented in Table 8.
Based on these results, hydroxychloroquine and azithromycin, alone or in combination, did not appear to impact outcomes among COVID-19 patients. These results were consistent across datasets, with the only notable difference being the evidence of benefit with azithromycin treatment observed by Dascena. Besides the primary analysis, Health Catalyst performed additional sensitivity analyses based upon (a) inclusion criteria (positive lab or ICD-10; require primary discharge ICD-10; allow treatment initiation after the second day), (b) treating mortality as binary, (c) different confounders and confounding mitigation techniques. Results were as expected: (a) broader inclusion criteria resulted in more baseline group differences, (b) treating mortality as binary reduced potential treatment benefit especially when not requiring rapid treatment initiation (e.g., immortal time bias), and (c) more extreme results were observed in the absence of any confounder adjustment technique and different techniques had small impacts of results.
Discussion
In this study, a consortium of groups conducted parallel analyses of the effects of hydroxychloroquine, azithromycin, and their combination on health outcomes of COVID-19 patients. By conducting parallel analyses that aligned on a common protocol, while allowing for flexibility within each group to define covariates, exposure and outcome identification, this study aimed to provide a robust description of outcomes associated with the use of hydroxychloroquine for the treatment of COVID-19.
Among five sites that contributed race data, Syapse, the VA and Health Catalyst reported that Black patients made up a larger distribution of HCQ/HCQ+AZM recipients compared to their distribution without these treatments. Race did not appear to be associated with HCQ administration in the COTA/HMH dataset; and White patients represented a greater proportion of HCQ/HCQ+AZM recipients in the TriNetX dataset as compared to their distribution among those without HCQ treatment.
Across all datasets and treatment groups, the most prominent pre-existing conditions tended to be any cardiovascular disease, hypertension, diabetes and obesity. Overall, obesity was more prevalent among the HCQ treatment groups than in the neither group.
For most data partners, the proportion of patients treated with any of these comorbidities was lower or no different in the HCQ groups than in the neither group–with the exception of Syapse, which was a cancer cohort.
There are several limitations to this study that must be acknowledged. First, despite our goal of carrying out the same set of analyses on multiple datasets, analyses could not be carried out identically on all datasets due to differences in and limitations of the data. In particular, not all groups were able to carry out an adjusted analysis that controlled for confounding variables. Second, due to stratification on treatment groups, some analyses were conducted on small sample sizes. Data was also limited to those collected from United States sources. These results may therefore not be generalizable to international settings. Coding of certain outcomes, particularly adverse events, may have been incomplete. Therefore, not all outcomes may have been captured, potentially limiting the accuracy of our results.
These analyses were conducted in parallel among 7 individual groups using their own datasets. Characteristics, definitions, and methodologies used by each of the groups are summarized in Tables 1 and 4. A goal of this project was to develop a common analytical plan for multiple groups to apply to different datasets as they continued to aggregate data on the experience of diagnosis, treatment, and outcomes associated with COVID-19. Given the novelty of the virus and magnitude of the pandemic, the use of data derived from various sources of healthcare data presents an opportunity to augment clinical trial data with information about patients not enrolled in clinical studies, and provide information about treatment patterns and observations about those experiences in large, diverse populations. By using a common analysis plan, the resulting observations can be more readily compared and if consistent, can further support the findings among individual studies.
In this instance, several of the studies were already underway when the parallel analysis was started. This made it difficult to align on all aspects of the analyses and data parameters. As future parallel analyses are considered, it is important that participants seek to develop as much uniformity to the definitions and methods as possible. However, given the different sources of data, some aspects of the analysis will need to be tailored to the individual dataset and those variations should be clearly described. As future study questions are developed to further characterize COVID-19 treatments, it is important to select the sources of data that are best fit to answer each specific question. In addition, future parallel analysis should consider using a stepwise approach to perform a sample size, demographic, and feasibility assessment and use that initial step to optimally design subsequent comparative analyses.
Conclusion
The Evidence Accelerator successfully brought together seven partners to execute analyses in disparate populations. Representing more than 20,000 patients with COVID-19 across the U.S, we found similar trends in those getting HCQ treatment–despite minor differences in coding and cohort entry. Across the 5 groups who ran comparative analyses, we observed no association between HCQ treatment and mortality, overall patient condition, or evidence of benefit.
Acknowledgments
The COVID-19 Evidence Accelerator is a unique partnership convened by the Reagan-Udall Foundation for the FDA, in collaboration with Friends of Cancer Research, to create an interactive real-world evidence community focused on aspects of COVID-19 natural history, diagnosis and treatment. This manuscript was submitted on behalf of the organizations that make up the COVID-19 Evidence Accelerator (participants in the COVID-19 Evidence Accelerator are available at https://evidenceaccelerator.org/). We thank Grace Collins and Eric Hua (Friends of Cancer Research) for their assistance in manuscript preparation. We also thank the participants in the Evidence Accelerator activities, including representatives from the U.S. Food and Drug Administration, for their expertise, engagement, and technical assistance. https://evidenceaccelerator.org/
References
- 1. Cortegiani A, Ingoglia G, Ippolito M, Giarratano A, Einav S. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care. 2020 Jun;57:279–83. pmid:32173110
- 2. Fiolet T, Guihur A, Rebeaud ME, Mulot M, Peiffer-Smadja N, Mahamat-Saleh Y. Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis. Clin Microbiol Infect Off Publ Eur Soc Clin Microbiol Infect Dis. 2020 Aug;
- 3. Yao X, Ye F, Zhang M, Cui C, Huang B, Niu P, et al. In Vitro Antiviral Activity and Projection of Optimized Dosing Design of Hydroxychloroquine for the Treatment of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2). Clin Infect Dis an Off Publ Infect Dis Soc Am. 2020 Jul;71(15):732–9.
- 4. Singh AK, Singh A, Singh R, Misra A. “Hydroxychloroquine in patients with COVID-19: A Systematic Review and meta-analysis.”. Diabetes Metab Syndr. 2020;14(4):589–96. pmid:32417708
- 5. Gautret P, Lagier J-C, Parola P, Hoang VT, Meddeb L, Mailhe M, et al. Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial. Int J Antimicrob Agents. 2020 Jul;56(1):105949. pmid:32205204
- 6.
No clinical benefit from use of hydroxychloroquine in hospitalised patients with COVID-19—RECOVERY Trial [Internet]. 2020. Available from: https://www.recoverytrial.net/news/statement-from-the-chief-investigators-of-the-randomised-evaluation-of-covid-19-therapy-recovery-trial-on-hydroxychloroquine-5-june-2020-no-clinical-benefit-from-use-of-hydroxychloroquine-in-hospitalised-patients-with-co
- 7. Rosenberg ES, Dufort EM, Udo T, Wilberschied LA, Kumar J, Tesoriero J, et al. Association of Treatment With Hydroxychloroquine or Azithromycin With In-Hospital Mortality in Patients With COVID-19 in New York State. JAMA. 2020 Jun;323(24):2493–502. pmid:32392282
- 8. Ip A, Berry DA, Hansen E, Goy AH, Pecora AL, Sinclaire BA, et al. Hydroxychloroquine and tocilizumab therapy in COVID-19 patients-An observational study. PLoS One. 2020;15(8):e0237693. pmid:32790733
- 9. Cavalcanti AB, Zampieri FG, Rosa RG, Azevedo LCP, Veiga VC, Avezum A, et al. Hydroxychloroquine with or without Azithromycin in Mild-to-Moderate Covid-19. N Engl J Med [Internet]. 2020 Jul 23; Available from: https://doi.org/10.1056/NEJMoa2019014
- 10.
Coronavirus Disease 2019 (COVID-19) Treatment Guidelines [Internet]. 2020. Available from: Coronavirus Disease 2019 (COVID-19)%0ATreatment Guidelines
- 11. Kissler SM, Tedijanto C, Goldstein E, Grad YH, Lipsitch M. Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period. Science. 2020 May;368(6493):860–8. pmid:32291278
- 12. Chapman A, Peterson K, Turano A, Box T, Wallace K, Jones M. A Natural Language Processing System for National COVID-19 Surveillance in the US Department of Veterans Affairs [Internet]. 2020. Available from: https://openreview.net/pdf?id=ZQ_HvBxcdCv
- 13. Ridgeway G. Generalized Boosted Models: A guide to the gbm package. Update. 1(1):2007.
- 14. Greenwell B, Boehmke B, Cunningham J, Developers GBM. gbm: Generalized boosted regression models. R Packag version. 2019;2(5).
- 15. Greifer N. WeightIt: Weighting for Covariate Balance in Observational Studies [Internet]. 2020. Available from: https://cran.r-project.org/web/packages/WeightIt/index.html