https://orcid.org/0000-0002-6278-4969
Figures
Abstract
Background
Atrial fibrillation (AF) burden on patients and healthcare systems warrants innovative strategies for screening asymptomatic individuals.
Objective
We sought to externally validate a predictive model originally developed in a German population to detect unidentified incident AF utilising real-world primary healthcare databases from countries in Europe and Australia.
Methods
This retrospective cohort study used anonymized, longitudinal patient data from 5 country-level primary care databases, including Australia, Belgium, France, Germany, and the UK. The study eligibility included adult patients (≥45 years) with either an AF diagnosis (cases) or no diagnosis (controls) who had continuous enrolment in the respective database prior to the study period. Logistic regression was fitted to a binary response (yes/no) for AF diagnosis using pre-determined risk factors.
Results
AF patients were from Germany (n = 63,562), the UK (n = 42,652), France (n = 7,213), Australia (n = 2,753), and Belgium (n = 1,371). Cases were more likely to have hypertension or other cardiac conditions than controls in all validation datasets compared to the model development data. The area under the receiver operating characteristic (ROC) curve in the validation datasets ranged from 0.79 (Belgium) to 0.84 (Germany), comparable to the German study model, which had an area under the curve of 0.83. Most validation sets reported similar specificity at approximately 80% sensitivity, ranging from 67% (France) to 71% (United Kingdom). The positive predictive value (PPV) ranged from 2% (Belgium) to 16% (Germany), and the number needed to be screened was 50 in Belgium and 6 in Germany. The prevalence of AF varied widely between these datasets, which may be related to different coding practices. Low prevalence affected PPV, but not sensitivity, specificity, and ROC curves.
Citation: Mokgokong R, Schnabel R, Witt H, Miller R, Lee TC (2022) Performance of an electronic health record-based predictive model to identify patients with atrial fibrillation across countries. PLoS ONE 17(7): e0269867. https://doi.org/10.1371/journal.pone.0269867
Editor: Gaetano Santulli, Albert Einstein College of Medicine, UNITED STATES
Received: December 1, 2021; Accepted: May 27, 2022; Published: July 8, 2022
Copyright: © 2022 Mokgokong et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: Funding for this study was provided by Pfizer and Bristol Myers Squibb. Authors RM, HW, RM, and TL are employees and shareholders of Pfizer and were responsible for conceptualization, methodology, project administration, supervision, visualization, and writing (original draft and review/editing).
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: RBS has received lecture fees and advisory board fees from BMS/Pfizer outside this work. RM, HW, RM, and TL are employees and shareholders of Pfizer. RBS has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme under the grant agreement No 648131, from the European Union’s Horizon 2020 research and innovation programme under the grant agreement No 847770 (AFFECT-EU) and German Center for Cardiovascular Researh (DZHK e.V.) (81Z1710103); German Ministry of Research and Education (BMBF 01ZX1408A) and ERACoSysMed3 (031L0239). RBS has received lecture fees and advisory board fees from BMS/Pfizer outside this work. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Atrial fibrillation (AF) is a common type of arrhythmia that increases the risk for stroke by 5-fold potentially leading to early mortality [1, 2]. In 2016, the prevalence of AF was 46.3 million worldwide, and has increased roughly three-fold in the last 50 years [3, 4]. Additionally, AF risk level in older men is reportedly higher than in women [5, 6]. The overall prevalence of AF may be higher due to the clinical challenge of diagnosing paroxysmal and/or asymptomatic cases [7]. Timely treatment of AF could reduce the risk of stroke and stroke-related morbidity and mortality [8]; however, many patients are unaware of their condition until after enduring an acute ischemic stroke [9]. This arrhythmia is associated with significant healthcare expenditures primarily due to hospital admissions costs [10].
The burden of AF on patients and on the healthcare system warrants innovative strategies for screening asymptomatic individuals [11]. Mass screening with electrocardiography (ECG) would be expensive and 83 individuals would need to be screened to identify one individual with treatable AF [12]. Therefore, guidelines recommend opportunistic screening during regular health visits in patients >65 years by either ECG, which is the gold standard for diagnosing AF, or by pulse taking, and systematic screening in patients ≥75 years to detect AF [13]. The recommendation for opportunistic screening is designated as Class I (level B for evidence) and the recommendation for systematic screening is designated as Class IIa (level B for evidence) [13].
A more cost-effective and novel approach to pre-select patients for AF screening involves using routinely collected data. Machine learning can be used to develop predictive models to select patients who have specific risk factors and who should be screened for AF using an ECG. Automated machine learning has previously assessed the discriminatory ability of models to screen for AF using large claims databases and electronic health records [14–16]. Predictive tools that can identify risk factors and other predictors for AF and that can be applied to routinely collected data offer a targeted approach leading to screening in the primary care setting [7]. Big data and automated analysis in healthcare provides an avenue for digital innovation and personalized medicine, and further research is needed to evaluate whether predictive modeling effectively identifies AF in different sources of data. The purpose of this study was to externally validate a model developed by Schnabel et al. in a German population [16] for the prediction of incident AF utilising several real-world primary healthcare databases from countries in Europe and in Australia.
Methods
Study design
This retrospective cohort study used anonymized, longitudinal patient data from 5 country-level IQVIA databases, including Australia, Belgium, France, Germany, and the United Kingdom. The German dataset in this validation study is a primary care electronic medical record and differs from the dataset used in the model development study, which was a claims dataset (S1 Table in S1 File for countrywide comparison of datasets). The study eligibility included adult patients (≥45 years) who had continuous enrolment in the respective database prior to the study period (Fig 1). Patients were considered to have incident AF if they had an AF International Classification of Diseases– 10th edition (ICD-10) diagnosis code during the observation period. Control patients had no specific AF diagnosis during the entire study. The study was approved by IQVIA IMRD Independent Scientific Review Committee. Patient consent was waived by the ethics committee because data were fully anonymized.
Observation period
The baseline period was 1,095 days prior to the index date, and this was used to identify eligible cases and assess patient risk factor information for both cases and controls. The observation period for cases was from January 1st, 2013 to December 31st, 2016. Control patients who were not diagnosed with AF during the study duration were observed from January 1st, 2010 to December 31st, 2016. All patients had an index date recorded during the observation period. In cases, the index case was the first date of an AF diagnosis, whereas controls were assigned a pseudo index date given they had no AF diagnosis. A sensitivity analysis was performed using the most recent data extracted from the Germany and France database (S1 Fig in S1 File for the study design of sensitivity analysis). In the Germany dataset, the study period was shifted to start from January 1st, 2013 to December 31st, 2019, and the observation period was from January 1st, 2016 to December 31st, 2019. In the France dataset, the study period was shifted to start from January 1st, 2012 to December 31st, 2018, and the observation period was from January 1st, 2015 to December 31st, 2018.
Covariates
This study described and compared risk factors included in a predictive model developed using a German study population [16], which included age, sex, hypertension-treated, heart failure-treated, valvular heart disease, chronic kidney disease, stroke, hemiplegia, other pulmonary heart diseases, paroxysmal tachycardia, other cardiac arrhythmias, ulcer of lower limb, and personal history of medical treatment (S2 Table in S1 File). The model we used was based on a recent study (S3 Table in S1 File) [16]. In the model development study, risk factors for incident AF were identified via gradient-boosting with component-wise linear models. From the final gradient-boosted model, an initial list of 43 risk factors for incident AF were identified. Logistic regression models of increasing complexity were then fitted using subsets of the 43 risk factors identified by the gradient-boosted model. An optimal classification model containing 13 of the 43 selected risk factors was identified. These 13 risk factors were defined by three-digit ICD-10 GM10GM (Germany-specific) codes (diagnostic recording) and four- or five-digit ATC (Anatomical Therapeutic Chemical) codes (treatment recording) (S4 Table in S1 File).
In this study, risk factors were identified using the same ICD-10, ATC codes, and Read codes (United Kingdom; S4 and S5 Tables in S1 File).
Risk factors were identified during the baseline period or at the index date (single code per covariate). This was the same approach as in the model development study. For hypertension-treated, patients were required to have a recording of both an ICD-10 and ATC related code during the baseline period for the risk factor to be recorded as ‘present’ for that patient. This also applied to the heart failure-treated variable.
Statistical analysis
The analysis was implemented in three phases: 1) model-relatedness investigation i.e. a comparison of risk factors across databases; 2) predictive model specification and external validation; and 3) sensitivity analyses. To explore and compare similarities in variables across five country-level databases, descriptive statistics characterised the presence of the risk factors in both cases and controls. Two-way linear dependence on all risk factors was identified using Pearson’s correlation coefficient. Logistic regression model was then fitted to a binary response (yes/no) for AF diagnosis using the pre-determined risk factors. This predictive model was implemented in all databases.
In the external validation phase, performance metrics were evaluated, including model discrimination. Missing data were treated as absent, which is a common statistical approach to analysing observational data from electronic medical records [17]. Model predictions were added into one of the following categories:
- True positives (TP): Incident AF patients correctly identified by the predictive model as having had the outcome
- False positives (FP): Patients without incident AF incorrectly identified by the predictive model as having had the outcome
- True negatives (TN): Patients without incident AF correctly identified by the predictive model as NOT having had the outcome
- False negatives (FN): Incident AF patients incorrectly identified by the predictive model as NOT having had the outcome
Based on the definitions, the measures of model performance for each validation data source included:
- True positive rate (Sensitivity/Recall): This is ability of the predictive model to correctly identify patients with incident AF
- True negative rate (Specificity): This is the ability of the predictive model to correctly identify patients without incident AF
- Positive predictive value (PPV, Precision): This is the proportion of true positive model predictions out of all positive model predictions
The discriminative ability of the model was assessed using receiver operating characteristic (ROC) curves. The area under the curve (AUC) was used to determine the predictive performance of the model. Performance metrics such as PPV are dependent on class imbalance and, therefore, influenced by the AF prevalence in the datasets. To determine the level of clinical burden, PPV, the proportion of true positive model predictions out of all positive model predictions, and number needed to screen, the number of patients that it is necessary to further evaluate to detect one patient with AF were calculated.
In the analysis, the intercepts of all validation models were estimated to approximate the baseline AF risk in the model development population (S1 File). In the sensitivity analysis, the intercept was calibrated to each validation dataset.
The performance of the algorithm was also assessed by calculating the Brier score in each data source. This score considers both discrimination and calibration and is a useful summary measure of the overall accuracy of the algorithm. The Brier score can take values between 0 and 1, where a Brier score of 0 would indicate a perfectly predicting model (i.e. 0 error in model prediction).
All analyses were conducted with R version 3.6.1.
Results
Patient characteristics
The characteristics of patients with AF who were eligible for the study are highlighted in Table 1. Most cases were from Germany (n = 63,562), followed by the UK (n = 42,652), France (n = 7,213), Australia (n = 2,753), and Belgium (n = 1,371), and the majority of controls included in the study were from Germany (n = 1,543,436); the baseline characteristics of both cases and controls are shown in Table 2. The distribution of age and sex between AF and controls were comparable in the validation datasets and the original model development study (S2 Table in S1 File), although treated hypertension was more prevalent in cases than controls in all databases. The overall prevalence of risk factors in patients with AF differed between country-level databases, although the key risk factors were more common in AF versus control groups. No strong correlation (> ±0.5) was observed between risk factors. The correlation plots are included in the S2 Fig in S1 File. The highest proportion of patients with AF was observed in the Australia data set (n = 2,753/51,761, 5.3%), followed by Germany (n = 63,562/1,606,998, 4.0%), the UK (n = 42,652/1,278,423, 3.3%), France (n = 7,213/434,323, 1.7%), and Belgium (n = 1,371/123,464, 1.1%). In comparison, the original German model identified 66,697cases (5.6%) of incident AF.
Model relatedness between original model and validation databases
Patients with AF were more likely to have hypertension, heart failure or other cardiac conditions than controls in all validation datasets than the model development data. Certain comorbidities of interest were not significantly associated with AF in the validation datasets, such as chronic kidney disease in the France dataset, hemiplegia and other pulmonary heart disease in the Belgium dataset, and chronic kidney disease and lower limb ulcers in the Australian dataset.
Model performance
The original model had a sensitivity of 80% and a specificity of 72%. In the validation datasets at approximately 80% sensitivity, the model reported similar specificity within an acceptable range (France: 67% and UK: 71%), although the specificity in the Belgium dataset was 59% (Table 3). The original model with 80% sensitivity had a PPV of 14% and required 7 patients to be screened; in the validation datasets, PPV ranged from 2% (Belgium) to 16% (Germany), which required 50 and 6 patients to be screened, respectively. This was related to the prevalence of AF in the databases. The AUC in the validation datasets ranged from 0.79 (Belgium) to 0.84 (Germany), comparable to the original model, which had an AUC of 0.83 (Fig 2). All ROC curves bend to the top-left corner indicating good model performance. Brier scores are reported in S6 Table in S1 File.
Receiver operating characteristic (ROC) curve for IQVIA Medical Research Data (IMRD) in A) UK B) France C) Germany D) Belgium E) Australia.
Sensitivity results
The shift in study time period did not significantly change the baseline characteristics observed in patients with AF versus controls in the Germany and France datasets. However, patients in the German dataset had increased odds for having a personal history of medication for AF (OR: 9.45, 95% CI: 9.17−9.73) and decreased odds in other cardiac arrhythmias (OR: 3.77, 95%: 3.65−3.88). In the France data set, patients had increased odds for having paroxysmal tachycardia (OR: 2.50, 95% CI: 2.15−2.91). At 80% sensitivity, the PPV increased slightly in the Germany dataset, but remained the same for the France dataset. Compared to the main study, the ROC curves in the sensitivity analysis a similar AUC in the Germany and France dataset (S3A and S3B Fig in S1 File). Overall, the shift in study time period did not markedly alter the model validation results for both datasets.
Discussion
This study sought to assess the predictive capability of one model to screen for AF in several large databases. The validation datasets demonstrated a comparable performance in terms of sensitivity and specificity and AUC to the original German model. When controlling for AF, prevalence PPV was also similar. Predicting events such as AF with low prevalence is challenging and classic model performance metrics alone are not appropriate to evaluate the value of screening tools in this setting. Assessing algorithms in low prevalence diseases should consider benefit and burden to the system. This can be evaluated using the PPV and number needed to screen to assess the potential burden of clinical implementation as a result of false-positives, and metrics, such as sensitivity to assess the benefit of the algorithm in identifying positive cases [18]. In this analysis, the prevalence of AF varied widely between these datasets, and low prevalence affected some metrics, such as PPV, although sensitivity, specificity, and ROC were not affected by prevalence. Therefore, the model performs variably if the prevalence of AF is relatively low, however, performance may also be related to how variables were recorded in the datasets, in addition to differences in patient populations across datasets.
The higher sensitivity of the model can offer increased clinical value in screening for AF. This model can predict AF using electronic medical records in a primary care setting as demonstrated by the validation datasets in this study, and claims data including in hospital settings as shown by the original model development study, which supports implementation efforts in different care settings that utilise ICD-10 coding.
Algorithms that predict AF using clinical factors have shown to be relatively accurate in several studies. For instance, a 10-year risk score developed to identify AF in Caucasians and African Americans showed moderate discrimination (AUC = 0.78) [19]. Another study using machine learning to predict AF demonstrated 75% sensitivity compared to the CHARGE-AF model, thereby reducing the number of patients needed to be screened [14]. Further, a predictive model derived from pooled individual-level primary care data from several large US population cohorts was used to predict AF [20], and showed good discrimination in two validation cohorts [21]. Therefore, studies have increasingly implemented predictive algorithms using electronic health records for identifying AF [22, 23] and stratifying the risk for stroke [24]. Using routine clinical data to screen for AF enables researchers and clinicians to make earlier diagnoses of AF, assess the clinical burden over time and to improve the timeliness of guideline-recommended treatment. Algorithms can be integrated in clinical decision support systems to analyze data within electronic health care records automatically and to provide prompts and reminders to assist health care providers. This strategy may augment the implementation of guideline recommendations in clinical practice and may increase cost-effectiveness of pre-selecting patients at-risk for AF by using algorithms [25]. In the current study, the model was able to identify approximately 80% of all positive cases and 71% of negative cases correctly. Four of the five calibrated models performed at expected real-world prevalence rates but deviated when prevalence decreased below proportions one would expect to see in a real-world setting. It is likely that the model could be implemented in different settings and with different designs in a real-world setting. For example, Schnabel et al. (2022) also validated the model in a prospective cohort in ambulatory patients with cardiovascular risk [16]. If implemented in electronic health records, patients at high risk for AF could be identified and invited for confirmatory screening similar to a clinical study designed by Hill et al. [26]. The most suitable approach for implementation will likely depend on the healthcare system, provider, and data infrastructure.
Some limitations should be acknowledged. Controls required continuous enrolment from the start of the study period to the end of the study period or death. This was to ensure that controls did not develop AF during the entire study period. Continuous enrolment may cause generalizability issues, particularly in claims databases from countries such as the US where patients are more likely to move between health plans. However, continuous enrolment is likely to be limited in countries with EMR and national healthcare systems such as countries in Europe. Additionally, bias may be introduced into database studies because of miscoding. In this study, it is feasible that AF might have been coded as “other cardiac arrhythmias". However, AF is a well-defined and common arrhythmia which is known to physicians across specialties. Coding of AF is important for re-imbursement. Misclassification can certainly occur but is less likely for this arrhythmia. Moreover, data presented in this paper utilize Western countries only, and, therefore, may not be generalisable to other countries. Given that this study had a retrospective study design, future prospective studies would help to assess the utility of these models through real-world implementation.
Conclusions
Given the serious health consequences of AF, screening algorithms offer targeted ways to identify at-risk patients using electronic health records and may help to improve the cost-effectiveness of screening programs. Based on the results of this study, the algorithm can be transferred to databases from different healthcare systems.
Acknowledgments
The authors thank Katie Crosslin, PhD, and Holly Richendrfer, PhD, of Evidera/PPD for medical writing assistance.
References
- 1. January CT, Wann LS, Alpert JS, Calkins H, Cigarroa JE, Cleveland JC Jr., et al. 2014 AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines and the Heart Rhythm Society. Circulation. 2014;130(23):2071–104. pmid:24682348
- 2. Dilaveris PE, Kennedy HL. Silent atrial fibrillation: epidemiology, diagnosis, and clinical impact. Clin Cardiol. 2017;40(6):413–8. pmid:28273368
- 3. Kornej J, Börschel CS, Benjamin EJ, Schnabel RB. Epidemiology of Atrial Fibrillation in the 21st Century: Novel Methods and New Insights. Circ Res. 2020;127(1):4–20. pmid:32716709
- 4. Benjamin EJ, Muntner P, Alonso A, Bittencourt MS, Callaway CW, Carson AP, et al. Heart Disease and Stroke Statistics-2019 Update: A Report From the American Heart Association. Circulation. 2019;139(10):e56–e528. pmid:30700139
- 5. Zoni-Berisso M, Lercari F, Carazza T, Domenicucci S. Epidemiology of atrial fibrillation: European perspective. Clin Epidemiol. 2014;6:213–20. pmid:24966695
- 6. Magnussen C, Niiranen TJ, Ojeda FM, Gianfagna F, Blankenberg S, Njølstad I, et al. Sex Differences and Similarities in Atrial Fibrillation Epidemiology, Risk Factors, and Mortality in Community Cohorts: Results From the BiomarCaRE Consortium (Biomarker for Cardiovascular Risk Assessment in Europe). Circulation. 2017;136(17):1588–97. pmid:29038167
- 7. Sekelj S, Sandler B, Johnston E, Pollock KG, Hill NR, Gordon J, et al. Detecting undiagnosed atrial fibrillation in UK primary care: Validation of a machine learning prediction algorithm in a retrospective cohort study. Eur J Prev Cardiol. 2020:2047487320942338.
- 8. Friberg L, Rosenqvist M, Lindgren A, Terént A, Norrving B, Asplund K. High prevalence of atrial fibrillation among patients with ischemic stroke. Stroke. 2014;45(9):2599–605. pmid:25034713
- 9. Jones NR, Taylor CJ, Hobbs FDR, Bowman L, Casadei B. Screening for atrial fibrillation: a call for evidence. Eur Heart J. 2020;41(10):1075–85. pmid:31811716
- 10. Burdett P, Lip GYH. Atrial Fibrillation in the United Kingdom: Predicting Costs of an Emerging Epidemic Recognising and Forecasting the Cost Drivers of Atrial Fibrillation-related costs. Eur Heart J Qual Care Clin Outcomes. 2020.
- 11. Freedman B, Camm J, Calkins H, Healey JS, Rosenqvist M, Wang J, et al. Screening for Atrial Fibrillation: A Report of the AF-SCREEN International Collaboration. Circulation. 2017;135(19):1851–67. pmid:28483832
- 12. Lowres N, Olivier J, Chao TF, Chen SA, Chen Y, Diederichsen A, et al. Estimated stroke risk, yield, and number needed to screen for atrial fibrillation detected through single time screening: a multicountry patient-level meta-analysis of 141,220 screened individuals. PLoS Med. 2019;16(9):e1002903. pmid:31553733
- 13. Hindricks G, Potpara T, Dagres N, Arbelo E, Bax JJ, Blomström-Lundqvist C, et al. 2020 ESC Guidelines for the diagnosis and management of atrial fibrillation developed in collaboration with the European Association for Cardio-Thoracic Surgery (EACTS). Eur Heart J. 2021;42(5):373–498. pmid:32860505
- 14. Hill NR, Ayoubkhani D, McEwan P, Sugrue DM, Farooqui U, Lister S, et al. Predicting atrial fibrillation in primary care using machine learning. PLoS One. 2019;14(11):e0224582. pmid:31675367
- 15. Khurshid S, Kartoun U, Ashburner JM, Trinquart L, Philippakis A, Khera AV, et al. Performance of Atrial Fibrillation Risk Prediction Models in Over 4 Million Individuals. Circ Arrhythm Electrophysiol. 2021;14(1):e008997. pmid:33295794
- 16. Schnabel RB, Witt H, Walker J, Ludwig M, Geelhoed B, Kossack N, et al. Machine learning-based identification of risk-factor signatures for undiagnosed atrial fibrillation in primary prevention and post-stroke in clinical practice. European Heart Journal—Quality of Care and Clinical Outcomes. 2022. pmid:35436783
- 17. Reps JM, Schuemie MJ, Suchard MA, Ryan PB, Rijnbeek PR. Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data. J Am Med Inform Assoc. 2018;25(8):969–75. pmid:29718407
- 18. Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care. 2015;19(1):285.
- 19. Chamberlain AM, Agarwal SK, Folsom AR, Soliman EZ, Chambless LE, Crow R, et al. A clinical risk score for atrial fibrillation in a biracial prospective cohort (from the Atherosclerosis Risk in Communities [ARIC] study). Am J Cardiol. 2011;107(1):85–91. pmid:21146692
- 20. Alonso A, Krijthe BP, Aspelund T, Stepas KA, Pencina MJ, Moser CB, et al. Simple risk model predicts incidence of atrial fibrillation in a racially and geographically diverse population: the CHARGE-AF consortium. J Am Heart Assoc. 2013;2(2):e000102. pmid:23537808
- 21. Christophersen IE, Yin X, Larson MG, Lubitz SA, Magnani JW, McManus DD, et al. A comparison of the CHARGE-AF and the CHA2DS2-VASc risk scores for prediction of atrial fibrillation in the Framingham Heart Study. Am Heart J. 2016;178:45–54. pmid:27502851
- 22. Tu K, Nieuwlaat R, Cheng SY, Wing L, Ivers N, Atzema CL, et al. Identifying Patients With Atrial Fibrillation in Administrative Data. Can J Cardiol. 2016;32(12):1561–5. pmid:27742459
- 23. Hulme OL, Khurshid S, Weng LC, Anderson CD, Wang EY, Ashburner JM, et al. Development and Validation of a Prediction Model for Atrial Fibrillation Using Electronic Health Records. JACC Clin Electrophysiol. 2019;5(11):1331–41. pmid:31753441
- 24. Han L, Askari M, Altman RB, Schmitt SK, Fan J, Bentley JP, et al. Atrial Fibrillation Burden Signature and Near-Term Prediction of Stroke: A Machine Learning Analysis. Circ Cardiovasc Qual Outcomes. 2019;12(10):e005595. pmid:31610712
- 25. Orchard J, Freedman B, Lowres N, Neubeck L. AF-SMART eHealth system for atrial fibrillation screening: how can it fit into clinical practice? Expert Rev Med Devices. 2020;17(5):375–8. pmid:32270721
- 26. Hill NR, Arden C, Beresford-Hulme L, Camm AJ, Clifton D, Davies DW, et al. Identification of undiagnosed atrial fibrillation patients using a machine learning risk prediction algorithm and diagnostic testing (PULsE-AI): Study protocol for a randomised controlled trial. Contemporary Clinical Trials. 2020;99:106191. pmid:33091585