http://orcid.org/0000-0003-1009-6563
Figures
Abstract
Background
Risk models (RM) need external validation to assess their value beyond the setting in which they were developed. We validated a RM combining mpMRI and clinical parameters for the probability of harboring significant prostate cancer (sPC, Gleason Score ≥ 3+4) for biopsy-naïve men.
Material and methods
The original RM was based on data of 670 biopsy-naïve men from Heidelberg University Hospital who underwent mpMRI with PI-RADS scoring prior to MRI/TRUS-fusion biopsy 2012–2015. Validity was tested by a consecutive cohort of biopsy-naïve men from Heidelberg (n = 160) and externally by a cohort of 133 men from University College London Hospital (UCLH). Assessment of validity was performed at fusion-biopsy by calibration plots, receiver operating characteristics curve and decision curve analyses. The RM`s performance was compared to ERSPC-RC3, ERSPC-RC3+PI-RADSv1.0 and PI-RADSv1.0 alone.
Results
SPC was detected in 76 men (48%) at Heidelberg and 38 men (29%) at UCLH. The areas under the curve (AUC) were 0.86 for the RM in both cohorts. For ERSPC-RC3+PI-RADSv1.0 the AUC was 0.84 in Heidelberg and 0.82 at UCLH, for ERSPC-RC3 0.76 at Heidelberg and 0.77 at UCLH and for PI-RADSv1.0 0.79 in Heidelberg and 0.82 at UCLH. Calibration curves suggest that prevalence of sPC needs to be adjusted to local circumstances, as the RM overestimated the risk of harboring sPC in the UCLH cohort. After prevalence-adjustment with respect to the prevalence underlying ERSPC-RC3 to ensure a generalizable comparison, not only between the Heidelberg and die UCLH subgroup, the RM`s Net benefit was superior over the ERSPC`s and the mpMRI`s for threshold probabilities above 0.1 in both cohorts.
Conclusions
The RM discriminated well between men with and without sPC at initial MRI-targeted biopsy but overestimated the sPC-risk at UCLH. Taking prevalence into account, the model demonstrated benefit compared with clinical risk calculators and PI-RADSv1.0 in making the decision to biopsy men at suspicion of PC. However, prevalence differences must be taken into account when using or validating the presented risk model.
Citation: Radtke JP, Giganti F, Wiesenfarth M, Stabile A, Marenco J, Orczyk C, et al. (2019) Prediction of significant prostate cancer in biopsy-naïve men: Validation of a novel risk model combining MRI and clinical parameters and comparison to an ERSPC risk calculator and PI-RADS. PLoS ONE 14(8): e0221350. https://doi.org/10.1371/journal.pone.0221350
Editor: Pascal A. T. Baltzer, Medical University of Vienna, AUSTRIA
Received: May 3, 2019; Accepted: August 5, 2019; Published: August 26, 2019
Copyright: © 2019 Radtke et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The study group was funded by the UCL Graduate Research Scholarship (awarded to Dr Francesco Giganti), Brahm PhD scholarship in memory of Chris Adams (awarded to Dr Francesco Giganti), United Kingdom National Institute of Health Research (NIHR) Senior Investigator(awarded to Prof Mark Emberton), UCL Hospitals (awarded to Prof Mark Emberton), Profound Medical (awarded to Prof David Bonekamp), Invivo Uronav (awarded to Dr Jan Radtke), Bender Group (awarded to Dr Jan Radtke), Beckelmann and Partners (awarded to Dr Jan Radtke), and Saegeling Medizintechnik (awarded to Dr Jan Radtke). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors of this manuscript have read the journal's policy and have the following competing interests: DB is a speaker for Profound Medical Inc.. JPR is company consultant for Invivo Uronav, Bender Group, Beckelmann and Saegeling Medizintechnik. FG is funded by the UCL Graduate Research Scholarship and the Brahm PhD scholarship in memory of Chris Adams. ME is a United Kingdom National Institute of Health Research (NIHR) Senior Investigator and receives research support from UCL Hospitals–UCL NIHR Biomedical Research Centre. However this funding is not related to this work and does not alter our adherence to PLOS ONE policies on sharing data and materials.
Abbreviations: AUC, Area-under-the-curve; DCA, Decision curve analysis; DCE, Dynamic contrast-enhanced Imaging; DRE, Digital-rectal examination; ERSPC, European Randomised study of Screening for PC; FTB, MRI-fusion targeted biopsy; GS, Gleason score; log, logarithmic; mpMRI, multiparametric Magnetic Resonance Imaging; MRI, Magnetic Resonance Imaging; PC, Prostate cancer; PI-RADS, Prostate Imaging—Reporting and Data System; PSA, Prostate specific antigen; PV, Prostate volume; RC, Risk calculator; RM, Risk model; ROC, Receiver-operating-characteristics; RP, Radical prostatectomy; SB, Systematic transperineal saturation biopsy; sPC, Significant prostate cancer; STARD, Standards of Reporting of Diagnostic Accuracy; START, Standards of Reporting for MRI-targeted Biopsy Studies; TRUS, transrectal ultrasound; TV, Tumor volume; UCLH, University College London Hospital
Introduction
The decision-making process of prostate cancer (PC) diagnosis is still controversial. The traditional approach we have taken in diagnosing PC, with raised prostate specific antigen and transrectal prostate ultrasound-guided (TRUS)-biopsy, do not appear to result in overall mortality benefit yet do lead to overdiagnosis and overtreatment of indolent disease [1]. The introduction of multiparametric magnetic resonance imaging (mpMRI) of the prostate has significantly improved the diagnostic accuracy of radiology in PC [2–7]. This has led to a higher detection rate of clinically significant PC (sPC) by means of targeted biopsies [7–9]. In addition, approaches using upfront MRI and targeted biopsies help to reduce the diagnosis of clinically indolent disease [7–9].
The implementation of clinical data with mpMRI findings has become of significant importance for urologists in order to better stratify those men who may deserve (or not) a prostate biopsy [10,11]. At this regard, there has been a considerable interest in developing new clinical tools and multivariable risk calculators (RC) and models (RM) able to predict the probability of a patient to harbour sPC, including data from mpMRI [10–15]. There is evidence suggesting that the addition of mpMRI findings to clinical information could increase the accuracy of diagnosis, but only few of these models have been either internally or externally validated [10,11,15]. Therefore, there is a need of further robust studies to develop more accurate RMs [16].
The purpose of this study was to validate a previously published RM predicting the probability of harbouring clinical sPC based on clinical parameters and mpMRI features focusing on cohorts of biopsy-naïve men undergoing mpMRI and subsequent prostate biopsy [10].
Material and methods
Study population
The study population comprised 160 consecutive biopsy-naïve patients from University Hospital Heidelberg and 139 men from University College London Hospital (UCLH). At Heidelberg, patients were enrolled and registered into a prospective database assessing MRI-targeted/TRUS-fusion biopsy between 2012 and 2017. Institutional review board approval was obtained (S011/2011) and all subjects provided written informed consent. Subgroups were previously reported [10,17]. The original RM-development contained data of patients undergoing MRI and fusion-biopsy between 2012–2015 [10]. Data of the 160 consecutive subjects in the present study, who underwent MRI and fusion-biopsy from 2016–2017, were not used in original RM-development. At UCLH, institutional review board exemption was granted from the local joint research office. The subgroup from UCLH consisted of 139 consecutive men undergoing transperineal prostate biopsy agreed to be sampled with additional targeted biopsy directed to suspicious lesions on prostate mpMRI from 2010–2012.
Data were retrospectively analysed. Inclusion criteria were suspicion of PC based on prostate specific antigen (PSA) or digital rectal examination (DRE), mpMRI with Prostate Imaging Reporting and Data System (PI-RADS) version 1.0 scoring according to the European Society of Urogenital Radiology guidelines and fusion-biopsy [18]. Men under active surveillance and men who had missing data were excluded (S1 Fig). Of all 299 biopsy-naive men, full data on PI-RADSv1.0, biopsy-outcome, PSA, age, DRE, prostate volume (PV), PSA-density, lesions on TRUS and the European Randomised Study of Screening for Prostate Cancer Risk calculator (ERSPC-RC) 3 were available for 293 men (160 from Heidelberg and 133 from UCLH). Those samples served for validation and for comparisons to ERPSC-RC3, PI-RADSv1.0 and combined ERSPC-RC3 and PI-RADSv1.0.
Imaging
All mpMRI examinations were performed using a 3-Tesla-system (Magnetom, Siemens, Erlangen, Germany) using a multichannel-body-surface coil at Heidelberg and two scanners (one 1.5 Tesla- and one 3-Tesla-scanner (Avanto and Verio, Siemens, Erlangen, Germany) at UCLH, as previously reported [10,19]. In Heidelberg, all image analyses were prospectively performed according to PI-RADSv1.0 by or under supervision of expert uroradiologists (HPS, DB, MCR, 7–18 years of experience in prostate-MRI). In this scoring system every parameter T2WI, DWI and DCE is scored on a five-point scale separately. Additionally, each lesion is given an overall score, to predict its chance of being a clinically sPC. According to the PI-RADS guidelines for Version 1, no dominant sequence for different zones in which the lesion occur, as e.g. DWI for peripheral zone in Version 2, were applied.
For the UCLH cohort, each scan was retrospectively reported by a radiologist highly-experienced in prostate-MRI according to PI-RADSv1.0 (FG, 5 years of experience) [18]. This was done to make results from the two subgroups comparable, as the original prospective MR reading at UCLH was performed according to a 5-point Likert scale, but not PI-RADSv1.0. To detect potential differences or drawbacks of the RM using PI-RADSv1.0, all scans were retrospectively reported using PI-RADSv2.0.
Biopsy protocol
Transperineal grid-directed biopsy performed under general anaesthesia is the standard technique at both sites, whose sPC-detection accuracy has been validated [17,19]. At Heidelberg, all men underwent transperineal fusion-targeted biopsy (FTB) with rigid (BiopSee, MedCom, Darmstadt, Germany) or elastic (Uronav, Invivo, Gainesville, USA) software-registration of MRI-suspicious lesions first (2–5 cores, median 2 per lesion) and then standard biopsy (SB) adjusted to PV (median 24 cores, according to the Ginsburg scheme) [17]. At UCLH, transperineal cognitive targeted biopsies were performed using the Hitachi Preirus (Hitachi Aloka Medical, Wallingford, USA) first, followed by SB using the Barzell 20-sector scheme [20]. Thus, both SB techniques were performed transperineally with an extended number of cores.
Histopathology
Histopathological analyses were performed by or under supervision of uropathologists specialized in prostate assessment according to International Society of Urological Pathology standards. sPC was defined as Gleason score (GS)≥3+4 (WR, AS, AF).
Assessment of the ERSPC-RC3
The ERSPC-RCs (www.prostatecancer-riskcalculator.com) are prediction models based on data of men in the ERSPC Rotterdam [21]. ERSPC-RC3 uses TRUS lesions, DRE, TRUS-measured PV and PSA. RC3 calculates the risk of finding any higher-grade (GS≥3+4) and/or locally advanced (T-stage≥T2c) PC in a conventional random-biopsy for men who have never been screened [21]. We retrospectively analysed ERSPC-RC3 on both subcohorts. ERSPC-RC3 were calculated manually per single patient using the original online RCs. These ERSPC-RCs were also used to combine ERSPC and PI-RADSv1.0 (ERSPC-R3+PI-RADSv1.0).
Statistical analysis
Patient demographics, MRI and biopsy results were analysed descriptively, according to START recommendations (Table 1) [22]. Differences of the subcohorts of both sites were detailed (Table 2). For further information on how the RM was derived, see the original manuscript [10]. The regression equations were as follows:
Patients’ demographics including baseline clinical parameters, MRI and MRI/TRUS-fusion biopsy results according to START criteria.
Differences in subgroups of UCLH and Heidelberg University Hospital for patient demographics, mpMRI PI-RADS scoring and biopsy results.
Discrimination
Discrimination of ERSPC-RC3, PI-RADSv1.0, ERSPC-RC3+PI-RADSv1.0 and the RM were compared using Area-under-the-curve (AUC) of receiver-operating-characteristic (ROC) curve analysis (Fig 1). Statistical differences between predictive models were analysed using DeLong`s test and Holm adjustment for multiple testing (Table 3).
ROC curve analysis for the performance of mpMRI PI-RADSv1.0 (green line), ERSPC-RC3 (pink line), ERSPC-RC3+mpMRI PI-RADSv1.0 (purple line) and the risk model (orange line) to predict sPC for a) Heidelberg validation cohort, b) UCLH validation cohort. AUCs are given in Table 3.
Calibration of the RM
The extent of over- or underestimation of predicted probabilities relative to observed probabilities of sPC was explored graphically using calibration plots for both subcohorts. The calibration plot demonstrated that the RM overestimates event probabilities at UCLH (Fig 2). This seems to be mainly due to major differences between the prevalences of sPC in the Heidelberg training cohort and the UCLH validation cohort (46% and 29%, respectively). It is well known that one has to be cautious about extrapolating risk models built on populations very different from the population of interest [23]. Although the prevalence in the general population is comparable in Germany and the United Kingdom, the prevalence in academic centers might be variable. Therefore we thought to update our RM in terms of prevalence-adjustment and give both, the original and the adjusted risk model.
a) Calibration plots for the Heidelberg validation cohort, b) Calibration plots for the UCLH validation cohort.
Adjustment procedure for different prevalences
A modification of the nomogram to the present situation is to use the prior correction strategy described e.g. in Prentice and Pyke [24] and Janssen et al. [25].
Thereby, we correct the intercept of the RM by subtracting the log odds of the prevalence in the development cohort (0.456) and adding the log odds of the prevalence in the ERSPC-RC3 cohort (0.245) to provide equal calibration-in-the-large of the two prediction models:
logit (π_i) = -3.623 –logit(0.456) + logit(0.245) +0.550 log (PSA_i)-0.020 volume_i+ 0.034 age_i+0.712 I(DRE_i≥cT2)+0.299 I(PIRADS_i = 2)+1.198 I(PIRADS_i = 3)+1.841 I(PIRADS_i = 4)+ 3.094 I(PIRADS_i = 5)
with predictors PSA, prostate volume (in ml), DRE results, age (in years) and PI-RADSv1.0 [10].
However, note that this requires assumptions about the distribution of the population characteristics that might be critical. The resulting RM shifts the probabilities of sPC while keeping the scale fixed (the odds ratios of the predictors remain the same), thus only the last line of the nomogram is changed. This implies that discrimination performance is not affected.
Decision curve analyses
Last, we assessed the clinical usefulness of the original and the prevalence-adjusted RM by using decision curve analysis (DCA)(Fig 3) [26]. We compared the RM and ERSPC regarding its clinical usefulness on DCA, performing a recalibration with respect to the ERSPC prevalence (24.5%), as mentioned before (Fig 3C and 3D).
Net decision curve analyses demonstrating the benefit for predicting sPC on biopsy: a) for the unadjusted RM in the Heidelberg cohort, b) for the unadjusted model in the UCLH cohort, c) for the adjusted model in the Heidelberg cohort (according to the prevalence of the ERSPC 24.5%) and d) for the adjusted model in the UCLH cohort. The black line is the net benefit of providing all patients with MRI/TRUS-fusion biopsy and the horizontal green line is the net benefit of providing no patients with biopsy. The net benefit provided by each prediction tool is given (pink line for ERSPC-RC3, green line for mpMRI PI-RADSV1.0, purple for ERSPC RC3+mpMRI PI-RADSv1.0 and orange line for the RM).
Statistical analyses were performed using R version 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria), packages ModelGood, DCA and rms [26]. Reporting followed Standards of Reporting of Diagnostic Accuracy (STARD) guidelines (S1 Table)[27].
Results
In total 293 men underwent mpMRI and subsequent fusion-biopsy. Of them, 160 underwent procedures in Heidelberg und 133 men at UCLH. Patient demographics, MRI and biopsy data are given in Table 1. sPC was detected in total 114 patients (39%). Stratification into centers revealed sPC in 76 men (48%) in the Heidelberg and 38 (29%) men in the UCLH cohort.
Differences on patients’ demographics, mpMRI and biopsy outcomes between the both subcohorts are given in Table 2.
The discrimination of the RM was compared to ERSPC-RC3, PI-RADSv1.0 and ERSPC-RC3+PI-RADSv1.0 using ROC curve analyses (Fig 1, Table 3). In the Heidelberg validation cohort, the RM reached a higher AUC (0.86), compared to ERSPC-RC3 (0.77), and PI-RADSv1.0 (0.79; section a of Fig 1, section a of Table 3). The RMs AUC was comparable to ERSPC-RC3+PI-RADSv1.0 (0.84). At UCLH, the discrimination of the RM (0.86) was superior to ERSPC-RC3 (0.77) and comparable to PI-RADSv1.0 (0.82) and ERSPC-RC3+PI-RADSv1.0 (0.82)(Table 3).
Calibration plots of the RM (Fig 2A and 2B) demonstrate that there are no untoward deviations of predicted from observed risk of sPC over the entire range in the Heidelberg subgroup. At UCLH, the calibration plot demonstrated systematic overestimation of the sPC probability for both, the RM as well as ERSPC-RC3 (Fig 2B), but for ERSPC-RC3 to a much smaller extend.
In adjusted (with respect to the original ERSPC prevalence of 24.5% of sPC) DCA the RM had a higher net benefit in terms of accurately detecting patients with sPC compared to mpMRI and the ERSPC-RC (Fig 3C and 3D) [21]. The RM showed a benefit for sPC threshold probabilities above 10% at both centres (Fig 3C and 3D).
Discussion
The results of our study showed that the novel RM, combining clinical and MRI parameters discriminates well in both, a consecutive validation cohort of the same institution on whose patients the RM was developed, and an external cohort. The discrimination performance of the RM was similar in both cohorts and significantly better than the performance of ERSPC-RC3. This is in line with recent results demonstrating added benefit of MRI in combination with clinical parameters [11,15].
Second, this study shows that the present RM performs well in two different cohorts using different MRI scanners and radiologists with different experience in prostate-MRI. Our results furtherly support that standardized mpMRI reading is reliable not only within a nomogram combining MRI and clinical parameters, but also as a standalone screening test. The discrimination performance of expert MRI reading was comparable, with an AUC of 0.82 at UCLH and 0.79 in Heidelberg. However, DeLong`s test demonstrated that adding clinical parameters within the RM significantly enhances the accuracy of MRI alone to predict sPC in Heidelberg, but not at UCLH.
A third point is the comparison to the clinical ERSPC-RC. Additional benefit of adding MRI to ERSPC-RC3 has recently been shown in a multicentric study, with comparable results [15]. Even if the novel RM outperforms the ERSPC-RC alone, our study could demonstrate that the ERSPC-RC3 alone has a good discrimination performance. Considering that Poyet et al. recently demonstrated that the performance of a nomogram depends on the number of biopsy cores and decreases with the increase of the cores taken, we could support the generalizability of the ERSPC-RC3, which was externally validated with an accurate AUC of 0.77 in both cohorts using extended biopsy protocols [21,28].
Additionally, in the light of the results of Poyet et al., who could demonstrate that an increased number of cores taken (12 core compared to 6–8 core) negatively influenced the performance of a validated risk model, the use of a robust reference test, combining extended SB and FTB with a median cores of 27, as in the present study, emphasizes the discrimination performance of the novel RM in the ROC curve analyses [28]. However, regarding the clinical applicability of the RM, we have to acknowledge that the RM systematically overestimated the sPC probability in a subgroup with lower prevalence (UCLH subgroup) than in the cohort on which the RM was developed (Heidelberg subgroup). Thus, the clinical applicability of the RM in the external validation needs to be discussed. In particular, this study emphasizes the prevalence-dependence of risk models.
To analyse the real net benefit of a novel RM, it has to be adjusted according to the actual prevalence of the considered population. This is critical, because one has to assume the prevalence of the cohort, as the prevalence is unknown in most cases. This is important, if one wants to extrapolate the RM and consecutively to introduce it in common clinical practice.
However, this is a general problem in any risk model. We acknowledge that any risk model is only applicable on comparable populations and extrapolation to populations with different characteristics is problematic. Alternatively, when different prevalences occur, the intercept has to be adjusted for correct risk assessment. However, we emphasized that this does not affect discrimination performance in external validation. Moreover this is a reflection to a general problem of risk modeling.
Furthermore, in clinical practice, both physicians and different patients have different trade-offs of individuals`sPC risk. Therefore there are no single risk thresholds and DCA is the most informative tool to weigh the relative harm of potentially unnecessary biopsy and the benefit of diagnosing sPC.
In our study, the discrimination performance of the original RM, analyzed by ROC curve analyses (Fig 1, Table 3) was good in both, a cohort that is comparable in terms of prevalence to the cohort in which the model has been derived (Heidelberg), and also in a cohort, in which the prevalence is different (UCLH). It is well known that ROC curve analyses are not affected by prevalences. However, individuals`predicted probabilities of sPC are trivially affected by the prevalences. The predicted and the empirical sPC probability was visualized in calibration curves, demonstrating that the original model overestimated the probabilities due to prevalence differences (Fig 2B). In decision curve analyses the net benefit of the original model was therefore limited in the cohort in which the prevalence is smaller, because the patients`individual probability is overestimated (Fig 3B). After adjustment of the RM such that prevalences underlying the RM and ERSPC-RC are equivalent has been performed, as in the Material and methods section, the overestimation is extenuated (compare Fig 2B, in which the upper curve gives the calibration of the original model as compared to the adjusted one in the lower curve). This also affects the decision curve analysis, demonstrating improved benefit of the adjusted RM over a threshold probability from 0.10 to approximately 0.50 (Fig 3D).
Our study has limitations. First, as mentioned before, prevalence-dependence of the RM limits its generalizability. In order to correctly determine the individuals`risk of harbouring sPC, it is mandatory to be aware of the sPC prevalence in the current population or to assume this to adjust the RMs`intercept.
Both cohorts used a similar biopsy approach including MRI/TRUS-fusion and template SB with an extended number of biopsy cores. Therefore, we cannot clearly state how the RM performs on cohorts using different biopsy strategies (e.g. using widespread standard 12-core TRUS-biopsy). However, the application of a robust reference test is a strength of the study, and the relative poor performance of the 12-core TRUS-biopsy has been recently confirmed [3,4]. One might assume that differences occur due to different fusion-biopsy techniques. With regards to the biopsy approach the available data in the literature is rather scarce. The latest review and clinical trials were not able to show significant differences between cognitive and software-assisted biopsy for MRI-targeted cores, but one of the limitations given were limited amounts of studies or patients, respectively [29–32]. Additionally, Wysock et al. compared MRI/TRUS-fusion-guided biopsies versus visual estimated targeting in a prospective study including 125 men with suspicious lesions [33]. They found that MRI/TRUS-fusion-guided biopsies had a slightly improved cancer detection rate compared to visual estimation for GS≥3+4 (20% versus 15%, p = 0.05). Puech et al. observed no difference in the cancer detection rate of PC for rigid software co-registration using software-guided compared to cognitive TB (53% versus 47%)[34]. However, Delongchamps et al. reported that cognitive fusion-biopsy was not significantly better than systematic biopsies, while both software co-registration devices tested (Esaote/MyLabTMTwice and Koelis/Urostation) significantly increased the cancer detection rate compared to systematic biopsies using logistic regression analysis in a cohort of 391 patients. In our study, the additional systematic biopsy part of the biopsy procedure was comparable in both centers, using a transperineal extended SB scheme with a median of 24 cores. In a subanalysis (data not shown) we could demonstrate that in the Heidelberg subgroup the software-assisted approach missed 4% of sPC as compared to the systematic saturation biopsy, whereas the cognitive-fusion biopsy approach at UCLH missed 9% of sPC. Therefore, the differences in prevalence are not fully explainable by different TB approaches.
We prospectively used PI-RADSv1.0, while v2.0 has recently become the favoured approach, since the time period for data accrual predominantly was before publication of v2.0. As the original RM was constructed using v1.0, the retrospective MR reading at UCLH also used v1.0. However, data are still not fully decisive as to the advantage of either over the other version. In a recent meta-analysis, a higher pooled sensitivity (0.95) for v2.0 compared to v1.0 (0.88) but a comparable pooled specificity (0.73 versus 0.75) were detected [35]. Contrary, Auer found a significantly larger discrimination for PI-RADSv1.0 (0.96) versus v2.0 (0.90) [36]. However, utilization of PI-RADSv1.0 is a common problem of recent RMs, and data on the impact of v1.0 versus v2.0 on RMs are lacking [10,15]. When retrospectively reading MR scans according to PI-RADSv2.0, we could demonstrate the discrimination performance of PI-RADSv2.0 in the UCLH cohort was higher than for PI-RADSv1.0 (AUC 0.89), but slightly lower in the Heidelberg subgroup (AUC 0.75). In addition, the RMs performance at UCLH would be improved when replacing PI-RADSv1.0 by v2.0 (AUC 0.89). Results for ROC curve analyses and Decision curve analyses are given in S2 Fig. However, we did not develop a completely novel RM, as the purpose of our study was to validate an existing one. Additionally, when all available risk models combining clinical and MR parameters are analysed, the predictive performance of standardized MR reading is comparable when using PI-RADSv1.0 in the studies of Alberts et al., van Leeuwen et al. and the original RM, or PI-RADSv2.0 in the study of Mehralivand et al. [10,11,15,37]. All risk models enhance the discrimination compared to MRI scoring and clinical parameters alone. Further investigations are needed to investigate the performance of a novel RM including PI-RADSv2.0.
In addition, costs and capacity of MRI as an upfront test have to be considered. However, the cost-effectiveness of using upfront mpMRI has been demonstrated recently [38].
The sample size of this study population is limited, in particular in the UCLH cohort. This might also affect the applicability of both, the RM and the study`s results. However, by analysing subcohorts of consecutive men in both centers, potential bias by patient selection should be decreased.
Lastly, our sPC definition of GS≥3+4 is debatable. However, to compare our results with ERSPC-RC3, GS≥3+4 was appropriate. We acknowledge that the comparison to ERSPC-RCs is not perfect, since our sPC definition did not include clinical T-staging [21]. Additionally, tumor involved core length / cancer core length had not available. Thus we could not include it into the definition of sPC.
The conclusion of this study is that the novel RM can improve the prediction of harboring sPC. If prevalence is properly acknowledged, the RM provides benefit in the decision-making process of which patient should be biopsied over clinical risk modeling and imaging alone.
Supporting information
S1 Table. Standards of Reporting of Diagnostic Accuracy (STARD) checklist.
https://doi.org/10.1371/journal.pone.0221350.s001
(DOCX)
S1 Fig. Study flowchart with inclusion criteria.
https://doi.org/10.1371/journal.pone.0221350.s003
(JPG)
S2 Fig.
ROC curve analysis (a and b) and Net decision curve analysis (c and d) for the performance of mpMRI PI-RADSv1.0 (green line), ERSPC-RC3 (pink line), ERSPC-RC3+mpMRI PI-RADSv1.0 (purple line), the risk model (orange line), mpMRI PI-RADSv2.0 (grey line), ERSPC-RC3+mpMRI PI-RADSv2.0 (brown line) and the risk model with mpMRI PI-RADSv2.0 (yellow line) for Heidelberg validation cohort and UCLH validation cohort. On Net decision curve analysis, the black line is the net benefit of providing all patients with MRI/TRUS-fusion biopsy and the horizontal green line is the net benefit of providing no patients with biopsy. The net benefit provided by each prediction tool is given.
https://doi.org/10.1371/journal.pone.0221350.s004
(PNG)
References
- 1. Schröder FH, Hugosson J, Roobol MJ, Tammela TLJ, Zappa M, Nelen V, et al. Screening and prostate cancer mortality: results of the European Randomised Study of Screening for Prostate Cancer (ERSPC) at 13 years of follow-up. Lancet. 2014;384: 2027–35. pmid:25108889
- 2. Fütterer JJ, Briganti A, De Visschere P, Emberton M, Giannarini G, Kirkham A, et al. Can Clinically Significant Prostate Cancer Be Detected with Multiparametric Magnetic Resonance Imaging? A Systematic Review of the Literature. Eur Urol. 2015; 1–9. pmid:25656808
- 3. Kasivisvanathan V, Rannikko AS, Borghi M, Panebianco V, Mynderse LA, Vaarala MH, et al. MRI-Targeted or Standard Biopsy for Prostate-Cancer Diagnosis. N Engl J Med. 2018; NEJMoa1801993. pmid:29552975
- 4. Ahmed HU, El-Shater Bosaily A, Brown LC, Gabe R, Kaplan R, Parmar MK, et al. Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. Lancet. The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY license; 2017;6736: 32401–1.
- 5. van der Leest M, Cornel E, Israël B, Hendriks R, Padhani AR, Hoogenboom M, et al. Head-to-head Comparison of Transrectal Ultrasound-guided Prostate Biopsy Versus Multiparametric Prostate Resonance Imaging with Subsequent Magnetic Resonance-guided Biopsy in Biopsy-naïve Men with Elevated Prostate-specific Antigen: A Large Prospective Mu. Eur Urol. 2018; pmid:30477981
- 6. Drost FJ, Osses DF, Nieboer D, Bangma CH, Steyerberg EW, Roobol MJ, et al. Prostate MRI, with or without targeted biopsy and standard biopsy for detecting prostate cancer: A Cochrane systematic review and meta-analysis. Cochrane Database Syst Rev. 2019; No.: CD012663.
- 7. Schoots IG, Roobol MJ, Nieboer D, Bangma CH, Steyerberg EW, Hunink MGM. Magnetic Resonance Imaging–targeted Biopsy May Enhance the Diagnostic Accuracy of Significant Prostate Cancer Detection Compared to Standard Transrectal Ultrasound-guided Biopsy: A Systematic Review and Meta-analysis. Eur Urol. European Association of Urology; 2014; 1–13. pmid:25480312
- 8. Valerio M, Donaldson I, Emberton M, Ehdaie B, Hadaschik BA, Marks LS, et al. Detection of clinically significant prostate cancer using magnetic resonance imaging-ultrasound fusion targeted biopsy: A systematic review. Eur Urol. European Association of Urology; 2015;68: 8–19. pmid:25454618
- 9. Moore CM, Robertson NL, Arsanious N, Middleton T, Villers A, Klotz L, et al. Image-guided prostate biopsy using magnetic resonance imaging-derived targets: a systematic review. Eur Urol. 2013;63: 125–40. pmid:22743165
- 10. Radtke JP, Wiesenfarth M, Kesch C, Freitag MT, Alt CD, Celik K, et al. Combined Clinical Parameters and Multiparametric Magnetic Resonance Imaging for Advanced Risk Modeling of Prostate Cancer—Patient-tailored Risk Stratification Can Reduce Unnecessary Biopsies. Eur Urol. 2017;72. pmid:28400169
- 11. Mehralivand S, Shih JH, Rais–Bahrami S, Oto A, Bednarova S, Nix JW, et al. A Magnetic Resonance Imaging–Based Prediction Model for Prostate Biopsy Risk Stratification. JAMA Oncol. 2018;20892: 1–8.
- 12. Bjurlin MA, Rosenkrantz AB, Sarkar S, Lepor H, Huang WC, Huang R, et al. Prediction of Prostate Cancer Risk among Men Undergoing Combined MRI-Targeted and Systematic Biopsy Using Novel Pre-Biopsy Nomograms That Incorporate MRI Findings. Urology. Elsevier Inc.; 2017; pmid:29155186
- 13. Kim EH, Weaver JK, Shetty AS, Vetter JM, Andriole GL, Strope SA. Magnetic Resonance Imaging Provides Added Value to the Prostate Cancer Prevention Trial Risk Calculator for Patients With Estimated Risk of High-grade Prostate Cancer Less Than or Equal to 10%. Urology. 2017;102: 183–189. pmid:27919668
- 14. Dwivedi DK, Kumar R, Dwivedi AK, Bora GS, Thulkar S, Sharma S, et al. Prebiopsy multiparametric MRI-based risk score for predicting prostate cancer in biopsy-naive men with prostate-specific antigen between 4-10ng/mL. J Magn Reson Imaging. 2017; pmid:28872226
- 15. Alberts AR, Roobol MJ, Verbeek JFM, Schoots IG, Chiu PK, Osses DF, et al. Prediction of High-grade Prostate Cancer Following Multiparametric Magnetic Resonance Imaging: Improving the Rotterdam European Randomized Study of Screening for Prostate Cancer Risk Calculators. Eur Urol. 2018; pmid:30082150
- 16. Pavlou M, Ambler G, Seaman SR, Guttmann O, Elliott P, King M, et al. How to develop a more accurate risk prediction model when there are few events. BMJ. 2015;351: h3868. pmid:26264962
- 17. Radtke JP, Schwab C, Wolf MB, Freitag MT, Alt CD, Kesch C, et al. Multiparametric Magnetic Resonance Imaging (MRI) and MRI–Transrectal Ultrasound Fusion Biopsy for Index Tumor Detection: Correlation with Radical Prostatectomy Specimen. Eur Urol. European Association of Urology; 2016;70: 846–853. pmid:26810346
- 18. Barentsz JO, Richenberg J, Clements R, Choyke P, Verma S, Villeirs G, et al. ESUR prostate MR guidelines 2012. Eur Radiol. 2012;22: 746–757. pmid:22322308
- 19. Simmons LAM, Kanthabalan A, Arya M, Briggs T, Barratt D, Charman SC, et al. The PICTURE study: diagnostic accuracy of multiparametric MRI in men requiring a repeat prostate biopsy. Br J Cancer. 2017;116: 1159–1165. pmid:28350785
- 20. Kasivisvanathan V, Dufour R, Moore CM, Ahmed HU, Abd-Alazeez M, Charman SC, et al. Transperineal magnetic resonance image targeted prostate biopsy versus transperineal template prostate biopsy in the detection of clinically significant prostate cancer. J Urol. 2012/10/16. 2013;189: 860–866. pmid:23063807
- 21. Roobol MJ, van Vugt H a., Loeb S, Zhu X, Bul M, Bangma CH, et al. Prediction of Prostate Cancer Risk: The Role of Prostate Volume and Digital Rectal Examination in the ERSPC Risk Calculators. Eur Urol. 2012;61: 577–583. pmid:22104592
- 22. Moore CM, Kasivisvanathan V, Eggener S, Emberton M, Futterer JJ, Gill IS, et al. Standards of reporting for MRI-targeted biopsy studies (START) of the prostate: recommendations from an International Working Group. Eur Urol. 2013/03/30. 2013;64: 544–552. pmid:23537686
- 23. Iasonos A, Schrag D, Raj G V, Panageas K. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26: 1364–1370. pmid:18323559
- 24. Prentice RL, Pyke R. Logistic disease incidence models and case-control studies. Biometrika. 1979;66: 403–411.
- 25. Janssen KJM, Vergouwe Y, Kalkman CJ, Grobbee DE, Moons KGM. A simple method to adjust clinical prediction models to local circumstances. Can J Anesth. 2009;56: 194–201. pmid:19247740
- 26. Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8: 53. pmid:19036144
- 27. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis C a, Glasziou PP, Irwig LM, et al. Towards complete and,accurate reporting of studies of diagnostic accuracy: the STARD initiative. Radiology. 2003;226: 24–8. pmid:12511664
- 28. Poyet C, Nieboer D, Bhindi B, Kulkarni GS, Wiederkehr C, Wettstein MS, et al. Prostate cancer risk prediction using the novel versions of the European Randomised Study for Screening of Prostate Cancer (ERSPC) and Prostate Cancer Prevention Trial (PCPT) risk calculators: independent validation and comparison in a contemporary Europe. BJU Int. 2015;117: 401–8. pmid:26332503
- 29. Wegelin O, Exterkate L, van der Leest M, Kummer JA, Vreuls W, de Bruin PC, et al. The FUTURE Trial: A Multicenter Randomised Controlled Trial on Target Biopsy Techniques Based on Magnetic Resonance Imaging in the Diagnosis of Prostate Cancer in Patients with Prior Negative Biopsies. Eur Urol. European Association of Urology; 2019;75: 582–590. pmid:30522912
- 30. Wegelin O, van Melick HHE, Hooft L, Bosch JLHR, Reitsma HB, Barentsz JO, et al. Comparing Three Different Techniques for Magnetic Resonance Imaging-targeted Prostate Biopsies: A Systematic Review of In-bore versus Magnetic Resonance Imaging-transrectal Ultrasound fusion versus Cognitive Registration. Is There a Preferred Technique? Eur Urol. European Association of Urology; 2017;71: 517–531. pmid:27568655
- 31. Yaxley AJ, Yaxley JW, Thangasamy IA, Ballard E, Pokorny MR. Comparison between target magnetic resonance imaging (MRI) in-gantry and cognitively directed transperineal or transrectal-guided prostate biopsies for Prostate Imaging-Reporting and Data System (PI-RADS) 3–5 MRI lesions. BJU Int. 2017; 43–50. pmid:28749035
- 32. Hamid S, Donaldson IA, Hu Y, Rodell R, Villarini B, Bonmati E, et al. The SmartTarget Biopsy Trial: A Prospective, Within-person Randomised, Blinded Trial Comparing the Accuracy of Visual-registration and Magnetic Resonance Imaging/Ultrasound Image-fusion Targeted Biopsies for Prostate Cancer Risk Stratification. Eur Urol. 2019; pmid:30527787
- 33. Wysock JS, Rosenkrantz AB, Huang WC, Stifelman MD, Lepor H, Deng F-M, et al. A Prospective, Blinded Comparison of Magnetic Resonance (MR) Imaging-Ultrasound Fusion and Visual Estimation in the Performance of MR-targeted Prostate Biopsy: The PROFUS Trial. Eur Urol. European Association of Urology; 2013;66: 343–351.
- 34. Puech P, Rouviere O, Renard-Penna R, Villers A, Devos P, Colombel M, et al. Prostate cancer diagnosis: multiparametric MR-targeted biopsy with cognitive and transrectal US-MR fusion guidance versus systematic biopsy—prospective multicenter study. Radiology. 2013/04/13. 2013;268: 461–469. pmid:23579051
- 35. Woo S, Hyun C, Youn S, Yeon J, Hyup S, Novara G. Diagnostic Performance of Prostate Imaging Reporting and Data System Version 2 for Detection of Prostate Cancer: A Systematic Review and Diagnostic Meta-analysis. Eur Urol. European Association of Urology; 2017; pmid:28196723
- 36. Auer T, Edlinger M, Bektic J, Nagele U, Herrmann T, Schäfer G, et al. Performance of PI-RADS version 1 versus version 2 regarding the relation with histopathological results. World J Urol. 2016;doi:10.100
- 37. van Leeuwen PJ, Hayen A, Thompson JE, Moses D, Shnier R, Böhm M, et al. A multiparametric magnetic resonance imaging-based risk model to determine the risk of significant prostate cancer prior to biopsy. BJU Int. 2017; n/a-n/a. pmid:28207981
- 38. Faria R, Soares MO, Spackman E, Ahmed HU, Brown LC, Kaplan R, et al. Optimising the Diagnosis of Prostate Cancer in the Era of Multiparametric Magnetic Resonance Imaging: A Cost-effectiveness Analysis Based on the Prostate MR Imaging Study (PROMIS). Eur Urol. European Association of Urology; 2017;10.1016/j. pmid:28935163