Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Machine Learning for Outcome Prediction of Acute Ischemic Stroke Post Intra-Arterial Therapy

  • Hamed Asadi,

    Affiliation Melbourne Brain Centre, Department of Medicine, Royal Melbourne Hospital, University of Melbourne, Parkville, Victoria, Australia

  • Richard Dowling,

    Affiliation Melbourne Brain Centre, Department of Medicine, Royal Melbourne Hospital, University of Melbourne, Parkville, Victoria, Australia

  • Bernard Yan,

    Affiliation Melbourne Brain Centre, Department of Medicine, Royal Melbourne Hospital, University of Melbourne, Parkville, Victoria, Australia

  • Peter Mitchell

    peter.mitchell@mh.org.au

    Affiliation Melbourne Brain Centre, Department of Medicine, Royal Melbourne Hospital, University of Melbourne, Parkville, Victoria, Australia

Abstract

Introduction

Stroke is a major cause of death and disability. Accurately predicting stroke outcome from a set of predictive variables may identify high-risk patients and guide treatment approaches, leading to decreased morbidity. Logistic regression models allow for the identification and validation of predictive variables. However, advanced machine learning algorithms offer an alternative, in particular, for large-scale multi-institutional data, with the advantage of easily incorporating newly available data to improve prediction performance. Our aim was to design and compare different machine learning methods, capable of predicting the outcome of endovascular intervention in acute anterior circulation ischaemic stroke.

Method

We conducted a retrospective study of a prospectively collected database of acute ischaemic stroke treated by endovascular intervention. Using SPSS®, MATLAB®, and Rapidminer®, classical statistics as well as artificial neural network and support vector algorithms were applied to design a supervised machine capable of classifying these predictors into potential good and poor outcomes. These algorithms were trained, validated and tested using randomly divided data.

Results

We included 107 consecutive acute anterior circulation ischaemic stroke patients treated by endovascular technique. Sixty-six were male and the mean age of 65.3. All the available demographic, procedural and clinical factors were included into the models. The final confusion matrix of the neural network, demonstrated an overall congruency of ∼80% between the target and output classes, with favourable receiving operative characteristics. However, after optimisation, the support vector machine had a relatively better performance, with a root mean squared error of 2.064 (SD: ±0.408).

Discussion

We showed promising accuracy of outcome prediction, using supervised machine learning algorithms, with potential for incorporation of larger multicenter datasets, likely further improving prediction. Finally, we propose that a robust machine learning system can potentially optimise the selection process for endovascular versus medical treatment in the management of acute stroke.

Introduction

Stroke and Endovascular Treatment

Stroke is a major global public health issue and is considered the third most costly health condition in developed countries [1]. Approximately 800,000 cases of stroke are reported in United States of America per annum, leading to 200,000 deaths, almost one of every 16 deaths [2], [3]. For those who survive, it is the most common cause of adult disability in the modern world [2], [4], requiring expensive long term rehabilitation care[2], [5][7] amounting to costs estimated at over 60 billion dollars per year in the United States of America alone [2], [5], [8]. More than 80% of stroke cases are ischaemic, with the remainder being haemorrhagic [2].

Urgent reperfusion of the ischemic brain is the primary treatment aim, either by intravenous thrombolysis or by endovascular interventional techniques [9]. These treatments focus on vascular recanalisation and restoration of blood flow to the ischemic tissue [10]. Although there are varying estimates to the potential number of patients who may benefit from endovascular intervention, there will likely be expansion of the number of patients treated using these techniques [2], [11], [12].

Initial focus was on intraarterial thrombolysis, proposed to be safe up to 6 hours post ictus in Pro-Urokinase for Acute Cerebral Thromboembolism II (PROACT-II) trial [1], [13]; however, rapid mechanical clot extraction with decreased time to cerebral reperfusion has obvious appeal and in fact is theoretically ideal for platelet poor, fibrin rich, well organized cardiogenic emboli, refractory to mechanical lysis2. Therefore, subsequent developments of various mechanical thrombectomy devices has gained much interest with the theoretical advantage of faster recanalisation and potential lower rate of hemorrhagic transformation; possibly leading to an extended time window in stroke intervention [1].

FDA approval for MERCI Retrieval and Penumbra Stroke Systems[2], [14][19] as the first generation of mechanical thrombectomy devices was followed by introduction of Solitaire and Trevo as stent retrievers [10]. Introduction of these devices backed up by the pioneering studies MERCI, multi-MERCI, and Penumbra, as well as Pivotal and SWIFT, have further strengthened the importance of the mechanical techniques in large vessels occlusion[10], [20][26]. However, despite more than 80% recanalisation success, randomised controlled trials such as Interventional Management of Stroke (IMS) - III [1], [27], have still failed to show a significant improvement in the clinical outcome, evaluated by 90 day modified Rankin Scale score[2], [10], [22], [28][33].

The SYNTHESIS trial also failed to show any superiority of endovascular intervention or even for combined endovascular and peripheral thrombolysis over traditional intravenous tPA[34][36]. This conundrum was made more complicated when the MR-RESCUE trial demonstrated not only that embolectomy was no better than standard care, but also a favorable penumbral pattern on imaging does not necessarily indicates patients who would benefit from endovascular therapy [37].

This discrepancy between the IMS-III, SYNTHESIS and MR-RESCUE outcomes and what may have been intuitively expected is likely related to the multiple potential pitfalls in the design of these trials which could change the interpretation of the results [1], [38], [39]. The most commonly speculated factor is that patient selection was neither targeted to those who failed IV thrombolysis, nor to those with large vessel occlusion or large clot burden ≥8 mm, who are usually not responsive to chemical treatment alone; as no vascular imaging was required prior to inclusion into the studies[1], [38][40]. These limitations could certainly influence the accuracy of the studies in evaluation of the clot retrieval techniques. On the other hand, stentretrievers, now acknowledged as more effective devices, were included only very late into the studies like IMS-III, with less than 1% of cases treated using Solitaire. Since the release of these preliminary results at least six additional devices have started premarket testing[1], [38][40].

It appears that the situation is different for posterior circulation involvement. Although causing only 6–10% of large vessel strokes, posterior circulation occlusions have a relatively different course, and failure of recanalisation, in particular in comatose patients or those with basilar trunk involvement results in a very poor prognosis [2], [41]. The BASICS (Basilar Artery International Cooperation Study) did not show a definite superiority for intraarterial intervention over intravenous thrombolysis [2], [42]; and the overall outcome is quite variable in patients who are treated with intraarterial or intravenous thrombolysis, in particular depending on the therapeutic delay [2], [43]. On the other hand, some trials have already demonstrate more than 50% recanalisation success rate for intraarterial techniques, with relatively good outcome [2], [44].

However, randomized control trials are restricted and limited by the lower incidence of posterior circulation strokes, and the results are potentially influenced by the heterogeneity of both the presentations and the causes; and at this stage the rational for aggressive treatment is mainly based on anecdotal evidence [2], [22], [23], [45].

Overall the major obstacle in endovascular intervention of the ischemic stroke is to establish a set of criteria identifying those patients who may benefit from intervention, whilst avoiding potential unwanted catastrophic treatment related complications.

There is currently level I evidence that NIHSS (National Institute of Health Stroke Score) [46], [47] is a quick and relatively simple guide to estimate the extent and the severity of a stroke, and probably correlates with the clinical outcome [46]. It is however, unable to measure the size of established infarction, separate from the salvageable parenchyma, and is therefore unable to predict potential outcome after endovascular intervention accurately. This is consistent with the well known fact that multiple factors contribute to and influence recanalisation success, including the extent and site of the vascular occlusion; and that the overall outcome also depends on patient demographic factors as well as clinical setting such as the time from onset, duration and the severity of the presenting neurological insult [10].

The complexity of the all of these factors involved, makes prediction of the final outcome difficult. On the other hand, undoubtedly, accurately predicting the outcome from a set of predictive variables is an important aspect of clinical work, which can assist in identifying high-risk patients and guide treatment approaches, thus potentially decreasing morbidity and mortality. Such a model in prediction of the outcome, not only may be crucial in prognostication, but can also have future roles in patient selection for the variety of the treatment options available and the relevant studies.

Prognostic Modeling and Machine Learning

The usual approach to analyse the stroke outcomes data is to develop logistic regression models; however, machine learning algorithms have been proposed as an alternative, in particular for large-scale multi-institutional data, with the advantage of easily incorporating newly available data to improve prediction performance [48], [49].

Machine Learning algorithms can be applied and its trained, under two major different scenarios; supervised and unsupervised. In supervised scenario the predicted outputs are known and used to train the models. In unsupervised machines, the desired output is unknown, and the objective is to discover structure in the data, not to generalise a mapping from inputs to outputs [49], [50].

Two of the most commonly used machine learning methods include artificial neural network and support vector machine. These models are trained supervised, with neural network algorithms capable of unsupervised training as well [48], [49].

Although the technical details of theses algorithms are beyond the scope of this article, a summary of them follows:

Artificial neural network is a mathematical and computational model that is inspired by the structure and functional aspects of biological neural systems [49], [50]. It consists of interconnected nodes, processing information using a connectionist computational approach. The central connectionist principle proposes that complex neurological and mental phenomena can be described by an interconnected network of simple uniform units [50], adaptively changing their structure based on external or internal information, which flows during the learning phase, forming a robust dynamic system modelling the complex relationships between inputs and outputs or patterns in data[49][51].

From the different topological types of neural networks, the commonly used feed-forward is a network where connections between the units do not form a directed cycle or loop, and the information moves in only one direction, forward, from the input nodes, through the hidden nodes to the output nodes [50]. Back propagation algorithm is a supervised learning method divided into propagation and weight update phases, which are repeated until the performance of the network is good enough, while the output values are compared with the correct answer to compute the value of some predefined error-function [49], [50]. This calculated error is then fed back through the network, adjusting the weights of each connection accordingly, in order to reduce the error function [50]. Repeating this process usually eventually converges to some state where the error of the calculations is minimised, at which point the network is considered trained for a certain target function [49].

In comparison to the artificial neural network, the support vector machine works very differently. It takes a set of input data and predicts which of the different possible classes comprises the input, making it a non-probabilistic linear classifier. A set of training data is given and marked as belonging to one of the categories. An SVM training algorithm builds a model that assigns new data into one category or the other.

The example data points are initially mapped as points in space, so that the examples of the separate categories are divided by a clear gap that is as wide as possible, and then unknown data is represented in that same space, and predicted to belong to a category based on which side of the gap they fall on[49][51]. In doing so, the algorithm constructs a hyperplane or a set of hyperplane in an infinite-dimensional space, which can be used for classification, regression, or other tasks. Intuitively, a good separation is achieved by the hyperplane that has the largest distance to the nearest training data points of any class[49][51]. This gap is called functional margin, and in general the larger the margin the lower the generalisation error of the classifier[49][51].

Whereas the original problem may be stated in a finite dimensional space, it often happens that the sets to discriminate are not linearly separable in that space. For this reason, it was proposed that the original finite-dimensional space be mapped into a much higher-dimensional space, presumably making the separation easier in that space[49][51]. To keep the computational load reasonable, the mapping is designed to ensure that dot products may be computed easily in terms of the variables in the original space, by defining them in terms of a kernel function K(x,y) selected to suit the problem. The hyperplanes in the higher dimensional space are defined as the set of points whose inner product with a vector in that space is constant[49][51].

Our Study

We aimed to design a prognostic model for the endovascular intervention in acute ischemic stroke using machine learning algorithms. We compared and assessed these two advanced methods in terms of their capability in predicting outcome.

We decided to separate anterior and posterior circulation strokes and model them independently to avoid potential inadvertent underlying inhomogeneities.

Methods

This is a retrospective study on a prospectively collected completely de-identified clinical database, which received approval from the ethics committee at our institution, and our review board has waived the need for consent (HREC: QA2011100). The technical details is provided below to facilitate reproducibility for other datasets if available.

Demographics and clinical details of 107 patients who presented with acute anterior or posterior circulation stroke to our institution who underwent endovascular treatment, over a period of approximately five years, were extracted from a prospectively maintained stroke database (Table 1).

Patients were all screened for relevant comorbidities at the time of presentation, including: diabetes mellitus, hypertension, hypercholesterolaemia, atrial fibrillation, history of ischemic heart disease and previous cerebral stroke or transient ischaemic attack. Neurological examination was performed for all of the patient prior to any intervention and the baseline National Institute of Health Stroke Scores were recorded in the database.

From the initial diagnostic angiogram occluded vessels were identified (Table 2). In the case of multiple sequential occlusions, the proximal vessel was used as a data point, and depending on the extent and segments involved, the artery was categorised as first, second and third occlusion.

Some of the patients also had IV-tPA prior to endovascular intervention. Different endovascular recanalisation devices were used, including Solitaire stent retriever, MERCI and Penumbra devices.

In addition to mechanical thrombectomy, some cases also received intraarterial chemical thrombolytic agents; and if present, associated or post-recanalisation hemodynamically significant stenoses were also treated with angioplasty or stent insertion.

After treatment of the occluded artery(s), recanalisation success was scaled using Thrombolysis in Cerebral Infarction (TICI) Score, by the blinded consensus of the treating neurointerventionalists. TICI score in conjunction with the number of attempts for recanalisation, procedure duration, and time of onset to recanalisation, as well as patient general anesthesia status, were all recorded into the database.

All procedural or delayed post-procedural complications were also recorded, including: arterial perforation and puncture site haematoma or pseudoaneurysm.

Post-procedure CT scans of the brain at 24–36 h were all assessed by neuroradiologist and neurointerventionists assessing for the presence of acute stroke and intracranial haemorrhage. Intracranial haemorrhagic transformations, were divided into clinically silent or symptomatic, and then classified into different categories (Table 3).

Procedural outcome was monitored with modified Rankin Score, measured 90 days after onset. A final dichotomised good and bad outcome was also recorded for the patients as per mRS, with less than or equal 2 considered as good.

First, using SPSS® (IBM Corporation), a Standard Linear Model was designed, using Forward-Stepwise as the model selection method, and Information Criterion (AICC) as the criteria for entry. Potential predictors of the mRS as the outcome measure was identified and a prediction model was formed and compared with the observed outcome for validation.

Supervised machine learning was then attempted. Initially using MATLAB® (MathWorks Inc.) and its Neural Network Toolbox, a two-layer Feed-Forward network with sigmoid hidden and linear output neurons, was designed.

The data was then randomly divided into 70, 15 and 15 percents subsets and the network was trained using Levenberg-Marquardt algorithm, validated and tested using the modified Rankin score as outcome; with the performance of the model monitored using Mean Squared Error. Prediction errors were also depicted on a histogram.

In addition, for comparison, the network was also trained using the dichotomised mRS, >2 or ≤2, to evaluate a binary classifier for potential good and poor outcomes.

For the seven scale mRS network, linear regressions were also performed between the observed and estimated outcome, over the training, validation and test datasets independently using Theil–Sen estimator.

However, with the dichotomised model being a binary classifier, Receiver Operating Characteristic curves were calculated to illustrate the performance of the system over each dataset as its discrimination threshold is varied. In addition, confusion matrices or contingency tables were also calculated, allowing better representation of the performance of the network.

The designed network and its calculated weighting matrix was then saved to be imported into the Simulink Toolbox of MATLAB® (MathWorks Inc.) for outcome prediction of the future data.

Subsequently to assess the capabilities of other supervised machine learning systems, the dataset with scaled and dichotomised mRS were imported into the data-mining program, Rapidminer® (Rapid-I Inc.). The filtered data was then given to the input training port of a nested cross-validation operand, with the relative number of validation of 10% and a shuffled sampling type, as well as “Leave One Out”.

The cross-validation operand consisted of two components, training and testing. The testing component contained a Support Vector Machine, with ANOVA Kernel, which is defined by raised to power “d” of summation of “exp(−g (x–y))” where “g” is gamma and “d” is the degree. “g” and “d” were set to be 1 and 2 in our machine.

The size of the cache for kernel evaluations was set to be 200 megabytes. The complexity constant (“C”) which sets the tolerance for misclassification, was set to be zero. The convergence epsilon, which is an optimizer parameter specifying the iterations stop point, was set to 0.001, with maximum iteration set to be 100000. In our machine, the loss function positive and negative complexity constant was set to 1.0. The insensitivity constant, epsilon as well as the epsilon for positive and negative deviations, were all set to be zero.

The model calculated in this machine is passed onto the testing component of the parent x-validation operand and then applied on to the test dataset.

The performance of the machine was monitored by a classic performance monitor operand and was reported as the mean squared error as well as its root. In addition, accuracy of the machine was also assessed by aggregation of a hidden confusion matrix constructed by evaluating different models on different test sets.

The designed model is finally incorporated into an apply operand ready for the prediction of the outcome of the future patients.

Results

Average of the patients’ baseline stroke score was 17.7 (SD: ±7.9).

44 of our patients also had IV-tPA prior to endovascular intervention. The remainder of the patients did not receive tPA due to a variety of contraindications.

81% of the procedures were performed under generalized anesthesia, and our three neurointerventionalists (BY, PJM and RJD) performed 24, 48, 35 cases respectively, as the primary operator.

Solitaire stent retriever was used in the majority of the cases 73, MERCI and Penumbra devices were used in 17 and 2 cases respectively. In some cases instrumentation was repeated up to 6 times to improve recanalisation.

2 cases received urokinase, 4 tissue plasmin activator, and 4 plasmin, as adjunct intraarterial chemical thrombolytic agents.

Overall recanalisation has been very successful with TICI 2b or 3 demonstrated on the final angiographic run in approximately 50% of cases. Significant associated and post-recanalisation arterial stenosis was also noted in some cases, with 23 patients requiring angioplasty, with 10 patients eventually stented.

On average procedures have taken 82.3 minutes (SD: ±39.0). The time of onset to recanalisation was on average 339.7 minutes (SD: ±91.8).

Immediate procedural complications including arterial perforation or puncture site haematoma and pseudoaneurysm were uncommon, only seen in 1 and 2 cases respectively.

31 patients were diagnosed with intracranial haemorrhage on the delayed post procedural CT, with a wide spectrum of locations and severities, from subarachnoid hemorrhage to asymptomatic or large intra-parenchymal bleedings (Table 4).

The average of mRS at 90 days was 2.57 (SD: ±2.21), with median of 2 and mode of 0 (Figure 1 & Table 5).

Standard Modelling

The information criterion and accuracy of the proposed linear model were calculated as 119.67 and 43.5% respectively (Figure 2). The most influential predictor was baseline NIHSS, with relative predictive value of 0.4 (Figure 3).

thumbnail
Figure 2. Relative importance of the model’s variables in prediction of mRS at 90 days (outcome).

https://doi.org/10.1371/journal.pone.0088225.g002

thumbnail
Figure 3. Comparison between predicted and observed outcome.

https://doi.org/10.1371/journal.pone.0088225.g003

Artificial Neural Network

The best validation performance was 6.94 and 2.98, at epoch 6 and 5 for seven scale and dichotomised mRS models respectively (Figure 4).

thumbnail
Figure 4. Network performance, for seven scale (left/top) and dichotomised mRS models (right/bottom).

Legend: Blue-Training, Green-Validation, Red-Test, Dashed Lines-Best, Vertical Axis-MSE, Horizontal Axis-Epochs.

https://doi.org/10.1371/journal.pone.0088225.g004

Gradient of 1.305×10−7 and 3.038×10−1 were calculated at epoch 12 and 11 for seven scale and dichotomised mRS models respectively.

Error histograms were calculated as the difference between the target and output which are equivalent of observed and estimated outcome, from the training, test and validation datasets, for seven scale and dichotomised mRS models (Figure 5).

thumbnail
Figure 5. Error Histogram, for seven scale (left/top) and dichotomised mRS models (right/bottom).

Legend: Blue-Training, Green-Validation, Red-Test, Orange Line-Zero Error, Vertical Axis-Instances, Horizontal Axis-Error.

https://doi.org/10.1371/journal.pone.0088225.g005

Using Theil–Sen estimator, the root of the Coefficient of Determination calculated as 0.95, 0.47 and 0.32 for each subset respectively. However, overall network estimated and observed outcome for the whole dataset, demonstrate a relatively good linear correlation with an R of 0.79 in a linear regression (Figure 6).

thumbnail
Figure 6. Linear fit between the estimated and observed outcome.

https://doi.org/10.1371/journal.pone.0088225.g006

Overall, favourable ROC curves; however, the test ROC curve is relatively poor, with the estimated area under curve (AUC) of 0.6.

The contingency table, with each column representing the instances of the predicted outcome and each row demonstrating the observed outcome, confirming acceptable model sensitivity and specify (Figure 7).

thumbnail
Figure 7. Over all ROC curve and Confusion Matrix for the dichotomised outcome network.

https://doi.org/10.1371/journal.pone.0088225.g007

Support Vector Machine

For the scaled mRS outcome, our support vector machine, had a good performance with a mean squared error of 4.489 (SD: ±2.438) and the estimated root at 2.064 (SD: ±0.480). Also, the system accuracy was assessed by “mikro”, calculated as 2.128.

On the other hand, system’s MSE and “mikro” were calculated as 0.262 (SD: ±0.068), and 0.514, respectively in prediction of the dichotomised outcome, with the precision of 80% in prediction of the poor outcome and an overall precision of 87% and model accuracy of approximately 70% (Table 6).

thumbnail
Table 6. Model precision of support vector machine in prediction of the dichotomised intervention outcome.

https://doi.org/10.1371/journal.pone.0088225.t006

Best performance of the scaled mRS model was in prediction of the patients’ prognosis with mRS of 3, with a class precision of 100% and the least precise prediction was for those patient with mRS of 6, which was only approximately 25%.

Estimated machine’s performance was improved even further when the cross validation operand set to work with “Leave Out One” sampling rather than “Shuffled”, with a MSE of 4.347 (SD: ±4.425) and rMSE of 1.809 (SD: ±1.037) for the scaled mRS outcome, and 0.286 (SD: ±0.367) and rMSE of 0.441 (SD: ±0.302) for the for the dichotomised model.

The “mikro” indicator of accuracy, was calculated as 2.085 and 0.535, with the “Leave Out One” sampling for the scaled mRS and dichotomised outcome predictor machines, respectively.

Conclusions

We showed, despite a small dataset, that there was promising accuracy, approaching 70%, of predicting outcome. There is the likely potential of further improving prediction by the incorporation of larger multicenter datasets.

There has been recent interest in adopting machine learning techniques in the prediction of the outcome of stroke patients. A recent study proposed spatial regularisation of the diffusion-weighted images acquired at the acute stage using support vector machine with a Graph encoding the voxels’ proximity, and concluded significant accuracy in prediction of the motor outcome at 90 days, showing that poor motor outcome is associated with the changes in the corticospinal bundle and white matter tracts originating from the premotor cortex [52].

Another study has proposed use of machine learning in individualised stroke treatment decision making by accurate identification of the extent of salvageable tissue on MRI in rats, based on measurement of a perfusion-diffusion mismatch and calculation of infarction probability. This study has compared generalized linear model (GLM), generalized additive model, support vector machine, adaptive boosting, and random forest; proposing that assessment of the heterogeneity of infarction probability with imaging based algorithms enables estimation of the extent of potentially salvageable tissue after acute ischemic stroke [53].

Conversely, attempts to prove the effectiveness of the invasive stroke treatments have shown inconsistent results. However, more than ever before, endovascular treatments of acute ischemic stroke are opening their way into the mainstream management of the acute stroke, in particular for those patient with contraindication for IV thrombolysis or large vessel occlusions[10], [54][58].

To our knowledge there is no comprehensive multifactorial study in humans, attempting to apply machine learning algorithms in acute ischemic stroke outcome prediction, after invasive endovascular management. Undoubtedly numerous factors, including extensive clinical features, can influence the final stroke outcome with varying significance and mechanisms, making conventional modelling challenging and perhaps inaccurate.

Machine learning models however, being, relatively independent of the unknown potential underlying interactions between these factors, are probably able to simulate the eventual result of such a complex system.

Such models can be of extreme use not only for prognostication and in predicting the outcome under different circumstances, but also in near future as an assistant in clinical decision making in particular identifying those patients who may benefit from a variety of possible treatment options, including more aggressive management, like endovascular interventions.

Limitations

Parallel to the all abovementioned advantages of the machine learning algorithms, there are important underlying assumptions and limitations that should not be forgotten. These models although can be accurate, and perhaps useful in answering the primary question, but more or less behave as a “black box” requiring large training datasets to improve their performance, with the true underlying relationships between influential factors remaining undiscovered to the user [48], [49], [51].

This inherent need for large training datasets may affect the accuracy of the machines in studies, like the current study, when only representative training data is used. In addition, with no clear understanding of the true predictors, an overcorrected conservative design may lead to the models being over-fitted by irrelevant demographics or clinical factors, thus increasing the random error and covering the desired signal with noise, a phenomenon which may explain the poor ROC curve for the test group, in this study. To avoid this, techniques like cross-validation, regularization, pruning or Bayesian model comparison, can be used to indicate the tipping point when further training no longer results in a better performance; or alternatively decision tree learning methods can be employed, providing more interpretable models [48], [49], [51].

Future Work

All of the underlying methodological and computational complexities aside, our long term goal is to design an easy to use online system, allowing for relative prediction of the clinical outcome based on the demographics and clinical findings, which can be used as a guide in making appropriate therapeutic decisions (Figure 8).

thumbnail
Figure 8. Ultimate goal is to design a system capable of proposing a dichotomised outcome for each patient with and without endovascular intervention.

https://doi.org/10.1371/journal.pone.0088225.g008

Such a system has the potential for fine adjustment from the continuous training provided via handling large-scale national or international multi-institutional users, with the advantage of easily incorporating newly available data to improve prediction performance.

Author Contributions

Conceived and designed the experiments: HA RJD BY PJM. Performed the experiments: HA RJD BY PJM. Analyzed the data: HA RJD BY PJM. Wrote the paper: HA RJD BY PJM.

References

  1. 1. Ferrell AS, Britz GW (2013) Developments on the horizon in the treatment of neurovascular problems. Surg Neurol Int 4: S31–37.
  2. 2. Blackham KA, Meyers PM, Abruzzo TA, Albuquerque FC, Fiorella D, et al. (2012) Endovascular therapy of acute ischemic stroke: report of the Standards of Practice Committee of the Society of NeuroInterventional Surgery. J Neurointerv Surg 4: 87–93.
  3. 3. Lloyd-Jones D, Adams R, Carnethon M, De Simone G, Ferguson TB, et al. (2009) Heart disease and stroke statistics–2009 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation 119: 480–486.
  4. 4. Sacco RL, Benjamin EJ, Broderick JP, Dyken M, Easton JD, et al. (1997) American Heart Association Prevention Conference. IV. Prevention and Rehabilitation of Stroke. Risk factors. Stroke 28: 1507–1517.
  5. 5. Hacke W, Furlan AJ, Al-Rawi Y, Davalos A, Fiebach JB, et al. (2009) Intravenous desmoteplase in patients with acute ischaemic stroke selected by MRI perfusion-diffusion weighted imaging or perfusion CT (DIAS-2): a prospective, randomised, double-blind, placebo-controlled study. Lancet Neurol 8: 141–150.
  6. 6. Adelman SM (1981) The National Survey of Stroke. Economic impact. Stroke 12: I69–87.
  7. 7. Taylor TN, Davis PH, Torner JC, Holmes J, Meyer JW, et al. (1996) Lifetime cost of stroke in the United States. Stroke 27: 1459–1466.
  8. 8. Rosamond W, Flegal K, Friday G, Furie K, Go A, et al. (2007) Heart disease and stroke statistics–2007 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation 115: e69–171.
  9. 9. Barreto AD, Alexandrov AV (2012) Adjunctive and alternative approaches to current reperfusion therapy. Stroke 43: 591–598.
  10. 10. Agrawal A, Golovoy D, Nimjee S, Ferrell A, Smith T, et al. (2012) Mechanical thrombectomy devices for endovascular management of acute ischemic stroke: Duke stroke center experience. Asian J Neurosurg 7: 166–170.
  11. 11. Cloft HJ, Rabinstein A, Lanzino G, Kallmes DF (2009) Intra-arterial stroke therapy: an assessment of demand and available work force. AJNR Am J Neuroradiol 30: 453–458.
  12. 12. Hirsch JA, Yoo AJ, Nogueira RG, Verduzco LA, Schwamm LH, et al. (2009) Case volumes of intra-arterial and intravenous treatment of ischemic stroke in the USA. J Neurointerv Surg 1: 27–31.
  13. 13. Furlan A, Higashida R, Wechsler L, Gent M, Rowley H, et al. (1999) Intra-arterial prourokinase for acute ischemic stroke. The PROACT II study: a randomized controlled trial. Prolyse in Acute Cerebral Thromboembolism. JAMA 282: 2003–2011.
  14. 14. (2012) MERCI Retriever. Clin Privil White Pap: 1–18.
  15. 15. Alshekhlee A, Pandya DJ, English J, Zaidat OO, Mueller N, et al. (2012) Merci mechanical thrombectomy retriever for acute ischemic stroke therapy: literature review. Neurology 79: S126–134.
  16. 16. Wechsler LR (2006) Does the Merci Retriever work? Against. Stroke 37: 1341–1342 discussion 1342–1343.
  17. 17. Saver JL (2006) Does the Merci Retriever work? For. Stroke 37: 1340–1341 discussion 1342–1343.
  18. 18. Becker KJ, Brott TG (2005) Approval of the MERCI clot retriever: a critical view. Stroke 36: 400–403.
  19. 19. (2004) Merci retriever. Clin Privil White Pap: 1–8.
  20. 20. Smith WS (2006) Safety of mechanical thrombectomy and intravenous tissue plasminogen activator in acute ischemic stroke. Results of the multi Mechanical Embolus Removal in Cerebral Ischemia (MERCI) trial, part I. AJNR Am J Neuroradiol 27: 1177–1182.
  21. 21. Smith WS, Sung G, Saver J, Budzik R, Duckwiler G, et al. (2008) Mechanical thrombectomy for acute ischemic stroke: final results of the Multi MERCI trial. Stroke 39: 1205–1212.
  22. 22. The penumbra pivotal stroke trial: safety and effectiveness of a new generation of mechanical devices for clot removal in intracranial large vessel occlusive disease. Stroke 40: 2761–2768.
  23. 23. Smith WS, Sung G, Starkman S, Saver JL, Kidwell CS, et al. (2005) Safety and efficacy of mechanical embolectomy in acute ischemic stroke: results of the MERCI trial. Stroke 36: 1432–1438.
  24. 24. Saver JL, Jahan R, Levy EI, Jovin TG, Baxter B, et al. (2012) Solitaire flow restoration device versus the Merci Retriever in patients with acute ischaemic stroke (SWIFT): a randomised, parallel-group, non-inferiority trial. Lancet 380: 1241–1249.
  25. 25. Dorn F, Stehle S, Lockau H, Zimmer C, Liebig T (2012) Endovascular treatment of acute intracerebral artery occlusions with the solitaire stent: single-centre experience with 108 recanalization procedures. Cerebrovasc Dis 34: 70–77.
  26. 26. Stampfl S, Hartmann M, Ringleb PA, Haehnel S, Bendszus M, et al. (2011) Stent placement for flow restoration in acute ischemic stroke: a single-center experience with the Solitaire stent system. AJNR Am J Neuroradiol 32: 1245–1248.
  27. 27. Khatri P, Hill MD, Palesch YY, Spilker J, Jauch EC, et al. (2008) Methodology of the Interventional Management of Stroke III Trial. Int J Stroke 3: 130–137.
  28. 28. Kulcsar Z, Bonvin C, Pereira VM, Altrichter S, Yilmaz H, et al. (2010) Penumbra system: a novel mechanical thrombectomy device for large-vessel occlusions in acute stroke. AJNR Am J Neuroradiol 31: 628–633.
  29. 29. Tenser MS, Amar AP, Mack WJ (2011) Mechanical thrombectomy for acute ischemic stroke using the MERCI retriever and penumbra aspiration systems. World Neurosurg 76: S16–23.
  30. 30. Jeromel M, Milosevic ZV, Kocijancic IJ, Lovric D, Svigelj V, et al. (2013) Mechanical revascularization for acute ischemic stroke: a single-center, retrospective analysis. Cardiovasc Intervent Radiol 36: 338–345.
  31. 31. Bosel J, Hacke W, Bendszus M, Rohde S (2012) Treatment of acute ischemic stroke with clot retrieval devices. Curr Treat Options Cardiovasc Med 14: 260–272.
  32. 32. Cohen JE, Gomori JM, Leker RR, Moscovici S, Ramirez-Denoriega F, et al. (2012) Recanalization with stent-based mechanical thrombectomy in anterior circulation major ischemic stroke. J Clin Neurosci 19: 39–43.
  33. 33. Day JS, Hurley MC, Chmayssani M, Rahme RJ, Alberts MJ, et al. (2011) Endovascular stroke therapy: a single-center retrospective review. Neurosurg Focus 30: E10.
  34. 34. Ciccone A, Valvassori L (2013) Endovascular treatment for acute ischemic stroke. N Engl J Med 368: 2433–2434.
  35. 35. Ciccone A, Valvassori L, Nichelatti M (2011) SYNTHESIS expansion: design of a nonprofit, pragmatic, randomized, controlled trial on the best fast-track endovascular treatment vs. standard intravenous alteplase for acute ischemic stroke. Int J Stroke 6: 259–265.
  36. 36. Ciccone A, Valvassori L, Nichelatti M, Sgoifo A, Ponzio M, et al. (2013) Endovascular treatment for acute ischemic stroke. N Engl J Med 368: 904–913.
  37. 37. Kidwell CS, Jahan R, Gornbein J, Alger JR, Nenov V, et al. (2013) A trial of imaging selection and endovascular treatment for ischemic stroke. N Engl J Med 368: 914–923.
  38. 38. Riedel CH, Zimmermann P, Jensen-Kondering U, Stingele R, Deuschl G, et al. (2011) The importance of size: successful recanalization by intravenous thrombolysis in acute anterior stroke depends on thrombus length. Stroke 42: 1775–1777.
  39. 39. Saqqur M, Uchino K, Demchuk AM, Molina CA, Garami Z, et al. (2007) Site of arterial occlusion identified by transcranial Doppler predicts the response to intravenous thrombolysis for stroke. Stroke 38: 948–954.
  40. 40. Zaidat OO, Lazzaro MA, Gupta R, Yavagal DR, Rasmussen PA, et al. (2012) Interventional Management of Stroke III trial: establishing the foundation. J Neurointerv Surg 4: 235–237.
  41. 41. Levy EI, Firlik AD, Wisniewski S, Rubin G, Jungreis CA, et al. (1999) Factors affecting survival rates for acute vertebrobasilar artery occlusions treated with intra-arterial thrombolytic therapy: a meta-analytical approach. Neurosurgery 45: 539–545 discussion 545–538.
  42. 42. Schonewille WJ, Wijman CA, Michel P, Rueckert CM, Weimar C, et al. (2009) Treatment and outcomes of acute basilar artery occlusion in the Basilar Artery International Cooperation Study (BASICS): a prospective registry study. Lancet Neurol 8: 724–730.
  43. 43. Lindsberg PJ, Mattle HP (2006) Therapy of basilar artery occlusion: a systematic analysis comparing intra-arterial and intravenous thrombolysis. Stroke 37: 922–928.
  44. 44. Smith WS (2007) Intra-arterial thrombolytic therapy for acute basilar occlusion: pro. Stroke 38: 701–703.
  45. 45. Nogueira RG, Yoo AJ, Buonanno FS, Hirsch JA (2009) Endovascular approaches to acute stroke, part 2: a comprehensive review of studies and trials. AJNR Am J Neuroradiol 30: 859–875.
  46. 46. Gonzalez RG, Copen WA, Schaefer PW, Lev MH, Pomerantz SR, et al. (2013) The Massachusetts General Hospital acute stroke imaging algorithm: an experience and evidence based approach. J Neurointerv Surg 5 Suppl 1i7–12.
  47. 47. Jauch EC, Saver JL, Adams HP Jr, Bruno A, Connors JJ, et al. (2013) Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 44: 870–947.
  48. 48. Warwick K (2004) March of the machines : the breakthrough in artificial intelligence. Urbana: University of Illinois Press. x, 307 p.
  49. 49. Alpaydin E (2010) Introduction to machine learning. Cambridge, Mass.: MIT Press. xl, 537 p.
  50. 50. Russell SJ, Norvig P, Davis E (2010) Artificial intelligence : a modern approach. Upper Saddle River: Prentice Hall. xviii, 1132 p.
  51. 51. Warwick K (2012) Artificial intelligence : the basics. New York: Routledge. viii, 183 p.
  52. 52. Cuingnet R, Rosso C, Lehericy S, Dormont D, Benali H, et al. (2010) Spatially regularized SVM for the detection of brain areas associated with stroke outcome. Med Image Comput Comput Assist Interv 13: 316–323.
  53. 53. Bouts MJ, Tiebosch IA, van der Toorn A, Viergever MA, Wu O, et al. (2013) Early identification of potentially salvageable tissue with MRI-based predictive algorithms after experimental ischemic stroke. J Cereb Blood Flow Metab 33: 1075–1082.
  54. 54. Alexandrov AV, Grotta JC (2002) Arterial reocclusion in stroke patients treated with intravenous tissue plasminogen activator. Neurology 59: 862–867.
  55. 55. Grotta JC, Welch KM, Fagan SC, Lu M, Frankel MR, et al. (2001) Clinical deterioration following improvement in the NINDS rt-PA Stroke Trial. Stroke 32: 661–668.
  56. 56. Intracerebral hemorrhage after intravenous t-PA therapy for ischemic stroke. The NINDS t-PA Stroke Study Group. Stroke 28: 2109–2118.
  57. 57. Rai AT, Raghuram K, Carpenter JS, Domico J, Hobbs G (2013) Pre-intervention cerebral blood volume predicts outcomes in patients undergoing endovascular therapy for acute ischemic stroke. J Neurointerv Surg 5 Suppl 1i25–32.
  58. 58. Roth C, Papanagiotou P, Behnke S, Walter S, Haass A, et al. (2010) Stent-assisted mechanical recanalization for treatment of acute intracerebral artery occlusions. Stroke 41: 2559–2567.