Figures
Abstract
Purpose
To evaluate the intra- and inter-observer variability of the North American Symptomatic Carotid Endarterectomy Trial (NASCET) and Warfarin-Aspirin Symptomatic Intracranial Disease (WASID) criteria for the evaluation of middle cerebral artery (MCA) stenosis using digital subtraction angiography (DSA).
Materials and Methods
DSA images of 114 cases with 131 stenotic MCAs were retrospectively analyzed. Two radiologists and a researcher measured the degree of MCA stenosis independently using both NASCET and WASID methods. To determine intra-observer agreement, all the observers reevaluated the degree of MCA stenosis 4 weeks later. The linear relation and coefficient of variation (CV) between the measurements made by the two methods were assessed by correlation coefficient and multi-factor analysis of variance (ANOVA), respectively. Intra- and inter-observer variability of the two methods was evaluated by intraclass correlation coefficient (ICC), Spearman’s R value, Pearson correlation coefficient and Bland-Altman plots.
Results
Despite the fact that the degree of MCA stenosis measured by NASCET was lower than measured using the WASID method, there was good linear correlation between the measurements made by the two methods (for the mean measurements of the 3 observers, NASCET% = 0.891 × WASID% - 1.89%; ICC, Spearman’s R value and Pearson correlation were 0.874, 0.855, and 0.874, respectively). The CVs of both intra- and inter-observer measurements of MCA stenosis using WASID were significantly lower than that using NASCET confirmed by the multi-factor ANOVA results, which showed only the measurement methods of MCA stenosis had significant effects on the CVs both in intra- and inter-observer measurements (both P values < 0.001). Intra-observer measurements showed good or excellent agreement with respect to WASID and NASCET evaluation (ICC, 0.656 to 0.817 and 0.635 to 0.761, respectively). Good agreement for the WASID evaluation (ICC, 0.592 to 0.628) and for the NASCET evaluation (ICC, 0.529 to 0.568) was observed for inter-observer measurements. Bland-Altman plots demonstrated that the WASID method had better reproducibility and intra-observer agreement than NASCET method for evaluating MCA stenosis.
Citation: Chen L, Zhan Q, Ma C, Liu Q, Zhang X, Tian X, et al. (2015) Reproducibility of Middle Cerebral Artery Stenosis Measurements by DSA: Comparison of the NASCET and WASID Methods. PLoS ONE 10(6): e0130991. https://doi.org/10.1371/journal.pone.0130991
Editor: Rex Eugene Jung, University of New Mexico, UNITED STATES
Received: October 14, 2014; Accepted: May 27, 2015; Published: June 26, 2015
Copyright: © 2015 Chen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: This work was supported by the 1255 Academic Discipline Project of Shanghai Changhai Hospital (CH125520800) and the Shanghai Shenkang Project SHDC12013110 (Clinical Research Registration Number: ChiCTR-DDT-14004284), Shanghai, China.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Intracranial artery atherosclerosis is increasingly being recognized as a major cause of stroke worldwide, and patients with intracranial steno-occlusive disease have an augmented risk of vascular events [1]. Intracranial arterial stenosis (IAS) corresponds to luminal narrowing of large intracranial arteries [2]. Primary atherosclerosis is the main cause of IAS, although sometimes embolic events can result in severe stenosis [2]. Atherosclerotic IAS usually occurs in the middle cerebral artery (MCA), which is the principle intracranial artery perfusing the cerebral hemispheres [3]. Moreover, patients with symptomatic MCA stenosis have a higher prevalence (12.5%) of stroke than those with asymptomatic MCA disease (2.85%) [4].
Accurate measurement of the degree of stenosis is important to guide treatment decisions in the clinic [5], and several studies have suggested that patients with more than 70% stenosis may benefit in the long-term from artery stenting, while another study showed an increased risk of ischemic recurrence with significant stenosis [6,7]. Several imaging modalities, including digital subtraction angiography (DSA), computed tomography angiography (CTA), and magnetic resonance angiography (MRA) are used to assess intracranial atherosclerosis. However, the standard method for the evaluation of intracranial atherosclerosis is still DSA. North American Symptomatic Carotid Endarterectomy Trial (NASCET) and Warfarin-Aspirin Symptomatic Intracranial Disease (WASID) are the most commonly used methods to evaluate vascular stenosis. Both methods determine the degree of stenosis by taking the diameter of the residual lumen at the site of maximal luminal narrowing, and the evaluation of stenosis is based on the formula: % stenosis = [1 −(Dstenosis/Dnormal)] × 100 [1,8]. According to this equation, for the NASCET and WASID methods, the normal segment is ideally measured at a site distal and proximal to the stenotic lesion, respectively [9]. Although designed for carotid stenosis, the NASCET method has been widely employed to measure intracranial stenosis [10].
Although measurements of MCA stenosis using the same angiogram differ according to the method used, a mathematical relationship between the two methods can also be evaluated and it is possible to convert a percent stenosis made by one method to another based on the fact that after analyzing the stenosis degree measurements of a large series of patients [11]. Nevertheless, using two different methods may lead to confusion in clinical practice and, in addition, it prevents generalization of research results. Uniformity in reporting treatment results and consistency in patient management necessitates selection of one method as the standard method in measuring degree of MCA stenosis [11]. During this process, reproducibility is an important factor that should be taken into account.
Therefore, the aim of the present study was to evaluate intra- and inter-observer variability of NASCET and WASID criteria for the evaluation of MCA stenosis using DSA and to compare measurements made by the two methods.
Materials and Methods
Subjects
This retrospective study was approved by our Institutional Review Board, Shanghai Changhai Hospital Ethics Committee. Signed written informed consent was waived from all participants. A computerized search of the DSA imaging database and medical records from May 2011 through March 2014 at our institution yielded a list of 200 patients who underwent DSA imaging for suspected intracranial atherosclerotic disease using the aforementioned standard clinical protocol. The inclusion criteria for the present study were: ischemic stroke or transient ischemic attack in the target MCA territory within the past 30 days; and a stenosed vessel at the M1 segment of the MCA on DSA images. We excluded patients using the following criteria: 1) poor image quality for interpretation; 2) nonatherosclerotic vasculopathy, such as dissection or Moyamoya disease; and 3) normal or occluded arteries at the M1 segment of the MCA. In the final analysis, 114 subjects with 131 stenosed MCAs were enrolled in this study.
Digital subtraction angiography
DSA was carried out using a Siemens angiographic unit (Siemens Medical Solution, Erlangen, Germany) with a protocol involving femoral puncture and selective injection of Ultravist contrast agent (Bayer Healthcare, Erlangen, Germany) into the MCA vessels. Images were acquired using a 1024 × 1024 matrix, a 220 × 220 mm2 field of view, and a pixel size of 0.21 × 0.21 mm2 at 5 mL/s until the late venous phase. Three projections (anteroposterior, oblique, and lateral views) were acquired in all cases.
Image analysis
All measurements of luminal stenosis were performed by three independent observers: two radiologists (referred to as Observer 1 and 2, respectively) with 5 years of experience in neuroradiology, and one researcher (referred to as Observer 3) with 4 years of experience in neuroradiology, who were blinded to the clinical information of each patient. An electronic ruler from industry-standard Digital Imaging and Communications in Medicine reading software (General Electric Advantage Work-station, GE Healthcare, Milwaukee, WI, USA) configured on a physician workstation with a technical high resolution screen (Jusha Healthcare, Nanjing, People’s Republic of China) was used to measure vessel diameters on the anteroposterior view of DSA images. The images were zoomed to 250%, and window width and level were adjusted to optimize vessel contour. The degree of stenosis or the ratio between the residual lumen at the stenosis and the normal lumen without stenosis was determined using both NASCET and WASID criteria (Fig 1). To evaluate intra-observer variability, three observers measured MCA stenosis twice during two different sessions that were separated by at least 4-week interval to avoid any recall bias. All of the observers were blinded to theirs and each other’s results.
The NASCET method uses the distal segment as a comparator to the stenotic region. The WASID method divides the stenotic segment measurement by the proximal normal segment.
Statistical analysis
Statistical analyses were performed using the SPSS software for windows (version 16.0, SPSS Inc., Chicago, IL, USA). Quantitative data are described as means ± standard deviation and qualitative data are expressed as percentages. The relationship between measurements of the percentage of MCA stenosis made using NASCET and WASID methods was assessed by correlation coefficient and the results are displayed as scatterplots with four regression lines (the horizontal and vertical axes indicate the average degree of stenosis measured using WASID and NASCET methods, respectively). All values of the measurements are expressed as the mean ± SEM (Standard Error of Mean).
The coefficients of variation (CVs) of intra- and inter-observer measurements were used to evaluate the variability between the two methods. The CV was determined by the standard deviation (SD) of the two matched measurements of each method divided the mean of the two matched measurements of each method (CV = SD/mean × 100%). Multi-factor analysis of variance (ANOVA) (two factors in the present study, methods and observers) was used to assess the variability of CVs of intra- and inter-observer measurements between the two methods. The overall CVs and percentage difference of the MCA stenosis measurements made by the two methods were also calculated and compared, respectively.
Intraclass correlation coefficient (ICC) and Bland-Altman plots were used to assess intra- and inter-observer variability of both the NASCET and WASID methods. The ICC is the proportion of the total variance caused by variation between serial measurements or single measurements by different observers. Values were graded according to the method proposed by Shout and Fleiss [12]: < 0.4, poor agreement; 0.4–0.75, good agreement; > 0.75, excellent agreement. For the Bland-Altman analysis [13], differences in the two measurements were plotted against the mean of the two measurements to assess intra- and inter-observer agreement for each method. Therefore, bias and 95% confidence intervals of the mean difference (limits of agreement) were evaluated across the mean stenosis, and the less agreement, the wider the dispersion of the scatterplot at a given mean measurement. Spearman and Pearson correlation tests were also performed to assess relationships between measurements. A P-value < 0.05 was regarded as statistically significant.
Results
A total of 131 MCA M1 segments of 114 cases (75 males [65.8%], 39 females [34.2%], mean age: 56.8 ± 11.2 years) were retrospectively analyzed. Various degrees of stenosis were also detected in these segments. In 53 patients (46.5%), the stenosis was on the left M1 segment of the MCA, while in 44 patients (38.6%) it was on the right M1 segment of the MCA and in 17 patients (14.9%) it was bilateral.
Correlation between NASCET and WASID
The results of linear correlation between the NASCET and WASID methods are provided in Table 1. The mean MCA stenosis measurements of each observer and all the 3 observers for NASCET are ranging from 40.9% to 47.3%, and for WASID are ranging from 49.8% to 54.4%. The ICCs, Spearman’s R values and Pearson correlation coefficients of the measurements showed excellent agreement and high correlation between the two methods. The scatterplots showing the NASCET and WASID measurements are shown in Fig 2, where there are four evident linear regression lines between the two criteria measurements. Taking the mean measurements of the 3 observers as an example, the following equation can be deduced from the linear regression line (Fig 2D), and the R2 value was 0.762: NASCET% = 0.891 × WASID%– 1.89%; i.e., 70.0% NASCET stenosis is equal to 80.7% WASID stenosis.
(A) The measurements of Observer 1. (B) The measurements of Observer 2. (C) The measurements of Observer 3. (D) The mean measurements of all 3 observers.
Intra-observer agreement
Statistical results of intra-observer reproducibility are presented in Table 2. Intra-observer measurements showed good or excellent agreement both for the NASCET evaluation and the WASID evaluation using ICC (0.635 to 0.761 and 0.656 to 0.817, respectively, P < 0.01). Spearman’s R value and Pearson correlation coefficient also suggested a high correlation in intra-observer measurements. The bias and limits of agreement of intra-observer agreement for the measurements of MCA stenosis by using NASCET and WASID methods by employing Bland-Altman plots (Fig 3) were provided in Table 3 (Intra-observer variability). Compared with NASCET, WASID has lower bias and limits of agreement (Table 3) and the narrower the dispersion of the scatterplot at a given measurement (Fig 3). In addition, the WASID method showed better reproducibility than the NASCET method for evaluating of MCA stenosis with a higher ICC (0.791 versus 0.684 for 1st observer, 0.656 versus 0.635 for 2nd observer and 0.817 versus 0.761 for 3rd observer), Spearman’s R value (0.792 versus 0.681 for 1st observer, 0.649 versus 0.624 for 2nd observer and 0.814 versus 0.756 for 3rd observer), and Pearson correlation coefficient (0.791 versus 0.684 for 1st observer, 0.657 versus 0.636 for 2nd observer and 0.818 versus 0.762 for 3rd observer), respectively. Moreover, multi-factor ANOVA results showed the CVs of intra-observer measurements using WASID method were significantly lower than that using NASCET (Table 4) in all of the 3 observers (P < 0.001). Observer had no significant effects on the CVs of MCA stenosis in intra-observer measurements (P = 0.406).
(A, C and E) WASID method for Observer 1, 2 and 3, respectively. (B, D and F) NASCET method for Observer 1, 2 and 3, respectively.
Inter-observer agreement
Statistical results of inter-observer reproducibility are presented in Table 5. Inter-observer measurements showed good agreement for both the WASID evaluation and good agreement for the NASCET evaluation using ICC (0.592 to 0.628 and 0.529 to 0.568, respectively, P < 0.01). Spearman’s R value and Pearson correlation coefficient also suggested good correlation for inter-observer measurements. The bias and limits of agreement for the measurements of MCA stenosis of inter-observer agreement by employing Bland-Altman plots (Fig 4) for the NASCET and WASID methods were provided in Table 3 (Inter-observer variability), respectively. As with the intra-observer measurements, the WASID method also has lower bias and limits of agreement (Table 3) and the narrower the dispersion of the scatterplot at a given measurement (Fig 4) and had better reproducibility than the NASCET method for evaluating MCA stenosis, as illustrated by higher ICC (0.592 versus 0.529 for the 1st and 2nd observers, 0.628 versus 0.568 for 1st and 3rd observers, and 0.615 versus 0.562 for 2nd and 3rd observers), Spearman’s R value (0.589 versus 0.517 for the 1st and 2nd observers, 0.618 versus 0.560 for 1st and 3rd observers, and 0.605 versus 0.563 for 2nd and 3rd observers), and Pearson correlation coefficient (0.597 versus 0.536 for the 1st and 2nd observers, 0.632 versus 0.569 for 1st and 3rd observers, and 0.615 versus 0.566 for 2nd and 3rd observers), respectively. Moreover, multi-factor ANOVA results showed the CVs of inter-observer measurements using WASID method were significantly lower than that using NASCET (Table 4) in all of the 3 observers (P < 0.001). Observer had no significant effects on the CVs of MCA stenosis in inter-observer measurements (P = 0.148).
(A, C and E) WASID method for Observers 1&2, 1&3 and 2&3, respectively. (B, D and F) NASCET method for Observers 1&2, 1&3 and 2&3, respectively.
The results showed that the degree of MCA stenosis measured with WASID had lower overall CV than that with NASCET (CVs were 12.5% and 16.9% for WASID and NASCET, respectively), and the percentage difference was 36.0%.
Discussion
The present study is the first to evaluate the reproducibility of NASCET and WASID techniques for measuring MCA stenosis degree using DSA. Correct quantification of stenosis degree is fundamental in planning the correct therapeutic approach and, because of the confusion generated by the use of different methods, comparative analyses may be important [14]. It is essential to emphasize that the patient population in our study is considerably larger than in previous studies [5, 11]. On the whole, our data indicated good reproducibility of all measurements.
As the NASCET and WASID methods adopt different reference sites for determining normal vessel diameter, each method provides a different degree of stenosis for the same lesion based on the same angiogram [8, 15]. Therefore, referring to the percentage of MCA stenosis without regard for the measurement method is misleading and it may lead to confusion in clinical practice. In our study, however, measurements of MCA stenosis by employing NASCET and WASID methods were highly correlated with Spearman’s R values and Pearson correlation coefficients (Table 1). This correlation was consistent with a previous study from our institute [16]. We also deduced equations based on the scatterplots of NASCET and WASID measurements (Fig 2), which may be used to express the percentage of MCA stenosis measured using one method as a function of the corresponding value measured using another method. However, conversion between the two means of MCA stenosis degree measurements is only possible after analyzing a large series of patients and deriving an equation based on these patients. To our knowledge, no study has previously reported such equations. Moreover, several studies have suggested a linear relationship between the NASCET and WASID methods for the evaluation of carotid artery stenosis [15, 17]. It is worth noting that the degree of stenosis measured with the NASCET method was lower than measured by the WASID method, and the main reason for this is likely the reference points used to determine normal artery diameter by different methods. More specifically, the diameter of the proximal site of MCA is larger than the distal site in accordance with the natural morphology of the vessel [18]; in another words, the diameter of the reference lumen at the proximal site is larger than at the distal site. On the other hand, sometimes the overlapped vessels at the distal site of MCA results in difficulties discriminating the boundaries of the vessels and increases error when evaluating the degree of MCA stenosis [19].
Even though it is possible to convert the degree of stenosis calculated by one method to another method, the best solution is likely to use one method as the current standard [11]. An important factor in choosing a standard method is the level of reproducibility. In our study, with respect to both intra- and inter-observer agreement, we observed good results using NASCET and WASID methods. Moreover, the ICC values of intra- and inter-observer agreement indicated good or excellent and good agreement, respectively (Tables 2 and 5). In addition, the Spearman’s R values and Pearson correlation coefficients of the measurements obtained from different observers using different methods demonstrated to be of high relevance. Yet, the ICCs, Spearman’s R values, and Pearson correlation coefficients of the measurements obtained using WASID method were higher than using NASCET; i.e., Measurements of MCA stenosis using the WASID method were more reproducible than the NASCET method. Moreover, we employed the Bland-Altman plots to evaluate intra- and inter-observer agreement, and the diagrams showed a relatively wide interval of agreement compared with the average (Figs 3 and 4). This suggests possible discrepancies between the two methods and among the 3 observers. However, intra- and inter-observer variability was still acceptable. Furthermore, the CVs of both intra- and inter-observer measurements of MCA stenosis using WASID were significantly lower than that using NASCET confirmed by the multi-factor ANOVA results, which showed only the measurement methods of MCA stenosis had significant effects on the CVs both in intra- and inter-observer measurements (both P values < 0.001). C1R2In addition, our analysis also showed that the overall CVs of the measurements made by NASCET and WASID methods were 16.9% and 12.5%, respectively, and WASID has lower CV than NASCET. The variability of measurements of MCA stenosis employing the WASID method has an overall percentage difference of 36.0 than the NASCET method. According to the results of the present study, we found that the WASID has better reproducibility than the NASCET for the evaluation of MCA stenosis, and reproducibility of intra-observer measurements were superior to inter-observer measurements. Several previous studies have also shown the WASID method has better agreement for intra- and inter-observer measurements of carotid artery stenosis [17, 20].
To our knowledge, patients with a greater degree of stenosis of intracranial arteries are more likely to undergo pronounced clinical events and these patients are also more likely to experience symptom recurrence within the same territory as the stenosed arteries [21, 22]. Therefore, accurate measurement of the degree of stenosis and detection of intracranial steno-occlusive disease are vitally important, especially when planning stent surgeries for patients. In the present study, we found that the WASID method is more reproducible, and our results consistent with Samuels et al. [8] in which the WASID method was deemed a better candidate for the worldwide standard for measuring intracranial stenosis on angiograms, as well as for measuring stenosis using non-invasive techniques.
Conclusions
In conclusion, the present study showed a linear relationship between the NASCET and WASID methods for measuring MCA stenosis. Both NASCET and WASID methods have an acceptable level of agreement; however, the WASID method provided higher values and offered better reproducibility, therefore, it is better suited to serve as the standard for measuring the degree of MCA stenosis in future studies.
Author Contributions
Conceived and designed the experiments: CM LGC QZ QL JPL. Performed the experiments: SYC YLJ. Analyzed the data: LGC QZ CM YLJ. Contributed reagents/materials/analysis tools: LGC XFZ XT YMD. Wrote the paper: LGC CM QZ.
References
- 1. Turan TN, Makki AA, Tsappidi S, Cotsonis G, Lynn MJ, Cloft HJ, et al. Risk factors associated with severity and location of intracranial arterial stenosis. Stroke. 2010; 41: 1636–1640. pmid:20616323
- 2. Carvalho M, Oliveira A, Azevedo E, Bastos-Leite AJ. Intracranial arterial stenosis. J Stroke Cerebrovasc Dis. 2014;23: 599–609. pmid:23850084
- 3. Tarasów E, Abdulwahed Saleh Ali A, Lewszuk A, Walecki J. Measurements of the middle cerebral artery in digital subtraction angiography and MR angiography. Med Sci Monit. 2007;13: 65–72. pmid:17507888
- 4. Kern R, Steinke W, Daffertshofer M, Prager R, Hennerici M. Stroke recurrences in patients with symptomatic vs asymptomatic middle cerebral artery disease. Neurology. 2005;65: 859–864. pmid:16186524
- 5. Liu Q, Huang J, Degnan AJ, Chen S, Gillard JH, Teng Z, et al. Comparison of high-resolution MRI with CT angiography and digital subtraction angiography for the evaluation of middle cerebral artery atherosclerotic steno-occlusive disease. Int J Cardiovasc Imaging. 2013;29: 1491–1498. pmid:23686460
- 6. Tang CW, Chang FC, Chern CM, Lee YC, Hu HH, Lee IH. Stenting versus medical treatment for severe symptomatic intracranial stenosis. AJNR Am J Neuroradiol. 2011;32: 911–916. pmid:21393399
- 7. Mazighi M, Tanasescu R, Ducrocq X, Vicaut E, Bracard S, Houdart E, et al. Prospective study of symptomatic atherothrombotic intracranial stenoses: the GESICA study. Neurology. 2006;66: 1187–1191. pmid:16636236
- 8. Samuels OB, Joseph GJ, Lynn MJ, Smith HA, Chimowitz MI. A standardized method for measuring intracranial arterial stenosis. AJNR Am J Neuroradiol. 2000;21: 643–646. pmid:10782772
- 9. Fox AJ. How to measure carotid stenosis. Radiology. 1993;186: 316–318. pmid:8421726
- 10. Bash S, Villablanca JP, Jahan R, Duckwiler G, Tillis M, Kidwell C, et al. Intracranial vascular stenosis and occlusive disease: evaluation with CT angiography, MR angiography, and digital subtraction angiography. AJNR Am J Neuroradiol. 2005; 26: 1012–1021. pmid:15891154
- 11. Kılıçkap G, Ergun E, Başbay E, Koşar P, Kosar U. Carotid stenosis evaluation by 64-slice CTA: comparison of NASCET, ECST and CC grading methods. Int J Cardiovasc Imaging. 2012;28: 1257–1266. pmid:21858656
- 12. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86: 420–428. pmid:18839484
- 13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1: 307–310. pmid:2868172
- 14. Saba L, Mallarini G. A comparison between NASCET and ECST methods in the study of carotids: evaluation using Multi-Detector-Row CT angiography. Eur J Radiol. 2010;76: 42–47. pmid:19464837
- 15. Eliasziw M, Smith RF, Singh N, Holdsworth DW, Fox AJ, Barnett HJ. Further comments on the measurement of carotid stenosis from angiograms. North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group. Stroke. 1994;25: 2445–2449. pmid:7974588
- 16. Huang J, Degnan AJ, Liu Q, Teng Z, Yue CS, Gillard JH, et al. Comparison of NASCET and WASID criteria for the measurement of intracranial stenosis using digital subtraction and computed tomography angiography of the middle cerebral artery. J Neuroradiol. 2012;39: 342–345. pmid:22197402
- 17. Rothwell PM, Gibson RJ, Slattery J, Sellar RJ, Warlow CP. Equivalence of measurements of carotid stenosis. A comparison of three methods on 1001 angiograms. European Carotid Surgery Trialists' Collaborative Group. Stroke. 1994;25: 2435–2439. pmid:7974586
- 18. Zhu XJ, Du B, Lou X, Hui FK, Ma L, Zheng BW, et al. Morphologic characteristics of atherosclerotic middle cerebral arteries on 3T high-resolution MRI. AJNR Am J Neuroradiol. 2013;34: 1717–1722. pmid:23639560
- 19. Nederkoorn PJ, Elgersma OE, Mali WP, Eikelboom BC, Kappelle LJ, van der Graaf Y. Overestimation of carotid artery stenosis with magnetic resonance angiography compared with digital subtraction angiography. J Vasc Surg. 2002;36: 806–813. pmid:12368742
- 20. Staikov IN, Arnold M, Mattle HP, Remonda L, Sturzenegger M, Baumgartner RW, et al. Comparison of the ECST, CC, and NASCET grading methods and ultrasound for assessing carotid stenosis. European Carotid Surgery Trial. North American Symptomatic Carotid Endarterectomy Trial. J Neurol. 2000;247: 681–686. pmid:11081806
- 21. Famakin BM, Chimowitz MI, Lynn MJ, Stern BJ, George MG, WASID Trial Investigators. Causes and severity of ischemic stroke in patients with symptomatic intracranial arterial stenosis. Stroke. 2009;40: 1999–2003. pmid:19407228
- 22. Wang Y, Zhao X, Liu L, Soo YO, Pu Y, Pan Y, et al. Prevalence and outcomes of symptomatic intracranial large artery stenoses and occlusions in China: the Chinese Intracranial Atherosclerosis (CICAS) Study. Stroke. 2014;45: 663–669. pmid:24481975