Literature Search

An experienced librarian (M-DW) undertook searches of the MEDLINE and EMBASE databases, from inception (1946 and 1974, respectively) for all available peer-reviewed literature. Two search strategies were used: (1) All studies where administrative data was used to identify cardiovascular diseases; (2) All studies reporting on the validity of administrative data for identifying cardiovascular diseases. Our MEDLINE and EMBASE search strategies are available as (S1, S2, S3, and S4 Texts). To identify additional studies, the authors hand-searched the reference lists of the key articles located. As well, the Cited-By tools in PubMed and Google Scholar were used to find relevant articles that had cited the articles located through the database search. The databases were originally searched from inception to November 2010, with the handsearch conducted up to February 2011. These searches were updated in February 2015.

Two reviewers independently screened the titles and abstracts of the located records for relevance to the study objectives. In the next step, full text publications were evaluated against the inclusion criteria. Any discrepancies were discussed until consensus was reached. When the conflict persisted a third reviewer (JAA-Z) was consulted. No protocol for this systematic review has been published, though more information is available in the following publication [27]. Our review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [34] statements, and our completed PRISMA checklist is provided as (S1 Checklist).

Inclusion Criteria

We considered full-length, English-language, peer-reviewed articles that used administrative data and either reported validation statistics for the International Classification of Diseases (ICD) codes of interest, or provided sufficient data for their calculation. We first included studies that evaluated at least one code pertaining to a subtype of acute stroke, being ICD-8/9 430 or ICD-10 I60 for subarachnoid haemorrhage (SAH), and ICD-8/9 431 or ICD-10 I61 for intracerebral haemorrhage (ICH). For ischaemic stroke, the main codes are ICD-8 433/434 and ICD-9 434 (occlusion of the cerebral arteries), and ICD-10 I63 (cerebral infarction).

Stroke is a heterogeneous disease that is not defined consistently by clinicians or researchers [35]. It has traditionally been distinguished from transient ischaemic attack (TIA) by way of duration (more or less than 24 hours) and the presence/absence of permanent brain infarction. Although advances in neuroimaging have resulted in many events that would previously have been labelled as TIA now being considered as minor strokes, this is an area of ongoing controversy [35]. As such, we took a conservative approach by not considering episodes of TIA as acute stroke, and so excluded studies that solely evaluated codes for TIA (ICD-9 435 or ICD-10 G45).

Although our focus was on the validity of codes for acute stroke-specifically (defined as SAH, ICH, or ischaemic stroke), we also included studies that evaluated a range of codes (ICD-8/9 430–438 or ICD-10 I60-I69) pertaining to a broader group of cerebrovascular diseases. Included in these ranges were the codes for acute stroke listed above, along with codes for acute but ill-defined stroke (ICD-9 436 and ICD-10 I64), other types of ill-defined stroke (ICD-9 437) and other cerebrovascular diseases (ICD-10 I67/68), other types of intracranial haemorrhage than ICH (ICD-9 432 and ICD-10 I62), TIA (ICD-9 435), and late effects of stroke or stroke sequelae (ICD-9 438 and ICD-10 I69). It was important to include these studies because, while reviewing the literature, we observed that this broad range of codes for cerebrovascular disease is frequently used to identify cases of acute stroke.

Data Extraction

Two independent reviewers (NM and VB) examined the full text of each selected record and abstracted data using a standardized collection form (a copy is provided in S5 Text). Information was gathered on the study population, administrative data source, stroke codes and algorithm, validation process, and gold standard. Validation statistics comparing the codes to definite, probable, or possible cases of acute stroke, or the specific diagnoses of SAH, ICH, or ischaemic stroke in particular, were abstracted. These statistics included sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and kappa. Wherever possible, we abstracted statistics for definite and probable cases of acute stroke. However, the number of categories available depended on the choice of gold standard. For example, under the World Health Organisation criteria, potential cases are categorized as either definite stroke or not stroke [36], while the National Survey of Stroke criteria [37] use the categories of definite, highly probable, and probable stroke. Statistics for each sex, for fatal versus non-fatal cases, and for each hospital discharge position (i.e. primary/principal and secondary diagnosis) were abstracted where reported. Data were independently abstracted by each reviewer who subsequently compared their forms to correct errors and resolve discrepancies, if any.

Quality Scores

The design and methods employed in each study, including the rigour of the reference standard, and generalizability of the study population, could influence the resultant validity statistics. Hence, all studies were evaluated for quality, with the validation statistics stratified by level of study quality. An adaptation of the QUADAS tool (Quality Assessment of Diagnostic Accuracy Studies) [38] was used to evaluate study quality. Our group previously used the QUADAS in assessing the validity of codes for diabetes mellitus [30], myocardial infarction [32], heart failure [33] and osteoporosis and fractures [31].

Statistical Analysis

All validation statistics were abstracted as reported. Where sufficient data were available we calculated 95% confidence intervals (95% CI) and validation statistics not directly reported in the original publication. Kappa values (a measure of agreement beyond that expected by chance) greater than 0.60 indicated substantial/perfect agreement, 0.21–0.60 were considered as fair/moderate agreement and those 0.20 or lower as light/poor agreement [39].

Literature Search

We identified 1,587 citations through our original searches (inception to November 2010) of the MEDLINE and EMBASE databases, and an additional 2,160 citations in our updated searches of these databases (January 2010 to February 2015). All citations were screened for relevance to our study objectives, with 198 full-text articles assessed for eligibility (Fig 1), and 39 of these selected for inclusion. We also assessed 75 full-text articles for eligibility that were identified from hand searches, and selected 38 additional articles therein. Thus, a total of 273 articles were assessed for eligibility, from which 196 were excluded, mainly because they reported on the validity of other cardiovascular diseases (n = 44), or did not actually validate stroke diagnoses in administrative data (n = 61). Nine articles were excluded because they were not published in English; their languages of publication were Danish, German, Italian, Japanese, Portuguese, Spanish (two articles), French, and Chinese. Ultimately 77 articles were included for the systematic review of acute stroke.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-style Flowchart of Study Selection and Review.

ICD = International Classification of Diseases.

https://doi.org/10.1371/journal.pone.0135834.g001

Study Characteristics

Of the 77 articles evaluating stroke diagnoses that were included in the final review, 31 (40%) were from the United States (USA), 26 (34%) were from Europe, 13 (17%) were from Canada, four were from Asia (5%), two (3%) were from Australia, and one (1%) was from Sri Lanka. Characteristics of these studies are presented in Table 1. Validation was the primary objective in all but ten [40–49] of these studies. Two articles [40,50] reported on the validity of stroke diagnoses exclusively in a paediatric population. Most studies evaluated the diagnostic codes in hospitalization databases though 16 studies [47,51–65] evaluated stroke as a cause-of-death on death certificates and one study [66] reported on outpatient data exclusively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of Included Studies.

https://doi.org/10.1371/journal.pone.0135834.t001

Gold standard

Chart reviews, sometimes in conjunction with unspecified diagnostic criteria, formed the basis of the gold standard in 35 studies, patient self-report was used in one [112], and national and regional stroke registries or clinical databases served as the gold standard in 12 [52,58–60,65,67,73,74,76,81,91,92]. One study [23] utilized two gold standards, with the reference diagnosis for some cases established upon prospective clinical examination by a neurologist, and for other cases, established after retrospective chart review by a different neurologist. The 28 remaining studies used a specific set of diagnostic criteria, most often the WHO criteria, to evaluate the stroke diagnosis.

Study quality was evaluated based on the QUADAS tool [38], with 54 of 77 studies (70%) categorized as high quality, and the remaining 23 studies as medium quality. A detailed breakdown of the quality assessment for each study is provided in S1 Table. Seven of the medium-quality studies [47,57,59,64,71,73,104] did not adequately describe the validation process or other key methodological aspects, while nine employed a selected study population [25,40,50,77,89,92,105,107,111] (e.g. atrial fibrillation cohort, kidney transplant recipients), and seven used a less-reliable gold standard [24,66,70,82,85,94,112], typically chart review by an individual other than a clinician or trained hospital coder.

Validity of Stroke Codes on Aggregate

The validation statistics reported by each of the included studies are provided in S2 and S3 Tables. We located 36 papers examining the validity of the codes for cerebrovascular disease as an aggregate (ICD-9 430–438 or ICD-10 I60-I69); these codes were compared to diagnoses of any type of cerebrovascular disease (usually as a comorbidity) in 16 studies [56,61,63,66,71,77,79,80,82,89,90,93,94,102,104,105], and to a diagnosis of acute stroke in particular in 21 [26,41,45,47,49,52,54,55,57,62,64,74,83,84,86,91,93,99,103,111,112]. The sensitivity of these codes for any type of cerebrovascular disease was ≥ 82% in seven of the 14 studies (range 32% to 100%). The PPV was ≥ 81% in seven of the 14 studies reporting this statistic (range 43% to 97%). Specificity, reported by ten studies [63,66,71,79,82,89,90,102,104,105], was ≥ 95% in nine of the ten (range 90% to 100%), while NPV was ≥ 95% in eight of the ten studies [63,71,79,82,89,90,94,102,104,105] where this statistic was reported (range 84% to 100%). Kappa values, as reported by seven studies, ranged from 0.52 [82] to 0.76 [89] to 0.91 [79].

Eight of the 16 studies [56,61,63,79,80,90,93,102] were rated as high quality and the other eight [66,71,77,82,89,94,104,105] were rated as medium quality. There was little difference in the sensitivity values between the medium- and high-quality studies: sensitivity ranged from 43% to 100% among the medium-quality studies, and from 32% to 96% among the high-quality studies. There was, however, more of a difference in the PPVs, which ranged from 43% to 83% among the medium-quality studies, and from 52% to 97% among the high-quality studies (with PPV ≥ 88% in five of the seven high-quality studies reporting on PPV).

The sensitivity of these codes for the narrower diagnosis of acute stroke (SAH, ICH, or ischaemic stroke), which was reported by ten studies [52,54,57,62,64,74,93,99,111,112], was ≥ 66% in seven of the ten (range 42% to 96%). The PPV for the reference diagnosis of acute stroke was generally lower than that for the broader category of any cerebrovascular disease, being ≤ 68% in 12 of the 21 studies reporting this statistic (range 28% to 98%). Indeed, in one study that evaluated the PPV against both acute stroke and ‘any cerebrovascular disease’, the PPV for ‘any cerebrovascular disease’ (93%) was higher than that for acute stroke (66%) [93].

Subarachnoid Haemorrhage

Twenty-seven papers reported on the validity of codes for SAH (ICD-9 430 or ICD-10 I60), and the PPV was ≥ 86% in 16 of the 26 studies where this was reported (S2 Table). Fifteen papers compared the SAH code to any type of acute stroke (meaning a case coded for SAH was considered a true-positive if the diagnosis assigned upon validation was SAH, ICH, or ischaemic stroke), and the PPV was ≥ 86% in eight of these studies (range 33% to 100%). Thirteen studies compared the SAH code to the diagnosis of SAH in particular, and the PPV was ≥ 93% in seven of the 13 (range 46% to 100%). The sensitivity of these codes for SAH, reported by only four studies [52,56,84,93], ranged from 35% [93] to 95% [52]. Kappa, reported by only one study [108], was 0.88.

Intracerebral Haemorrhage

Thirty-four studies evaluated the validity of codes for ICH (ICD-9 431/432 or ICD-10 I61/62) (S2 Table). Twenty-six evaluated the main ICH codes 431/I61, and the PPV was ≥ 87% in 16 of the 25 studies reporting on PPV. The PPV, when compared to any type of acute stroke, was ≥ 87% in ten of 15 studies (range 78% to 100%), and when compared to the diagnosis of ICH in particular, the PPV was ≥ 89% in six of the 12 studies (range 63% to 100%). The sensitivity of these codes for ICH, as reported by three studies, ranged from 57% [93] to 69% [56] to 95% [52]. The kappa value for ICD-9 431, as reported by one study [108], was 0.82, while that for ICD-9 430 and 431 combined, was 0.84 [88]. The lesser-used ICH codes (ICD-9 432 or ICD-10 I62) were evaluated in 15 studies [26,41,47,49,55,62,74,78,84,91,93,95,96,100,106], in which the PPV was ≤ 67% in all but two [62,95]. The PPV of ICD-9 431 and 432 combined, available from 13 studies [26,41,42,47,49,53,55,57,74,84,91,93,95], ranged from 33% to 99%. In sum, the PPV of the main codes for ICH (ICD-9 431/ICD-10 I61) was ≥ 87% in most studies.

Ischaemic Stroke

We located 39 papers that examined the validity of codes for ischaemic stroke (S2 Table). From these, 28 evaluated the main code for ischaemic stroke (ICD-9 434 or ICD-10 I63) and the PPV was ≥ 82% in 20 of the 27 studies reporting this statistics (range 62% to 100%). The PPV was ≥ 83% in 13 of the 19 papers [26,41,42,47,49,50,53,55,62,69,74,78,84,91,93,95,96,100,106] where the gold standard diagnosis was any type of acute stroke (SAH, ICH, or ischaemic stroke) (range 62% to 100%), and ≥ 82% in 10 of the 13 papers [23,25,49,50,52,53,76,83,84,86,93,96,107] where the gold standard diagnosis was iscahemic stroke in particular (range 52% to 100%). The sensitivity of these codes for ischaemic stroke, available from six papers [23,52,56,76,84,93], ranged from 2% [23] to 80% [52]. ICD-9 433 was evaluated in 19 papers [25,26,40,41,47,49,50,53,55,69,74,84,91,93,95,96,100,106,107], with the PPV being ≤ 71% in 14 of the 19. Eight papers [25,49,50,53,84,93,96,107] reported on the PPV of ICD-9 433 for ischaemic stroke in particular, and it was ≤ 79% in six of the eight. Just two studies reported on the sensitivity of ICD-9 433 for ischaemic stroke, which was 2% [93] in one and 9% [84] in the other.

The combination of ICD-9 433 and 434 (occlusion of the precerebral or cerebral arteries) was reported on by 20 studies [26,40,41,47,49,51–53,55,57,69,72,74,81,84,93,95,99,107,110], in which the PPV for any stroke ranged from 23% to 100%, and that for ischaemic stroke ranged from 40% to 100%. The sensitivity of these codes for ischaemic stroke, as reported by five studies [52,81,84,93,99], was ≥ 76% in four of these. The code for acute but ill-defined stroke (ICD-9 436 or ICD-10 I64) was evaluated in 23 studies [23,25,26,40,41,47,49,50,53,55,62,69,74,78,83,84,91,93,95,96,100,106,107], with a PPV for any stroke of ≥ 75% in 12 of 19 studies (range 48% to 95%) and a PPV for ischaemic stroke of ≥ 75% in five of seven studies [23,25,49,50,84,96,107] where this statistic was reported (range 50% to 87%).

Nineteen studies [25,26,40,41,47,49,52,55,69,74,84,85,87,88,93,95,101,107,108] employed a broader case definition (ICD-9 433/434/436 or ICD-10 I63/64); the PPV was ≥ 77% in 12 of the 19 (range 46% to 94%). Two studies reported on kappa, which, for ICD-9 433, 434, and 436 together, was 0.82 [108] in one study and 0.85 [88] in the other. Sixteen studies [23,26,40,41,47,49,53,55,62,69,74,78,84,93,95,107] examined the validity of ICD-9 434 and 436 as a pair, and the PPV was ≥ 82% in ten of the 16 (range 66% to 94%). Of interest, Leone et al [93] compared the sensitivity and PPV of different codes for ischaemic stroke, and found that ICD-9 434 and 436 combined had a higher sensitivity (43% versus 35%) for ischaemic stroke, but similar PPV (87% versus 90%), than did ICD-9 434 alone. In sum, the PPV of the main codes for ischaemic stroke (ICD-9 434/ICD-10 I63) was ≥ 82% in most studies, and the PPV of codes for acute but ill-defined stroke (ICD-9 436 or ICD-10 I64) was ≥ 75%.

Validity of Sets of Stroke-Specific Codes

Thirty-six papers examined the validity of a set of stroke-specific codes for identifying any type of stroke (S3 Table) though in two [70,92], the reference diagnosis was stroke/TIA and not just acute stroke. There was some variability in the codes included in each set, but they generally excluded codes pertaining to occlusion of the precerebral arteries, intracranial haemorrhage other than ICH, late effects of stroke/stroke sequalae, and cerebrovascular disorders stemming from other conditions. The code for acute but ill-defined stroke was included in some of these sets. One study, by Reker et al [100], used both a high-sensitivity algorithm for detecting stroke (sensitivity = 91%, PPV = 52%) and high-specificity algorithm (sensitivity = 54%, PPV = 75%). Lakshminarayan et al [46] compared two diagnostic criteria, those from the WHO and Minnesota Stroke Survey (MSS), and found the PPV against the WHO criteria was 98%, while that against the stricter MSS criteria was 68% [46].

The sensitivity of these sets of codes for stroke was ≥ 82% in 13 of 22 studies [40,41,59,62,65,67,70,73–75,78,87,88,92,93,97–101,109,111] where this was reported (range 34% to 97%), and the PPV was ≥ 86% in 18 of the 34 studies [40,41,44,46–48,51,53,58–60,62,65,67,68,70,73–75,78,85,87,88,91,93,95,97–101,107,109,111] that reported this statistic (range 32% to 98%). Amongst these 34 studies, we observed a tendency towards lower PPV when codes for intracranial haemorrhage (ICD-9 432) or ill-defined stroke (ICD-9 437) were included in the search (S3 Table). Kappa ranged from 0.79 [108] to 0.81 [78] to 0.87 [88] in the three studies where this was reported.

Validity by Subgroups

The 77 studies included in this review were published over a 40-year period (1976–2015), though 81% of these (n = 62) were published from 1999-onwards. Few studies reported on any longitudinal changes in sensitivity, and few longitudinal trends in PPV were observed after the 77 studies were stratified by period of publication. For instance, amongst the 27 studies reporting on the PPV of ICD-9 434/ICD-10 I63, the PPV ranged from 64% to 100% in the eight-earliest studies (published from 1993 to 1998), from 72% to 100% in the ten middle studies (published from 1999 to 2004), and from 62% to 100% in the nine most-recent studies (published from 2005 to 2015). And among the 26 studies reporting on ICD-9 431/ICD-10 I61, the PPV ranged from 66% to 100% in the 12 earlier studies (published from 1993–2002) and from 65% to 100% in the 13 more-recent studies (published from 2004–2014). Still, several investigators collected data over ten or more years, and some improvements were observed in the PPV for stroke over time. In one study [91], the PPV of ICD 430–438 increased from 48% to 58% between 1970 and 1980, though by 1989 it had decreased slightly, to 54%. Moreover, Derby et al [42] found that the PPV of ICD-9 431, 432, 434, 435, 436, or 437 for stroke increased by 20% between 1980 and 1990, while Lakshminarayan et al [46] found that the PPV of ICD-9 431, 432, 434, 436, and 437 for stroke increased in this same period by 27% (from 55% in 1980 to 70% in 1990). These studies provide evidence for the accuracy of codes for acute stroke having improved over time.

The accuracy of fatal and non-fatal stroke diagnoses were compared in nine studies [47,51–53,58,60,62,65,91], and amongst these, the accuracy of fatal diagnoses was similar, and often slightly higher, than non-fatal diagnoses (S4 Table). Fifteen studies examined cerebrovascular disease or acute stroke as a cause-of-death on death certificates, and the PPV was ≥ 87% in ten of these (range 50% to 100%). However, the sensitivity of vital statistics data for deaths from stroke was ≤ 71% in six of the 10 studies reporting on this (range 32% to 96%), and the sensitivity of hospitalization data for fatal strokes—in the single study where this statistic was reported—was 55% (compared to 68% for non-fatal hospitalizations for stroke) [91].

Most studies examined codes from the ICD 8^th and 9^th revisions, though 22 studies examined codes from the 10^th revision. Just one of these studies [54] was conducted in the United States. Separate validation statistics for ICD-9 and ICD-10 codes were provided by three articles [52,85,104], and, overall, there were few differences in the accuracy of codes from the two revisions. One study was conducted exclusively on males [51], and two were conducted exclusively on females [78,88], and their findings were consistent with those of most studies where both sexes were represented. Sex-stratified statistics were provided by six studies [47,52,84,87,93,105], from which only minor differences in the accuracy of codes for males and females were observed.

Regional and Temporal Trends

The PPV of diagnostic codes for stroke appears to have improved over the decades, increasing by 20% in one study (where about 51% of stroke discharges identified in 1980, and about 61% of stroke discharges identified in 1990, were confirmed as stroke) [42], and by 27% in another [46] (from 55% in 1980 to 70% in 1990). Many papers included in this review [42,46,53,93,103] attribute these improvements to advances in neuroimaging technology, and increased use and availability of CT and MRI scanners in medical facilities. For example, the 20% increase in PPV reported by Derby et al [42] (in the northeastern US) was accompanied by a 120% increase in the proportion of potential cases who underwent CT or MRI, and a 60% increase in the proportion seen by a neurologist. CT rates have increased in other countries as well, in Finland from 18% in 1983 to 60% in 1989 [114], and in Sweden from 88% in 1995 to 98% in 2010 [115]. In fact, the introduction of CT scanning in the 1970’s is thought to have improved not only the detection of acute strokes, but also the ability of clinicians to make a more precise diagnosis [53]. For instance, when Lindblad et al [53] retrospectively reviewed the stroke codes assigned to cases over 1977–1987, the subtype was reclassified, often on the basis of CT findings, in 27% of the confirmed stroke cases [53]. More strokes were classified as thromboembolic, and fewer as ill-defined, over time in the Derby et al study [42], and similar trends have been reported in Denmark, where the incidence of unspecified stroke decreased from 1997 to 2009, while the incidence of ischaemic stroke increased [116].

Regional differences in access-to and use-of neuroimaging may explain other disparities that were observed in the validity of strokes diagnoses, even amongst studies conducted more recently (i.e. 1990 or later). Tolonen et al [52] observed that those aged 75 years and older were coded more often with non-specific stroke codes. They attributed this to older individuals being less likely to undergo CT and MRI, which could otherwise aide in making a more precise diagnosis. They also found that the sensitivity and PPV were lower at district hospitals than university hospitals [52], and attributed this to the limited neurological expertise and neuroimaging available at the district hospitals. Liu et al [26] also observed higher PPV’s amongst the tertiary hospitals than community hospitals, and attributed this to the greater level of testing performed in the tertiary hospitals. As neuroimaging services continue to become more widespread, and more technologies like CT angiography and perfusion imaging [117] emerge, we expect the validity of stroke codes to increase alongside.

It is possible that changes in billing and reimbursement practices, including the introduction of Diagnosis-Related Groups (DRGs) in the US Medicare program in the 1980’s, may also have contributed to the assignment of more precise stroke codes over time. To investigate this, Derby et al [118] compared how strokes were classified before and after DRGs were introduced, but their findings (increased use of ICD-9 434 and decreased use of ICD-9 436) did not directly correspond with any financial incentives. In addition, while the Medicare DRG system was implemented only in the US, this temporal trend was observed in other countries [115,116] as well.

In our systematic review of the validity of myocardial infarction (MI) diagnoses in administrative data [32], we observed that the accuracy of MI as a cause-of-death on death certificates was generally lower than hospital discharge diagnoses. In contrast, findings from this review suggest that diagnoses of fatal stroke are as accurate, if not more accurate, than diagnoses of non-fatal stroke. One article included in this review, by Tolonen et al [52], suggests this is because autopsies can provide additional information that improves the accuracy of the diagnosis. In that study, the sensitivity and PPV for SAH and ICH were markedly high (sensitivity 100% and 90%, and PPV 100% and 100%, for SAH and ICH, respectively) amongst cases diagnosed from outpatient clinics, and all of these were fatal cases that were autopsied [52]. With these findings, researchers should feel confident that most diagnoses of fatal stroke in administrative data correspond to true stroke deaths. However, our findings do suggest that, individually, these databases have suboptimal sensitivity for detecting fatal stroke. In one paper, 79% of deaths that were listed in the mortality register as being from unspecified cerebrovascular disease were re-classified by the investigating physician as deaths from acute stroke [53]. In another paper, only 21% of confirmed stroke deaths in the vital statistics database appeared in the hospitalization database [51]. Thus, when investigating fatal stroke, researchers could improve the ascertainment of cases by linking vital statistics data with hospitalization data, and attributing to stroke any deaths occurring within a certain period (i.e. 28 days, as used in the WHO MONICA Project [36]) of a hospitalization coded for stroke. Sensitivity analyses investigating the impact of longer periods of time (i.e. 90 days) on case ascertainment should be conducted alongside.

Consequences of Using Broad Sets of Cerebrovascular Disease Codes

Many of the studies that reported on the validity of ICD-9 codes 430–438, or ICD-10 codes I60-69, as a group were examining how well these codes performed for detecting preexisting cerebrovascular disease as a comorbidity. Searching for any one of the codes in this group is appropriate when seeking to identify and adjust for preexisting cerebrovascular disease in the analyses of other clinical conditions. The high sensitivity we observed for these codes, and high PPV they had when compared to the reference diagnosis of ‘any cerebrovascular disease’, provides additional support for this use. However, when studying acute stroke as a primary outcome, this broad group of codes, which includes the codes for non-acute and ill-defined cerebrovascular disease, and the late effects of stroke, should not be used. It is far more difficult to identify risk factors for stroke from this mixture of recent-onset and prevalent cerebrovascular disease because it is unclear which characteristics may have increased the risk of developing stroke, versus surviving the stroke. Instead, in pharmacoepidemiologic studies and other analyses of risk factors where acute stroke is the primary outcome of interest, and diagnostic specificity is of upmost importance, only stroke-specific codes should be used.

Limitations

We acknowledge some limitations to our systematic review. There is the potential for a language bias as we could not consider articles whose full-text was not available in English. We were also conservative in our definition of acute stroke, and excluded studies that only reported on the validity of diagnostic codes for TIA. Another potential limitation stems from the fact that, even though our database searches were conducted by an experienced librarian, administrative databases are not well catalogued in MEDLINE and EMBASE (e.g. no MeSH term pertaining to “administrative database”). Although the majority of the included studies were located through database searches, our subsequent hand search turned up other relevant articles that had not been indexed under terms relating to Administrative Data or Validation. As a result, despite our extensive hand search, we may have missed some relevant articles if they were not indexed in MEDLINE or EMBASE under a term relating to administrative data or validation. Our findings are also subject to publication bias, wherein reports of stroke codes having poor validity may have been differentially withheld from publication. We feel this is unlikely, however, given that we did locate reports of case definitions (i.e. ICD-9 432 or 433 individually) whose sensitivity and PPV for acute stroke were suboptimal.

Conclusions and Recommendations

Following our analysis of the evidence, we conclude that the diagnostic codes for acute stroke in administrative databases are valid. In fact, advances in neuroimaging and the increased availability of CT scanners may have helped improve diagnosis and coding of acute stroke subtypes over time. However, it is apparent that researchers have been using a variety of codes to identify acute stroke, some of which have suboptimal validity. Based on current evidence, we provide researchers with several recommendations for the use of diagnostic codes to capture cases of stroke in administrative data. We believe the findings of our review will help guide researchers in their efforts to better understand and decrease the burden of stroke.

1. 1. As a group, the range of codes for cerebrovascular disease (ICD-9 430-438/ICD-10 I60-I69) has good sensitivity (≥ 82%), specificity (≥ 95%), and PPV (≥ 81%) for identifying the aggregate diagnosis of acute or preexisting cerebrovascular disease.
2. 2. Codes that pertain to the diagnosis of acute stroke (ICD-9 430/ICD-10 I60, ICD-9 431/ICD-10 I61, ICD-9 434/I63, and ICD-9 436/ICD-10 I64) are highly predictive of true cases of acute stroke of any type, and of the particular subtype.
   1. ⚬ These are the codes that should be used when identifying acute stroke as an outcome, especially in pharmacoepidemiologic and other analyses of risk factors where diagnostic specificity is essential.
3. 3. When searching for cases of ischaemic stroke, including both the code for ischaemic stroke (ICD-9 434/ICD-10 I63) and the code for acute-but-ill-defined stroke (ICD-9 463/ICD-10 I64) in the case definition should help capture more cases of ischaemic stroke with little impact on the PPV.
4. 4. Whether identified from hospitalization or vital statistics data, diagnoses of fatal stroke generally correspond to true deaths from stroke.
5. 5. Hospitalization and vital statistics databases should be linked and searched together in order to maximize the capture of stroke deaths.