SPoRT derivation and validation cohorts

The derivation cohort consisted of 82 259 Ontario household respondents between the ages of 20 and 83 years from the combined 2001, 2003 and 2005 Canadian Community Health Surveys (CCHS [cycles 1.1, 2.1, and 3.1]), conducted by Statistics Canada.[16] The validation cohort consisted of respondents to the 2007/2008 CCHS survey (cycle 4.1).

These surveys, which used a multistage stratified cluster design that represented 98% of the Canadian population over the age of 12 years, attained an average response rate of 80.5%. The surveys were conducted through telephone and in-person interviews and all responses were self-reported. The details of the survey methods have been previously published.[16]

Consenting CCHS respondents who did not self-report a prior history of stroke were followed until incident stroke event, death, loss to follow-up (defined as loss of health care eligibility), or March 31, 2012. To ascertain stroke events, the CCHS respondents were individually linked to three population-based databases: 1) hospitalization records from the Canadian Institute for Health Information Discharge Abstract Database, 2) vital statistics (for cause of death—available only until Dec 31, 2009); and, 3) ambulatory physician records from the Ontario Health Insurance Program. Stroke events were ascertained using validated diagnostics codes and criteria. For hospitalized stroke, there was a 92% agreement between discharge diagnoses of stroke and chart reviews.[17] For stroke diagnosed in the hospital or community, the sensitivity was 68% and specificity 98.9%.[18] Diagnostic codes for stroke included TIA (unless otherwise specified) and followed the Canadian Stroke Network definition (ICD-9 codes: 362, 430, 431, 434, 435, 436; and ICD-10 codes: G45, H340, H34.1, I60, I61, I63, I64 excluding I608, I636, and G454 for most-responsible hospital diagnosis or underlying cause of death).[19] Stroke in the community setting were ascertained using similar ambulatory physician diagnoses (see Tu et al.[18] for details).

Across the three surveys, 99 929 Ontario CCHS respondents consented to health care follow-up. Respondents were excluded if they did not provide a valid universal health insurance program number (required for data linkage; n = 302), had suffered a stroke before the survey (n = 1 462), or were not aged between 20–83 years (n = 15 390). If a respondent was included in more than one CCHS cycle (n = 516), only their earliest survey response was included. The validation cohort consisted of 28 605 respondents after applying the same exclusion criteria (health insurance number n = 107; previous stroke n = 580; age n = 4 822; previous CCHS cycle n = 524).

Risk factors for stroke

We selected and examined the association between incidence of stroke and each of the following risk factors: age, sex, four health behaviours (smoking, alcohol consumption, diet, and physical activity), stress, sociodemographic factors (ethnicity, immigration status, income [individual and family], education [individual and highest family education], neighbourhood deprivation), chronic conditions (self-report of physician-diagnosed diabetes, coronary heart disease, and hypertension), and body mass index (calculated from self-reported height and weight). See S1 Table for definitions of the risk factors considered.

Model development

The primary outcome was incident stroke, resulting in hospitalization (study end-date March 31, 2012). There were five secondary outcomes: i) death from stroke; ii) death or hospitalization from stroke (study end-date for these two outcomes is December 31, 2009, reflecting the most recently available cause-specific mortality data); iii) hospitalized ischemic stroke only; iv) hospitalized stroke excluding TIA; and, v) stroke diagnosed in the community setting by a physician or resulting in a hospitalization. To increase statistical power, the secondary outcomes were assessed by combining the derivation and validation cohorts.

We used a Cox proportional hazards model to test the significance of each potential risk factor on the hazard of incident stroke. A competing risk approach was used for all analyses: all-cause death as a competing risk in the primary analyses and non-stroke death in the secondary analyses.[20–22] Time to stroke was calculated as the number of days from survey administration to admission date for incident stroke hospitalization or stroke death. Each exposure variable was centered on the cohort mean.

We created the models for males and females separately using a pre-specified stepwise approach that began with age, followed by health behaviours, sociodemographic indices, intermediate risk factors (such as body mass index) and proximal risks (such as self-reported diabetes, hypertension, and heart disease). Variables were added considering their ability to improve discrimination and calibration (as described below).

We included age with time interaction to address the proportional hazard assumption of traditional Cox models and to allow risk estimation for different follow-up times. We assessed age as a predictor using several different categorical and continuous forms, including spline functions.

We created an index that summarized behavioural risk factors to reflect the study’s focus on these factors. Typically, predictive risk indices are created after model development to facilitate interpretation by the general user. We generated the index of behavioural risk factors—called the SPoRT Behaviour Score—during model development to increase statistical and discriminating power when examining multiple behavioural risk factors and categories.[23] This process also supported the creation of a model structure that lessened the potential for intermediate and proximal risk factors to reduce the association between behavioural risk factors and stroke.[24–27] For example, we would expect a reduced effect size of diet and physical activity if BMI and diabetes were simultaneously included in the model without considering that BMI and diabetes are risk factors on the causal pathway between health behaviours and stroke.

The SPoRT Behaviour Score was created through the following steps. First, the hazards for individual risk factors were examined using a reference group of respondents with the most favourable behaviour for all risk factors. Age-adjusted hazards for each risk factor and exposure category were rank-ordered and scores assigned based on the estimated hazard ratios. The scores were then rounded to integer values while maintaining the initial rank order of hazards to minimize the difference in observed versus predicted number of events—overall and in predefined subgroups (see Assessment of predictive accuracy)—while preserving the initial rank-order of the respective risk factor scores.[23]

Next, we added intermediate and proximal risk factors to the model, assessing hazards and improvement in predictive accuracy. We assessed interaction terms, focusing on age and behavioural risks as well as interaction between behavioural risk factors. (see S2 Table for details).

The prevalence of missing values was less than 5% for any variable. In order to estimate a SPoRT Behaviour Score for each subject, missing values for behavioural risk factors were imputed based on mean values for the respondent’s age, sex and local health region.[28] Missing values for other risk factors were maintained as separate categories to allow future application of the algorithm for other similar population health surveys.

Assessment of predictive accuracy

We sought to develop a predictive algorithm that was both well calibrated and discriminating, with an emphasis on calibration for behavioural risks and use in the community setting.[29]

Calibration is the ability of an algorithm’s predictive estimates to closely approximate observed risk or to correctly rank subjects' risk.[30] We compared predicted to observed risk for the overall population, as well as across predefined subgroups (67 subgroups for males and 65 for females) identified as being important to clinicians and policy actors through a structured consultation process.[31] Calibration subcategories included: all behavioural risk categories, deciles of risk, age groups, health planning regions, sociodemographic groups, body mass index, hypertension status, and diabetes status. We predefined an important difference in calibration as a relative difference of greater than 20% between observed and predicted estimates for those categories with more than 5% of total stoke cases.[31]

Discrimination is the ability to differentiate individuals at high risk from those at low risk.[30] We assessed the C-statistic and 75:25 and 95:5 risk percentile ratios for survival data with time-dependent covariates.[32] Further details of the methods are provided in S2 Table.[33]

Implications for public health, community and clinical prevention

The SPoRT algorithm complements other approaches to stroke risk assessment by informing public health planners, patients and clinicians about the contribution of health behaviours. Clinical guidelines from the World Health Organization and most countries recommend a graded approach to cardiovascular disease prevention that includes interventions with low individual cost targeting the entire population combined with individual therapy tailored to risk levels.[34, 35] A graded prevention approach is best accompanied by a graded assessment of cardiovascular risk, which starts with simple and accessible assessment of as wide a target population as possible, followed by progressively more intensive risk assessment to discriminate among individuals with progressively less prevalent (but clinically important) risk factors. Ideally, each stage of risk assessment supports corresponding interventions for that setting.

In the public health setting, where risk assessment involves use of population health surveys to ascertain risk exposure and population diffusion of risk, multivariable risk algorithms have been shown to be the most discriminating approach.[5] Our study suggests that risk of stroke can be discriminately assessed using population health surveys and multivariable risk algorithm; and, that stroke risk is concentrated in the elderly and in groups with multiple risk factors.

In the general population, risk assessment is performed by individuals in the community and focuses on health behaviours and other risk factors that are common, contribute to a large burden of disease, and are modifiable in the community setting. That is not to say that SPoRT should replace other clinical algorithms that include measurement of blood pressure and lipids. Rather, we suggest a graded approach to risk assessment that begins in the community setting and focuses on health behaviours. In the primary care setting, risk stratification includes blood pressure, lipids and other more detailed risk information, which potentially improves risk stratification and supports decision-making about medication. In the speciality setting, progressively more intensive risk assessment corresponds to more intensive treatment options.

Opportunities in public health and international settings

Assessing population risk is useful for planning purposes, including predicting future disease incidence and assessing the effectiveness of community-wide prevention strategies.[5] Few jurisdictions have population data that contains the clinical and biophysical measures required for application of clinic CVD risk algorithms. However, many jurisdictions have self-reported health surveys that could be used to estimate risk using SPoRT or similar risk algorithms. Furthermore, SPoRT’s population-based focus enables several approaches for validation, recalibration and application that are not typically available to clinical risk algorithms.[5] For example, population health surveys from other countries can be used to recalibrate SPoRT based on the population-specific prevalence and distribution of risk factors. SPoRT risk estimates can be further calibrated by adjusting predicted population estimates against observed population stroke incidence.[36]

Current study in perspective

To our knowledge, SPoRT is the only cardiovascular risk algorithm that can be applied to population health surveys. As well, we are aware of only one other cardiovascular algorithm that includes all major behavioural risks.[9] History of hypertension, diabetes, and heart disease are included in SPoRT but only as self-reported measures rather than clinical measures or confirmed diagnosis. Despite this constraint, SPoRT has predictive accuracy as high, if not higher, than risk algorithms that rely on clinical measures.[4]

We purposefully emphasized the role of health behaviours over proximal risks, such as hypertension and diabetes, to facilitate prevention. By first including behavioural risks and summarizing these risks as the SPoRT Behaviour Score we created a simple hierarchical structure that preserves the contribution of behaviours to stroke risk. If our sole purpose was predicting stroke, rather than predicting stroke based on health behaviours, we would likely have found that behavioural risks have little additional prognostic ability over a smaller selection of traditionally included proximal risks (e.g., measured blood pressure). Furthermore, our emphasis on calibration informs how well SPoRT performs in assessing stroke risk based on behaviours.

There are other cardiovascular indices that summarize preventable risks, such as the index of Ideal Cardiovascular Health (ICH) developed by the American Health Association.[35] Uniquely, SPoRT can express individual health behaviours as either a relative or absolute stroke risk versus a categorical scale (see S6 Table for a comparison of SPoRT and ICH). Despite stroke being a leading cause of morbidity, many people in community setting will likely interpret large relative differences in stroke risk (over 500% relative difference in stroke risk across people of the same age) differently knowing that the baseline risk of stroke is low (1.09% 5-year risk in our Ontario population, 5 to 95% range 0.03 to 4.62%).

Limitations

The chief limitation of this study is potential misclassification error resulting from the exclusive use of self-reported risks and routinely-collected stroke data. While more accurate risk factor ascertainment could improve discrimination and calibration, SPoRT already has a high discrimination and favourable calibration. Other studies have also found that chronic diseases can accurately be assessed using self-reports. Gaziano et al. and Qiao et al. showed there are only modest classification differences when CVD risk assessment is performed with and without clinical and laboratory measures.[37, 38] As well, there are many diabetes risk algorithms developed to ascertain risk outside the clinic setting using only self-reported measures.[39] Furthermore, the most influential risk factors in SPoRT are extensively used and validated world-wide: there have been favourable studies for self-reported smoking status validated against urine cotinine levels, and heart disease, hypertension and diabetes validated against physician diagnoses.[40, 41] Thus, using physician-diagnosed disease or urine test for smoking would not improve stroke risk discrimination or accuracy. Similarly, self-reported height and weight were used to estimate BMI. Validation studies for the CCHS have confirmed a modest misclassification of self-reported BMI compare to measured BMI and correction factors are available.[42] We did not use those correction factors for two reasons: first, modest reclassification of BMI will have a small influence on predictive risk, given the BMI risk occurred only at high BMI levels (BMI 35+). Second, the main indented use of SPoRT is for population health surveys without measured BMI and self-reported use in the community setting. This means that our use of self-reported BMI is consistent in both development, validation and application, thus ensuring appropriate calibration. Regardless, it will be important to assess SPoRT in other external populations—particularly since our external validation population was similar to the derivation population.

There is a greater degree of misclassification error for alcohol consumption, physical activity and diet; however, it is reassuring that we found the measures used in our study are discriminating and have a similar association with stroke as seen in other studies. For alcohol, there are concerns that self-reports considerably underreport consumption. That stated, there is consistent evidence of a “J” shape relationship of hazards that was replicated in our study.[43] Physical activity has modest self-reported ascertainment accuracy compared to accelerometer measures, with about half of the respondent of self-reported surveys accurately reporting their activity and others equally over- and under-reporting activity up to 30 minutes per day.[44]

Diet, likely the most challenging CVD risk to ascertain using brief self-reports, is important to consider in risk assessment for at least three reasons: there is a clear and important relationship between diet and CVD; a high proportion of people in many countries have poor diet quality; and, diet is potentially modifiable with corresponding improvement in CVD risk.[45, 46] Increasingly, there is emphasis to ascertain overall diet quality rather than specific food types or nutrients. General population health surveys, such as the one used for our study cohort, use fruit and vegetable consumption as a proxy for overall diet quality. While there is modest over-report of fruit and vegetable consumption compared to repeated 24-hour food recall, there is good rank-order correlation between those that have high or low consumption of fruit and vegetables and overall diet quality.[46] Similar to previous studies examining all-cause mortality and all-cause hospitalization, we found that high potato and fruit juice consumption were hazardous for stroke risk and accordingly modified a brief dietary quality index.[47, 48] There is the potential for brief diet quality indices to have poor generalizability across jurisdictions. However, given the favourable predictive accuracy and a hazard that corresponds with the diet/CVD relationship seen in other studies, we believe that our study demonstrates the utility of brief self-reported diet questions for CVD risk assessment. Future studies should validate our (and other) brief diet quality measure for risk prediction.

The large study cohort and use of routinely-collected data meant that it was not feasible to individually verify the stroke events. However, we used identification approaches that have been shown to accurately ascertain stroke events.[17, 18] Moreover, we examined five different stroke outcomes using three different databases, with SPoRT showing equal predictive accuracy regardless of stroke endpoint. As expected, there was a small trend toward a lower hazard for the SPoRT Behaviour Score with a stroke definition that was broader and more heterogeneous.

Finally, our approach to create a hierarchical structure for the predictive algorithm, through the creation of the behavioural index, has limitations due to the early examination of the hazards of each behavioural risk. Current recommendations for prognostic algorithms recommend a more rigorous pre-specified approach that minimizes examination of outcome relationships when making decisions about predictor selection and form. In general, we had a high adherence to recommended algorithm development (see S2 Table) but allowed ourselves to deviate from recommendations, recognizing that development of algorithms for the population setting differs from the more common development in the clinical setting.[5, 49] For example, the large sample size and power of our study should reduce the risk of type 1 error compared to most clinical algorithms that use a much smaller sample of respondents. That said, in the future we plan to disaggregate the task of prognosis from etiognosis by developing a purely prognostic algorithm (ignoring causal pathways) and then separately perform analyses to estimate a hazard of risks from a causal perspective.[50]

Conclusion

Stroke risk can be accurately predicted solely using information on health behaviours and other self-reported risks. SPoRT does not require clinical or laboratory data, making it well-suited for application using population health surveys as well as easy to implement for general population use in the community setting. The focus on health behaviours further facilitates patient-centred and population approaches for stroke and cardiovascular disease prevention.