https://orcid.org/0000-0002-4561-0318
Figures
Abstract
Introduction
Alzheimer’s disease and related dementias (ADRD) affect over 5.7 million Americans and over 35 million people worldwide. Detection of mild cognitive impairment (MCI) and early ADRD is a challenge to clinicians and researchers. Brief assessment tools frequently emphasize memory impairment, however executive dysfunction may be one of the earliest signs of impairment. To address the need for a brief, easy-to-score, open-access test of executive function for use in clinical practice and research, we created the Number Symbol Coding Task (NSCT).
Methods
This study analyzed 320 consecutive patient-caregiver dyads who underwent a comprehensive evaluation including the Clinical Dementia Rating (CDR), patient and caregiver versions of the Quick Dementia Rating System (QDRS), caregiver ratings of behavior and function, and neuropsychological testing, with a subset undergoing volumetric magnetic resonance imaging (MRI). Estimates of cognitive reserve were calculated using education, combined indices of education and occupation, and verbal IQ. Psychometric properties of the NSCT including data quality, data distribution, floor and ceiling effects, construct and known-groups validity, discriminability, and clinical profiles were determined.
Results
The patients had a mean age of 75.3±9.2 years (range 38-98y) with a mean education of 15.7±2.8 years (range 6-26y) of education. The patients had a mean CDR-SB of 4.8±4.7 (range 0–18) and a mean MoCA score of 18.6±7.1 (range 1–30). The mean NSCT score was 30.1±13.8 and followed a normal distribution. All healthy controls and MCI cases were able to complete the NSCT. The NSCT showed moderate-to-strong correlations with clinical and neuropsychological measures with the strongest association (all p’s < .001) for measures with executive components (e.g., Judgement and Problem Solving box of the CDR, Decision Making and Problem Solving domain of the QDRS, Trailmaking B, and Cognigram Attention and Executive Composite Scores). Women slightly outperformed men, and individuals with lower educational attainment and lower education-occupation indices had lower NSCT scores. Decreasing NSCT scores corresponded to older age, worse cognitive scores, higher CDR sum of boxes scores, worse caregiver ratings of function and behavior, worse patient and informant QDRS ratings, and smaller hippocampal volumes and hippocampal occupancy scores. The NSCT provided excellent discrimination (AUC: .866; 95% CI: .82-.91) with a cut-off score of 36 providing the best combination of sensitivity (0.880) and specificity (0.759). Combining the NSCT with patient QDRS and caregiver QDRS ratings improved discrimination (AUC: .908; 95% CI: .87-.94).
Discussion
The NSCT is a brief, 90-second executive task that incorporates attention, planning and set-switching that can be completed by individuals into the moderate-to-severe stages of dementia. The NSCT may be a useful tool for dementia screening, case-ascertainment in epidemiological or community-based ADRD studies, and in busy primary care settings where time is limited. Combining the NSCT with a brief structured interview tool such as the QDRS may provide excellent power to detect cognitive impairment. The NSCT performed well in comparison to standardized scales of a comprehensive cognitive neurology evaluation across a wide array of sociodemographic variables in a brief fashion that could facilitate its use in clinical care and research.
Citation: Galvin JE, Tolea MI, Moore C, Chrisphonte S (2020) The Number Symbol Coding Task: A brief measure of executive function to detect dementia and cognitive impairment. PLoS ONE 15(11): e0242233. https://doi.org/10.1371/journal.pone.0242233
Editor: Linda Chao, University of California, San Francisco, UNITED STATES
Received: September 1, 2020; Accepted: October 28, 2020; Published: November 30, 2020
Copyright: © 2020 Galvin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: A de-identified dataset is available at Open Science Framework DOI (DOI 10.17605/OSF.IO/2CEWK). For questions regarding this dataset, please contact Michael Kleiman, PhD at mjkleiman@med.miami.edu.
Funding: This study was supported by grants from the National Institute on Aging of JEG (R01 AG040211-A1 and R01 NS101483-01A1), the Harry T. Mangurian Foundation, and the Leo and Anne Albert Charitable Trust. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. JEG is the creator of the Number Symbol Coding Task.
Competing interests: JEG is the creator of the NSCT. MIT, CM and SC report no conflicts of interest.
Introduction
Alzheimer’s disease and related dementias (ADRD) currently affect over 5.7 million Americans and over 35 million people worldwide [1]. The number of ADRD cases is expected to increase 3-fold by the year 2050 as the number of older adults is also increasing [1–3]. Community detection of mild cognitive impairment (MCI) and early stages of ADRD is a challenge to clinicians and researchers alike, requiring in-depth evaluations that can be time-consuming. Gold Standard evaluations such as the Clinical Dementia Rating (CDR) [4] and comprehensive neuropsychological testing are used in many research projects but require a trained clinician to administer, interpret, and score the CDR or neuropsychological testing and require an extended period of time with the patient, and in the case of the CDR, an informant. While feasible in research settings (e.g., clinical trials, longitudinal studies), these evaluations may not be practical in primary care settings or epidemiologic case-ascertainment projects [2]. Briefer evaluation tools are often used in these settings. These briefer tools can be grouped into performance-based assessments including the Montreal Cognitive Assessment (MoCA) [5], Mini Mental State Exam [6], or Mini-Cog [7], or interview-based assessments usually with an informant such as the AD8 [8], Informant-Questionnaire in Cognitive Decline in the Elderly [9], or Quick Dementia Rating System (QDRS) [10]. These brief tests are often more heavily weighted towards capturing memory impairment, however other important domains such as attention and executive function may not be well captured. This is unfortunate because alterations in executive problem solving and decision making may be one of the earliest signs of MCI and ADRD [11, 12].
Executive function is a broad construct that captures a number of different aspects including basic functions such as attention, inhibitory control, working memory, set switching, and higher order functions including planning, decision making, and problem solving [13–15]. Several neuropsychological tests characterize executive function and capture declines in individuals with cognitive impairment. Examples include the Digit Symbol Substitution Test from Wechsler Abbreviated Intelligence Scale-Revised (WAIS-R) [16], Stroop Color-Word Test [17] and the Cambridge Neuropsychological Test Automated Battery (CANTAB) [18]. Many of these batteries are lengthy, take expertise to administer, must be interpreted in terms of age and education of the patient, and are proprietary requiring licensing costs. While these tests are in the armamentarium of neuropsychologists, they are not readily accessible to physicians in their office settings or easy to use in community-based research projects. Additionally, some standardized batteries used in large multicenter projects such as the Uniform Data Set (UDS) [19, 20] in the National Institute of Aging Alzheimer Disease Center Program have minimal executive function tasks. To address the need for a brief and easy to score test of executive function for use in clinical practice and in research, we created the Number Symbol Coding Task (NSCT). The goal was to create a brief, valid, easy-to-score, open-access instrument that could discriminate between individuals with and without cognitive impairment capturing attention, problem-solving, and set-switching activities. We examined the NSCT compared with the Gold Standard assessments including the CDR, neuropsychological testing, and neuroimaging.
Materials and methods
Study participants
We evaluated 400 consecutive patient-caregiver dyads attending our center for clinical care or participation in cognitive aging research. During the visit, the patient and caregiver underwent a comprehensive evaluation including the Clinical Dementia Rating (CDR) and its sum of boxes (CDR-SB) [4], physical and neurological examination, assessment of mood, physical performance, neuropsychological testing, and caregiver ratings of patient cognitive abilities, behavior, and function. All components are part of standard of care at our center [21]. A waiver of consent was obtained for retrospective review of clinic patients and research participants provided written informed consent. Assent was obtained from all patients. Capacity to consent was determined by a semi-structured interview between the patient and a study clinician. This study was approved by the University of Miami Institution Review Board.
Development of the Number Symbol Coding Task
The NSCT was developed the lead author and reviewed by the study team. Numbers were selected to represent all of the single digits. Symbols were chosen to be easy to draw through the severe stages of dementia and consisted of simple shapes that could be completed with a maximum of 2 pen strokes. The layout was designed to fit on a single page. The pattern was reviewed by the research team to make sure all numbers and symbols were represented, and that no readily recognizable arrangement could be determined by cognitively healthy controls.
Administration and scoring of the Number Symbol Coding Task
The NSCT is presented in Fig 1. An answer key with 10 numbers corresponding to 10 symbols is provided at the top of the page. Before starting the task, two untimed practice sessions are offered. Practice #1 provides the patient an opportunity to re-code 5 numbers into 5 symbols. If they are able to complete this part, they move on to Practice #2, where they are asked to re-code 5 symbols into 5 numbers. The test then begins with 90 seconds permitted to correctly complete as much of the task as possible. Initially, the patient is presented with a series of numbers to re-code into symbols. In the 17th position, set switching begins with irregular cycles of converting symbols to numbers or numbers to symbols. There are 70 re-coding chances possible, with only correct re-coding counted to give a range of scores 0–70.
Clinical assessment
The clinical assessments were modelled after the UDS 3.0 [19, 20]. The CDR [4] was used to determine the presence or absence of dementia and to stage its severity; a global CDR 0 indicates no dementia; CDR 0.5 represents MCI or very mild dementia; CDR 1, 2, or 3 correspond to mild, moderate, or severe dementia. The CDR-SB was calculated by adding up the individual CDR categories giving a score from 0–18 with higher scores supporting more severe stages. The Global Deterioration Scale (GDS) [22] was used to provide a global cognitive and function stage: a GDS 1 indicates no impairment; GDS 2 indicates subjective cognitive impairment; GDS 3 corresponds to mild cognitive impairment; GDS 4–7 corresponds to mild, moderate, moderate-severe, or severe dementia [22]. Extrapyramidal features were assessed with the Movement Disorders Society-Unified Parkinson’s Disease Rating Scale, motor subscale part III (UPDRS) [23]. The Charlson Comorbidity Index [24] and Functional Comorbidity Index (FCI) [25] were used to measure overall health and medical comorbidities. Global physical performance was captured with the mini Physical Performance Test (mPPT) [26] and frailty was assessed with the Fried Frailty Scale [27]. Vascular contributions to dementia were assessed with the modified Hachinski scale [28]. Consensus diagnoses were determined using standard criteria for MCI [29], AD [30], dementia with Lewy bodies (DLB) [31], vascular contributions to cognitive impairment and dementia (VCID) [32], and frontotemporal degeneration (FTD) [33].
Estimates of cognitive reserve
Cognitive reserve is a latent moderation construct that represents an individual’s ability to maintain cognitive functioning despite the presence of underlying neurodegenerative pathology [34, 35]. While there is no consensus on determinants of cognitive reserve, two of the most important appear to be educational attainment and occupation [35]. Educational attainment was recorded as the number of years of formal schooling. However, the number of years of schooling may not be representative of the quality of the educational experience, and opportunities for advanced education may not be equal across different racial, ethnic and socioeconomic groups [36–40]. The combination of education and occupation was captured by the Hollingshead two-factor index of social status [41], composed of an educational scale (7 levels) and an occupational scale (7 levels) summed to give a social class rating from I-V. This index was used as a proxy for cognitive reserve with Class I representing the highest reserve, II-III representing midlevel reserve, and IV-V representing lowest reserve. Last, verbal IQ was determined with the Test of Premorbid Function (Pearson Assessments, San Antonio, TX) that tests the individual’s ability to read a list of 70 words with atypical or irregular grapheme to phoneme pronunciations and presented as tertiles (<100, 100–120, >120).
Caregiver ratings of patient cognition, function, and behavior
Caregivers completed the informant version of the Quick Dementia Rating System (QDRS) [10] to provide a global rating of cognitive, functional, and behavioral domains. Activities of daily living were captured with the Functional Activities Questionnaire (FAQ) [42]. Dementia-related behaviors were measured with the Neuropsychiatric Inventory (NPI) [43]. Patient daytime sleepiness was assessed with the Epworth Sleepiness Scale (ESS) [44] while daytime alertness was rated on a 1–10 Likert scale anchored by “Fully and normally awake” (scored 10) and “Sleeps all day” (scored 0) [45].
Cognitive assessment
Each patient was administered a 45-minute test battery modeled after the UDS battery used in the NIA Alzheimer Disease Centers [20] and supplemented with additional measures. The psychometrician was unaware of the diagnosis or CDR. The Montreal Cognitive Assessment [5] was used for a global screen. The rest of the battery included: 15-item Multilingual Naming Test (naming) [20]; Animal naming and Letter fluency (verbal fluency) [20]; Hopkins Verbal Learning Task (episodic memory for word lists–immediate, delayed, and cued recall) [46]; Number forward/backward and Months backwards tests (working memory) [20]; Trailmaking A and B (attention, processing and executive) [47]; the Noise Pareidolia Test [48] (visual perception); and King-Devick Test [49] (visual tracking and saccades). Mood was assessed with the Hospital Anxiety Depression Scale [50] providing subscale scores for depression (HADS-D) and anxiety (HADS-A). Additionally, patients completed the self-reported version of the QDRS [10] for a self-rating of cognitive abilities with scores greater than 1.5 supporting cognitive impairment. The NSCT was administered at the time of the cognitive assessment as an additional measure of executive function.
At a separate sitting, individuals completed the CogState Cognigram Brief Battery (CogState Healthcare LLC, Boston, MA), a well-validated computerized assessment for ages 6–99 years that uses playing cards to test 4 cognitive tasks, providing age normative scores [51, 52]. The Cognigram includes a Detection Task for psychomotor function, an Identification Task for visual attention, a One Card Learning Task for visual memory, and an N-Back Task for working memory. The Cognigram produces 2 composite scores (Memory and Executive).
Apolipoprotein E genotyping
Apolipoprotein E (ApoE) genotyping was performed by True Health Diagnostics LLC (Richmond, VA). Six possible allelic combinations were obtained with individuals dichotomized as being ApoE 4 carriers or non-carriers.
Volumetric MRI
A subset of individuals (n = 76) underwent volumetric MRI with NeuroQuant software (CorTechs Labs, San Diego, CA), a FDA-approved automated quantitative analysis of brain MRI images with normative reference data adjusted for age, sex and intracranial volume with high correlation to FreeSurfer [53] and visual assessment [54]. NeuroQuant provides volumes on seven regions of interest: Hippocampus (Bilateral, Right, and Left), Superior and Inferior Lateral Ventricle, Intracranial, Forebrain Parenchyma, Whole Brain, and White Matter Hyperintensities. While hippocampal volume is often used as a predictor of conversion of MCI to AD, hippocampal occupancy (HOC) measures the degree of hippocampal atrophy accounting for volume loss and compensatory inferior lateral ventricle expansion. It is calculated as a ratio of hippocampal volume to the sum of the hippocampal and inferior lateral ventricle volumes in each hemisphere separately, which are then averaged and normalized for age and sex [55]. This measure may aid in differentiation of individuals with congenitally small hippocampi from those with small hippocampi due to a degenerative disorder [55].
Statistical analyses.
Analyses were conducted with IBM SPSS Statistics v26 (Armonk, NY). Descriptive statistics were used to examine patient and caregiver demographic characteristics, informant rating scales, dementia staging, and neuropsychological testing. One-way analysis of variance (ANOVA) with LSD post-hoc tests were used for continuous data and Chi-square analyses were used for categorical data. To assess scale variability, the frequency distribution, range, and standard deviation were calculated, and data were examined for floor and ceiling effects. Kurtosis and skewness statistics were examined to characterize the shape, symmetry and outliers of the distribution. The NSCT was compared with patient and caregiver characteristics, rating scales, and neuropsychological test performance. Multiple comparisons were addressed using the Bonferroni correction.
Construct validity was assessed comparing the mean performance on each Gold Standard measure with the NSCT using Pearson correlation coefficients [10, 56]. Known-group validity was assessed by examining the NSCT scores by sociodemographic variables and dementia etiology [10, 56]. Receiver operator characteristic (ROC) curves were used to assess the ability of the NSCT to discriminate between individuals with and without cognitive impairment. We first discriminated CDR 0 from CDR >0 and repeated analyses discriminating CDR 0 vs 0.5, which is generally the most difficult staging to determine. The ROC curves were then presented using a potential dementia screening paradigm (a) the NSCT alone, (b) the patient reported QDRS alone, (c) combining NSCT and patient QDRS scores, and finally (d) combining NSCT and patient QDRS scores with the informant version of the QDRS. Results are reported as area under the curve (AUC) with 95% confidence intervals (CIs). Finally, we assessed the ability of the NSCT to differentiate stages of cognitive impairment using mean scores with standard deviations and 95% confidence intervals to provide risk profiles for healthy controls, very mild impairment, mild impairment, and moderate impairment.
Results
Sample characteristics
The patients had a mean age of 75.3±9.2 years (range 38-98y) with a mean education of 15.7±2.8 years (range 6-26y) of education, and 38.1% were ApoE 4 carriers. Overall, the sample was 46.9% female with an imbalance of more females (69.2%) in the control group and more males (57.7%) in the cognitively impaired sample (χ2 = 12.7, p = 0.001). The sample was 97.2% White and 2.3% African American, with 14.5% of the sample reporting Hispanic ethnicity. The patients had a mean CDR-SB of 4.8±4.7 (range 0–18) and a mean MoCA score of 18.6±7.1 (range 1–30). The sample covered a range of healthy controls (CDR 0 = 54), MCI or very mild dementia (CDR 0.5 = 161), mild dementia (CDR 1 = 92), moderate dementia (CDR 2 = 64), and severe dementia (CDR 3 = 29). Eighty individuals were unable to perform the NSCT due to cognitive impairment: 0% CDR 0, 0.1% CDR 0.5, 18.5% CDR 1, 57.8% CDR 2, and 82.8% CDR 3. All healthy controls and MCI cases were able to complete the task. Nearly all individuals who were unable to perform the task had ratings of 2 or 3 in the Judgment and Problem Solving domain of the CDR. This gives a final sample size of 320 composed of 53 healthy controls, 120 MCI, 58 AD, 64 DLB, 15 VCID, and 10 FTD cases. The mean NSCT score was 30.1±13.8, with a median of 30. The minimum score was 0 (floor effect: 0.6%) and maximum score was 69 (ceiling effect: 0%) covering nearly the full range of possible scores. Distribution statistics showed skewness was 0.17 (standard error = 0.14) and kurtosis was -0.47 (standard error = 0.27) supporting that the NSCT follows a normal distribution (Fig 2).
This histogram demonstrates that the Number Symbol Coding Task follows a normal distribution with a mean of 30.1, standard deviation of 13.8, and a median of 30.
Construct validity of the Number Symbol Coding Task with clinical measures
Construct validity is demonstrated in Table 1 by examining the strength of association between the NSCT and clinical, functional, behavioral, and informant ratings. The NSCT showed moderate-to-strong correlations with clinical measures but most strongly correlated (all p’s < .001) with age (R = -.511), FAQ (R = -.583), the informant QDRS (R = -.560), GDS (R = -.715), CDR (R = -.659) and CDR-SB (R = -.724). The NSCT correlated with all CDR domains with the Judgment and Problem Solving box (R = -.743) showing the strongest association and the Personal Care box (R = -.407) showing the weakest association.
Construct validity of the Number Symbol Coding Task with cognitive performance measures
Construct validity of the NSCT with measures of neuropsychological test performance, mood, and subjective cognitive complaints is shown in Table 2. The NSCT showed moderate-to-strong correlations with all neuropsychological tests (p < .001) with the strongest associations with Trailmaking B (R = -.728), MoCA (R = .689), Trailmaking A (R = -.685), Animal Naming (R = .668), and HVLT delayed recall (R = .667). A moderate correlation was were found between the NSCT and the patient QDRS (R = -.473) with Decision Making/Problem Solving (R = -.530) showing the strongest association and Mood (R = -.193) showing the weakest association. The NSCT was not associated with ratings of anxiety or depression. The NSCT was then compared to the Cogstate Brief Battery (Cognigram). There were moderate correlations with the Cognigram Attention (R = .451) and Executive Composite (R = .419) scores.
Known-groups validity
The performance of the NSCT was compared between patient age, sex, race, ethnicity, education, cognitive reserve, ApoE status, dementia ratings and etiologies in Table 3. Females scored higher than males (32.1±14.7 vs 28.1±12.3, p = .03) after controlling for imbalance of sexes in the sample. There was no difference in NSCT scores by race and ethnicity, however given the smaller number of African Americans and Hispanics in the sample, these results should be interpreted with caution. There was a significant difference between all age strata in NSCT scores (all post-hoc p’s < .001). There was a significant difference in NSCT by education with post-hoc analyses showing individuals with ≤12 years of education scoring the lowest (post-hoc p < .001) and individuals with the lowest cognitive reserve (i.e., education-occupation tertile) scoring the lowest (post-hoc p < .001). However, when examining Verbal IQ strata, there was no difference in NSCT scores. When comparing patient self-ratings of cognitive function with the QDRS, NSCT scores were significantly lower in those with QDRS ≥2 (p < .001). The NSCT total score decreased with higher CDR stages. Post-hoc analyses demonstrate that NSCT scores for each CDR stage is different from all other CDR stages. For GDS stages, GDS 1 (no impairment) and GDS 2 (subjective cognitive impairment) were not different, and NSCT scores decreased across other GDS stages. Post hoc analyses demonstrated that GDS 3 and 4 were different from all other GDS stages, and GDS 5 and 6 were not different from each other. When classifying individuals by consensus clinical diagnoses, NSCT scores in healthy controls were significantly higher than MCI and all dementia etiologies, while MCI individuals were higher than all dementia etiologies (post-hoc p’s < .001). Within dementia etiologies, DLB had the lowest NSCT scores.
Discriminability of the Number Symbol Coding Task
We tested the ability of the NSCT to discriminate between individuals with and without cognitive impairment using ROC analyses to provide area under the curve (AUC). We first compared healthy controls to individuals with any form of cognitive impairment. The NSCT provided excellent discrimination (AUC: .866; 95% CI: .82-.91) with a cut-off score of 36 providing the best combination of sensitivity (0.880) and specificity (0.759). As detecting the mildest forms of cognitive impairment is the biggest challenge in research and clinical practice, we repeated the analyses to discriminate controls (CDR 0) from those with CDR 0.5 (which includes MCI and very mild dementia). The NSCT provided very good discrimination (AUC: .785; 95% CI: .72-.85). To provide evidence for a brief paradigm for dementia screening, we included the patient reported QDRS as a patient reported outcome, and then combined the patient QDRS with the NSCT (Fig 3). The combined battery improved discrimination between healthy controls and impaired individuals (AUC: .890; 95% CI: .85-.93). Lastly, we repeated the analyses adding in the informant version of the QDRS to provide an independent caregiver rating of global cognitive abilities. The addition of the informant QDRS further increased discrimination (AUC: .908; 95% CI: .87-.94). This brief paradigm takes about 5 minutes (patient QDRS: 2–3 minutes, NSCT: 2 minutes) provides excellent ability to detect cognitive impairment. The addition of the informant QDRS when a caregiver is available does not add additional time since the caregiver can independently complete the 2-3-minute QDRS while the patient completes their evaluation.
ROC curves comparing the discriminability of the NSCT, patient and caregiver forms of the QDRS to differentiate healthy controls (CDR 0) from individuals with any form of cognitive impairment (CDR>0). The combination of the NSCT, a patient-reported outcome (patient QDRS), and informant-reported outcome (caregiver QDRS) correctly classified 90.8% of cases (see details in text). Key: AUC = Area under the curve; QDRS = Quick Dementia Rating System; NSCT = Number Symbol Coding Task.
Comparison of Number Symbol Coding Task with MRI
We examined the relationship between the NSCT and volumetric MRI performed with NeuroQuant (Table 4). NSCT scores positively correlated with hippocampal volume (R = .528, p < .001), hippocampal occupancy scores (R = .630, p < .001) and inversely correlated with superior lateral ventricle volume (R = -.493, p < .001). There was a stronger relationship between NSCT scores with the left hippocampus than with the right hippocampus. Using the cut-off score of 36 from the ROC analyses, we found significant differences in volume of both hippocampi with a greater difference in the left hippocampus, hippocampal occupancy scores, and in the superior and inferior lateral ventricles. Marginal differences were seen in Forebrain Parenchymal and Whole Brain volumes; however, these are not significant after correction for multiple comparisons.
Risk profiles
Lastly, to provide a framework for utilizing the NSCT in a clinical setting, we developed a profile of scores by level of impairment based on consensus clinical diagnoses from No Impairment to Moderate Impairment (Table 5). Mean NSCT scores and 95% confidence intervals are shown with corresponding global staging by CDR and GDS and patient characteristics. Decreasing NSCT scores correspond to older age, lower MoCA scores, higher FAQ, NPI, CDR-SB, patient and informant QDRS ratings, and smaller hippocampal volumes and hippocampal occupancy scores.
Discussion
The NSCT is a brief executive task that incorporates attention, planning and set-switching that can be completed by individuals into the moderate-to-severe stages of dementia. There was very good data quality with a normal distribution and minimal floor and ceiling effects. The NSCT performed equally well across most patient characteristics with women slightly outperforming men and individuals with lower educational attainment and lower education-occupation index scores performing worse. However, there was no difference in NSCT scores with Verbal IQ. There was strong correlation between the NSCT and gold standard measures of cognition, function, and behavior with the strongest association for measures with executive components (e.g., Judgment and Problem Solving box of the CDR, Decision Making and Problem Solving domain of the QDRS, Trailmaking B, and Cognigram Attention and Executive Composite Scores). Number Symbol Coding Task scores declined with greater CDR and GDS staging, and amongst dementia etiologies, individuals with DLB performed worst. A cut-off score of 36 provided the best combination of sensitivity and specificity allowing for the creation of profiles for clinical use. The NSCT correlated with hippocampal and ventricular volumes supporting its relationship with neurodegeneration.
Executive dysfunction in MCI and early stage ADRD may be an important but overlooked construct. Executive dysfunction may have significant impact on activities of daily living and quality of life, perhaps more so than memory impairment [57–60]. We previously demonstrated transitions in performance on traditional executive and attention tasks such as Trailmaking A and B, Block Design, and Visual Retention tests [61] occur up to three years prior to clinical diagnosis of MCI and dementia in individuals who eventually developed AD dementia [12] and Parkinson’s disease dementia [62, 63]. Furthermore, in healthy controls who came to autopsy with evidence of preclinical AD, judgement and problem solving (CDR box domain) and attention tasks (Trailmaking A) were among the first domains to exhibit clinically detectable change [11]. Similar findings of early executive dysfunction have been reported in other studies of ADRD [18, 58, 64, 65]. The challenge for clinicians is that most brief testing instruments have limited ability to test executive function, and when present, are limited to shortened versions of traditional attention-executive tasks such as a brief version of Trailmaking B in the MoCA [5] or a Clock Drawing in the Mini-Cog [7].
Complexity in testing is more likely to bring out deficits with an inability to compensate in individuals with neurodegenerative diseases, particularly if the task taps into basic (e.g., attention, inhibitory control, set switching) and higher order (e.g., planning, problem solving) functions [13]. The NSCT offers this complexity especially with switching back and forth from number-to-symbol coding to symbol-to-number coding. Individuals with MCI and ADRD have significant slowing down of the coding task compared with healthy controls. This set-switching component further differentiates the NSCT from other executive tests such as Digit Symbol Substitution of the WAIS-R [16].
There are several limitations in this study. The NSCT was validated in the context of an academic research setting where the prevalence of MCI and dementia are high, and the patients tend to be highly educated and predominantly White. Validation of the NSCT in other settings where dementia prevalence is lower (i.e. community samples) and the sample is more diverse is needed. As this is a cross-sectional study, the longitudinal properties of the NSCT still need to be elucidated. The clinical profile and cut-off scores were developed in this convenience sample and presented as a guide. Future studies should include a more diverse community population. The majority of cases consisted of MCI, AD, and DLB with fewer VCID and FTD cases. Biomarker examination was limited to ApoE genotypes and MRI. Although NeuroQuant and more commonly used research programs for volumetric analyses (i.e., Freesurfer) are similar [53], the number of regions available from NeuroQuant are limited. This is especially true for analysis of individual cortical volumes as executive tasks are traditionally associated with frontal lobe functioning. Analyses with other specific biomarkers such as amyloid β-protein, tau, or α-synuclein are needed.
Strengths of this study include the use of a comprehensive evaluation that is part of standard of care with extensive characterization of patients and measurement of cognitive, functional, and behavioral constructs using Gold Standard instruments. Another advantage of the NSCT is its brevity being completed in 90 seconds and easy scoring of counting only correct re-coding. The NSCT could be administered regardless of dementia etiology and can be completed by patients through the moderate-to-severe stages of dementia. The NSCT may be a useful tool for dementia screening, case-ascertainment in epidemiological or community-based ADRD studies, and in busy primary care settings where time is limited. Combining the NSCT with a brief structured interview tool such as the QDRS may provide excellent power to detect cognitive impairment. Patients or research participants could then be referred for a more extensive evaluation. The NSCT performed well in comparison to standardized scales of a comprehensive cognitive neurology evaluation across a wide array of sociodemographic variables in a brief fashion that could facilitate its use in clinical care and research.
References
- 1.
Alzheimer Association 2019 Facts and Figures. https://www.alz.org/alzheimers-dementia/facts-figures. Accessed August 20, 2020
- 2. Galvin JE. Using informant and performance screening methods to detect mild cognitive impairment and dementia. Curr Rep Gerontol 2018; 7:19–25. pmid:29963365
- 3. Galvin JE, Tolea MI, Chrisphonte SC. What do older adults do with results from dementia screening. PLoS One 2020;15:e0235534. pmid:32609745
- 4. Morris JC. The Clinical Dementia Rating (CDR): Current version and scoring rules., Neurol, 1993;43:2412–2414 pmid:8232972
- 5. Nasreddine ZS, Phillips NA, Bédirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53:695–699. pmid:15817019
- 6. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. pmid:1202204
- 7. Rosenbloom M, Barclay TR, Borson S, et al. Screening Positive for Cognitive Impairment: Impact on Healthcare Utilization and Provider Action in Primary and Specialty Care Practices. J Gen Intern Med. 2018;33:1746–1751. pmid:30097978
- 8. Galvin JE, Roe CM, Powlishta et al. The AD8: a brief informant interview to detect dementia. Neurology 2005;65:559–594. pmid:16116116
- 9. Jorm AF, Scott R, Cullen JS, MacKinnon AJ. Performance of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) as a screening test for dementia. Psychol Med. 1991;21:785–790 pmid:1946866
- 10. Galvin JE. The Quick Dementia Rating System (QDRS): A rapid dementia staging tool. Alzheimer Dem (DADM) 2015; 1:249–259. pmid:26140284
- 11. Galvin JE, Powlishta KK, Wilkins K, McKeel DW Jr, Xiong C, Grant E, et al. Predictors of preclinical Alzheimer disease and dementia: a clinicopathologic study. Arch Neurol. 2005;62:758–765. pmid:15883263
- 12. Johnson DK, Storandt M, Morris JC, Galvin JE. Longitudinal study of the transition from healthy aging to Alzheimer disease. Arch Neurol. 2009;66:1254–1259. pmid:19822781
- 13. Diamond A. Executive functions. Ann Rev Psychol 2013;64: 135–168
- 14. Chan RC, Shum D, Toulopoulou T, Chen EY. Assessment of executive functions: review of instruments and identification of critical issues. Arch Clin Neuropsychol. 2008;23:201–216. pmid:18096360
- 15. Alvarez JA, Emory E. Executive function and the frontal lobes: A meta-analytic review Neuropsychol Rev 2006;16:17–42 pmid:16794878
- 16. Berger S. The WAIS-R factors: usefulness and construct validity in neuropsychological assessments. Appl Neuropsychol. 1998;5:37–42 pmid:16318465
- 17. Golden CJ. Identification of brain disorders by the Stroop Color and Word Test. J Clin Psychol. 1976;32:654–658 pmid:956433
- 18. Wild K, Howieson D, Webbe F, et al. Status of computerized cognitive testing in aging: A systematic review. Alzheimer’s Dement. 2008;4:428–37 pmid:19012868
- 19. Beekly DL, Ramos EM, Lee WW, et al; NIA Alzheimer's Disease Centers. The National Alzheimer's Coordinating Center (NACC) database: The Uniform Data Set. Alzheimer Dis Assoc Disord. 2007;21:249–258 pmid:17804958
- 20. Weintraub S, Besser L, Dodge HH, et al. Version 3 of the Alzheimer Disease Centers' Neuropsychological Test Battery in the Uniform Data Set (UDS). Alzheimer Dis Assoc Disord. 2018;32:10–17. pmid:29240561
- 21. Galvin JE, Valois L, Zweig Y. Collaborative transdisciplinary team approach for dementia care. Neurodegener Dis Manag. 2014;4:455–469 pmid:25531688
- 22. Reisberg B. Global measures: utility in defining and measuring treatment response in dementia. Int Psychogeriatr. 2007;19:421–456 pmid:17480241
- 23. Goetz CG, Tilley BC, Shaftman SR, et al; Movement Disorder Society UPDRS Revision Task Force. Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov Disord. 2008;23:2129–2170 pmid:19025984
- 24. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383. pmid:3558716
- 25. Groll DL, Heland D, Caeser M, Wright JG. Assessment of long-term physical function in acute respiratory distress syndrome (ARDS) patients: Comparison of the Charlson Comorbidty index and the functional comorbidity index. Am J Phys Med 2006;7:574–581.
- 26. Wilkins CH, Roe CM, Morris JC. A brief clinical tool to assess physical function: the mini-physical performance test. Arch Gerontol Geriatr. 2010;50:96–100. pmid:19282039
- 27. Fried LP, Tangen CM, Walston J, et al, Cardiovascular Health Study Collaborative Research Group. Frailty in older adults: Evidence for a phenotype. J Gerontol A Biol Sci Med Sci 2001;56:M146–156 pmid:11253156
- 28. Rosen WG, Terry RD, Fuld PA, Katzman R, Peck A. Pathological verification of ischemic score in differentiation of dementias. Ann Neurol. 1980;7:486–488. pmid:7396427
- 29. Albert MS, DeKosky ST, Dickson D, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging–Alzheimer’s association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:270–279. pmid:21514249
- 30. McKhann GM, Knopman DS, Chertkow H, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging–Alzheimer’s association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:263–269. pmid:21514250
- 31. McKeith IG, Boeve BF, Dickson DW, et al. Diagnosis and management of dementia with Lewy bodies: Fourth consensus report of the DLB Consortium. Neurology. 2017;89:88–100. pmid:28592453
- 32. Skrobot OA, O'Brien J, Black S, et al. The Vascular Impairment of Cognition Classification Consensus Study. Alzheimers Dement. 2017;13:624–633 pmid:27960092
- 33. Olney NT, Spina S, Miller BL. Frontotemporal Dementia. Neurol Clin. 2017;35:339–374. pmid:28410663
- 34. Stern Y, Barulli D. Cognitive reserve. Handb Clin Neurol. 2019;167:181–190. pmid:31753132
- 35. Chapko D, McCormack R, Black C, Staff R, Murray A. Life-course determinants of cognitive reserve (CR) in cognitive aging and dementia—a systematic literature review. Aging Ment Health. 2018;22:915–926. pmid:28703027
- 36. Vonk JMJ, Arce Rentería M, Avila JF, et al. Secular trends in cognitive trajectories of diverse older adults. Alzheimers Dement. 2019;15:1576–1587. pmid:31672483
- 37. Avila JF, Rentería MA, Jones RN, et al. Education differentially contributes to cognitive reserve across racial/ethnic groups. Alzheimers Dement. 2020 Aug 22. [Online ahead of print] pmid:32827354
- 38. Zahodne LB, Mayeda ER, Hohman TJ, et al. The role of education in a vascular pathway to episodic memory: brain maintenance or cognitive reserve? Neurobiol Aging. 2019;84:109–118. pmid:31539647
- 39. Zahodne LB, Manly JJ, Smith J, Seeman T, Lachman ME. Socioeconomic, health, and psychosocial mediators of racial disparities in cognition in early, middle, and late adulthood. Psychol Aging. 2017;32:118–130 pmid:28287782
- 40. Zahodne LB, Stern Y, Manly JJ. Differing effects of education on cognitive decline in diverse elders with low versus high educational attainment. Neuropsychology. 2015;29:649–657. pmid:25222199
- 41.
Hollingshead AB. Two factor index of social position. New Haven: Yale University Press; 1957.
- 42. Tappen RM, Rosselli M, Engstrom G. Evaluation of the Functional Activities Questionnaire (FAQ) in cognitive screening across four American ethnic groups. Clin Neuropsychol. 2010;24:646–661. pmid:20473827
- 43. Kaufer DI, Cummings JL, Ketchel P, et al. Validation of the NPI-Q, a brief clinical form of the Neuropsychiatric Inventory. J Neuropsychiatry Clin Neurosci. 2000;12:233–239. pmid:11001602
- 44. Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14:540–545. pmid:1798888
- 45. Boeve BF, Molano JR, Ferman TJ, et al. Validation of the Mayo Sleep Questionnaire to screen for REM sleep behavior disorder in a community-based sample. J Clin Sleep Med. 2013;9:475–480. pmid:23674939
- 46. Shapiro AM, Benedict RH, Schretlen D, Brandt J. Construct and concurrent validity of the Hopkins Verbal Learning Test-revised. Clin Neuropsychol. 1999;13:348–358. pmid:10726605
- 47. Reitan RM, Validity of the trail making test as an indication of organic brain damage, Perceptual and Motor Skills, 1958;8:271–276.
- 48. Yokoi K, Nishio Y, Uchiyama M, Shimomura T, Iizuka O, Mori E. Hallucinators find meaning in noises: pareidolic illusions in dementia with Lewy bodies. Neuropsychologia. 2014;56:245–254. pmid:24491313
- 49. Oride MK, Marutani JK, Rouse MW, DeLand PN. Reliability study of the Pierce and King-Devick saccade tests. Am J Optom Physiol Optics 1986;63:419–424 pmid:3728637
- 50. Snaith RP. The Hospital Anxiety and Depression Scale. Health Qual Life Outcomes. 2003;1;1:29.
- 51. Buckley R.F., Sparks K.P, Papp K.V., Dekhtyar M., Martin C., Burnham S. Computerized cognitive testing for use in clinical trials: a comparison of the NIH toolbox and Cogstate C3 batteries. J Prev Alzheimers Dis. 2017;4:3–11 pmid:29188853
- 52. Darby DG, Pietrzak RH, Fredrickson J, et al. Intraindividual cognitive decline using a brief computerized cognitive screening test. Alzheimers Dement 2012;8:95–104. pmid:22404851
- 53. Ross DE, Ochs AL, Tate DF, Tokac U, Seabaugh J, Abildskov TJ, et al. High correlations between MRI brain volume measurements based on NeuroQuant(®) and FreeSurfer. Psychiatry Res Neuroimaging. 2018;278:69–76. pmid:29880256
- 54. Persson K, Barca ML, Cavallin L, Brækhus A, Knapskog AB, Selbæk G, et al. Comparison of automated volumetry of the hippocampus using NeuroQuant® and visual assessment of the medial temporal lobe in Alzheimer's disease. Acta Radiol. 2018;59:997–1001. pmid:29172642
- 55. Heister D, Brewer JB, Magda S, Blennow K, McEvoy LK. Predicting MCI outcome with clinically available MRI and CSF biomarkers. Neurology 2011;77:1619–1628. pmid:21998317
- 56.
Streiner DL, Norman GR. Health measurement scale: a practical guide to their development and use. 4th ed. Oxford, England: Oxford University Press; 2008.
- 57. Kuzmickiene J, Kaubrys G. Specific features of executive dysfunction in Alzheimer = type mild dementia based on computerized Cambridge Neurospychological Test Automated Battery (CANTAB) test results. Med Sci Monit 2006;22:3605–3613.
- 58. Pfitzenmeyer P, Mourey F, Manckoundia P, d’Athis P. A 4-year follow-up of very old patients presenting with frontal-subcortical dysfunction compared with Alzheimer’s disease patients. Gerontology 2005;51:62–65. pmid:15591758
- 59. Vidoni ED, Honea RA, Burns JM. Neural correlates of impaired functional independence in early Alzheimer’s disease. J Alzheimers Dis. 2010;19:517–527. pmid:20110598
- 60. Pereira FS, Yassuda MS, Oliveira AM, et al. Executive dysfunction correlates with impaired functional status in older adults with varying degrees of cognitive impairment. Int Psychogeriatr. 2008;20:1104–1115. pmid:18752698
- 61. Johnson DK, Storandt M, Morris JC, Langford ZD, Galvin JE. Cognitive profiles in dementia: Alzheimer disease vs healthy brain aging. Neurology 2008;71:1783–1789. pmid:19029518
- 62. Johnson DK, Galvin JE. Longitudinal changes in cognition in Parkinson's disease with and without dementia. Dement Geriatr Cogn Disord. 2011;31:98–108. pmid:21242691
- 63. Johnson DK, Langford Z, Garnier-Villarreal M, Morris JC, Galvin JE. Onset of Mild Cognitive Impairment in Parkinson Disease. Alzheimer Dis Assoc Disord. 2016;30:127–133. pmid:25850732
- 64. Baudic S, Barba GD, Thibaudet MC, et al. Executive function deficits in early Alzheimer’s disease and their relations with episodic memory. Arch Clin Neuropsychol. 2006;21:15–21. pmid:16125364
- 65. Logie R, Cocchini G, Della Sala S, Baddeley A. Is there a specific executive capacity for dual task coordination? Evidence from Alzheimer’s disease. Neuropsychology. 2004;18:504–513. pmid:15291728