http://orcid.org/0000-0002-4771-2916
Figures
Abstract
Objective
The on-road assessment is the gold standard because of its ecological validity. Yet existing instruments are heterogeneous and little is known about their psychometric properties. This study identified existing on-road assessment instruments and extracted data on psychometric properties and usability in clinical settings.
Method
A systematic review identified studies evaluating standardized on-road evaluation instruments adapted for people with cognitive impairment. Published articles were searched on PubMed, CINHAL, PsycINFO, Web of Science, and ScienceDirect. Study quality and the level of evidence were assessed using the COSMIN checklist. The collected data were synthetized using a narrative approach. Usability was subjectively assessed for each instrument by extracting information on acceptability, access, cost, and training.
Results
The review identified 18 published studies between 1994 and 2016 that investigated 12 different on-road evaluation instruments: the Performance-Based Driving Evaluation, the Washington University Road Test, the New Haven, the Test Ride for Practical Fitness to Drive, the Rhode Island Road Test, the Sum of Manoeuvres Score, the Performance Analysis of Driving Ability, the Composite Driving Assessment Scale, the Nottingham Neurological Driving Assessment, the Driving Observation Schedule, the Record of Driving Errors, and the Western University’s On-road Assessment. Participants were mainly male (64%), between 48 and 80 years old, and had a broad variety of cognitive disorders. Most instruments showed reasonable psychometric values for internal consistency, criterion validity, and reliability. However, the level of evidence was poor to support any of the instruments given the low number of studies for each.
Conclusion
Despite the social and health consequences of decisions taken using these instruments, little is known about the value of a single evaluation and the ability of instruments to identify expected changes. None of the identified on-road evaluation instruments seem currently adapted for clinical settings targeting rehabilitation and occupational priorities rather than road security alone.
Citation: Bellagamba D, Vionnet L, Margot-Cattin I, Vaucher P (2020) Standardized on-road tests assessing fitness-to-drive in people with cognitive impairments: A systematic review. PLoS ONE 15(5): e0233125. https://doi.org/10.1371/journal.pone.0233125
Editor: Sergio A. Useche, Universitat de Valencia, SPAIN
Received: December 24, 2019; Accepted: April 28, 2020; Published: May 18, 2020
Copyright: © 2020 Bellagamba et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data are now available on Zenodo (https://zenodo.org/record/3687014#.XqxMoqgzZPY).
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Societal changes such as urbanization, improved transport infrastructures, and increase in offers of geographically concentrated activities and commodities [1] has raised the need for effective and safe community mobility. Community mobility supports social participation [1, 2] and out-of-home activities, thus maintaining a sense of belonging, connections, and social roles [3] for community-dwelling older adults. In developed countries, motorized individualized vehicles remain the mean of transport used by most of the population [4]. Driving is also valued for its sense of independence, freedom, and competence [5, 6]. An ageing population and an increasing incidence of chronic conditions lead to more and more people living with physical, perceptual, or cognitive deficits where fitness-to-drive is compromised [7]. This growing population meets security challenges that might result in driving cessation. However, driving cessation can be seen as a health issue in itself [8], leading to identity changes (role changes), decreased self-esteem and sense of control [9], a reduction of outside-home activities [10], social isolation [11], and loss of independence [12]. A meta-analysis by Chihuri et al. [12] showed that driving cessation doubled the risks of depressive symptoms (OR = 1.91, 95% CI 1.61–2.27). Driver status is also considered as a predictor of mortality and entry into long-term care facilities among older adults [13; 14]. Finally, the transportation needs resulting from driving cessation are a predictor of caregivers burden [15]. Therefore, it is important to rely on accurate and equitable methods to assess fitness-to-drive [16] and to provide support for mobility transition for those who will face driving cessation [17].
Due to multiple interactions between individuals and their environment, driving is a complex activity [7]. It involves physical, cognitive, and perceptual (proprioceptive, visual, and auditory) skills that require fast cognitive processing and decision-making, as well as appropriate metacognition abilities to compensate difficulties [7; 18]. Michon's model breaks down cognitive load requirements for driving [19]. The strategic level requires abilities for trip planning as well as individual perceptions related to comfort. Cognitive processing is high and time pressure is low. Tactical level refers skills for maneuvering the vehicle, such as managing right-of-way, overtaking, or accepting gaps. The cognitive load is less important than at the strategic level, but the time pressure is higher. Finally, the operational level has the lowest cognitive demand but the highest time pressure for managing automated tasks (e.g., motor coordination or visual search) [19].
Health deficits affecting driving performance, such as dementia or stroke, require specific assessment approaches. While the impact of physical disorders on driving ability is often easier to compensate with vehicle adaptation, cognitive deficits are a challenge when assessing fitness-to-drive [20]. Furthermore, general practitioners mention cognitive disorders as the first reason to justify their decision to withdraw the driver’s license in 64% of cases [21], without always documenting the true effects of disorders on driving skills.
Different means are available to assess fitness-to-drive: off-road tests of prerequisites (physical, perceptual, and cognitive), simulator-based assessments, and on-road assessments. In the presence of cognitive impairments, neuropsychological tests are not considered a good predictor of fitness-to-drive [22]. It is thus advisable not to base decision-making solely on off-road testing, but also to conduct an assessment of on-road driving performance in case of doubt [16].
For this purpose, driving simulators can be used. This technology standardizes the procedure and controls a number of variables (e.g., traffic density and user behavior) [23]. This method also allows avoiding risky situations in traffic [24]. However, it is costly, and its acceptability to older adults is questionable (sickness simulation and lack of users’ familiarity with technology) [24, 25]. The use of the simulator is also questionable in case of visual-perceptive disorders due to the two-dimensional representation of driving [18]. Finally, the ecological validity of simulators is questionable [26].
In terms of ecological validity, the on-road assessment is therefore considered the gold standard [27]. This assessment usually includes a closed course (e.g., in a parking lot) that allows the operational level to be safely assessed before entering traffic [18]. Depending on the construction of the instrument, the open road evaluation then allows investigation of the three levels of skills and control from Michon's model [28]. However, many variables influence driving, including weather conditions, users’ behavior, road conditions, etc. The inability to control some variables makes it difficult to standardize this type of evaluation [16]. Psychometric and clinimetric values of on-road assessment methods need to be accounted for. To our knowledge, there are no systematic reviews investigating the added value of on-road assessments on the clinical decisional process for recommending driving cessation. Before starting the review, systematic review protocols on this subject were searched on PROSPERO database. No ongoing studies were found.
Method
The aim of this systematic review is to identify and describe psychometric and clinimetric values for existing standardized on-road tests adapted for people with cognitive impairment due to acquired brain injury, dementia, or age-related disorders. The review also aims to describe costs, training requirements, accessibility, and usability of each instrument for future practical implementation. A protocol was recorded on PROSPERO (registration number CRD42018103276).
Selection criteria
Articles meeting the following criteria were included: (a) assessments used with people with suspected or objective cognitive impairment related to acquired brain injury, dementia, or age; (b) on-road assessment; (c) standardized instruments; (d) original articles including a form of validation of the assessment (identifiable in the title or abstract); (e) articles written in English or French.
Excluded were articles comprising (a) a simulator based assessment; (b) a first driving license test; (c) the use of highly specialized equipment (high costs, low reproducibility, etc.).
Search strategy
The search for articles was conducted on the following databases: PubMed, CINAHL Complete via EBSCO, PsycINFO via Ovid, Web of Science (core collection), and ScienceDirect (inception—January 2018). Grey literature has not been explored, as this study focuses on available on-road assessments.
A pilot phase was carried out on PubMed to refine the equation until it could identify already known articles [29]. Three categories of keywords have been defined (assessment, driving, and cognitive impairment). To foster a conservative search, the category "cognitive disorders" was not systematically used. No additional limits were used. The same literature search was conducted at the end of the data extraction phase (January 2019) on PubMed, CINHAL and Web of Science. Full search is available at the following link: http://doi.org/10.5281/zenodo.3687014.
References of the selected articles were consulted (search backward) and the articles citing those selected were screened (search forward) for additional studies [30]. Authors of the selected articles were contacted for additional papers.
Study selection
Titles have been simultaneously screened by two reviewers (LV and DB) against the inclusion criteria. The abstracts were then independently screened by two reviewers (LV and DB). Following the removal of duplicates, the full-text articles were assessed for eligibility against selection criteria. Any disagreements that arose between the reviewers at each stage of the study selection process have been resolved through discussion or with a third reviewer (PV). Reasons for exclusion of full-text articles were documented.
Data extraction
Data extraction forms were created by two authors (LV and DB) and validated by a third author (PV). The extracted data include characteristics of the selected studies, characteristics of each identified on-road assessment, their psychometric and clinimetric properties, and characteristics for implementation.
The type of extracted data was defined by COnsensus-based Standards for the selection of health Measurment Instruments (COSMIN) manual [31] and included authors and year of publication, participant characteristics (number, age, gender, health condition, eligibility criteria), context, description of the test (materials, evaluators and process), and participant scores. The characteristics of the tests include: the name of the test and citation of the studies, the target population, the distance and duration of the course, the design of the road (closed and/or open road and difficulty), the items (number, categories and description), the rating system, the availability of a cut-off score, changes made between several versions, and the available versions (language) [31–33]. The psychometric and clinimetric properties of the identified on-road assessments were extracted according to the COSMIN taxonomy [34]. Finally, the implementation characteristics were cost, accessibility (i.e. how to obtain them), prerequisites (e.g. training required), and acceptability [31]. The latter represents the users' perspective on the relevance of the content and context of the test [35].
Data were extracted by two independent reviewers (LV and DB) using the standardized data extraction forms after a pilot phase on four articles. Any disagreements that arose between the reviewers during the data extraction process have been resolved through discussion or with a third reviewer (PV). Authors of selected articles have been contacted to request missing or additional data for clarification, when required.
Assessment of the risk of bias and the quality of evidence
Assessments of the risk of bias and the quality of evidence in the selected studies were conducted using the COSMIN checklist. Although this tool was originally developed for systematic reviews of Patient-Reported Outcome Measures (PROM), it can be used for Clinician-Reported Outcome Measures (ClinROM) [31, 36]. The evaluation of the quality of the studies was carried out in three stages: (a) assessment of the risk of bias by psychometric property by article; (b) assessment of the risk of bias by psychometric property by instrument (if more than one article); and (c) assessment of the quality of evidence by psychometric property by instrument, using a Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach adapted by COSMIN. These three steps were carried out jointly by two authors (LV and DB). A third author (PV) reviewed these evaluations.
Data analysis
It was anticipated that the heterogeneity of the identified on-road assessments would make it difficult to group the data for a meta-analysis. A narrative synthesis supported by tables was therefore carried out [37–39].
Assessment of publication bias
Assessing the risk of publication bias in studies on measurement properties is complicated because of a lack of registry for such studies [31]. Thus, the assessment of the risk of publication bias was not carried out.
Results
The literature search identified 5,463 records (Fig 1). Following the titles and abstracts screening and the removal of duplicates, 64 full-text articles were retrieved. Following additional search strategies (search backward, search forward, and emails to authors) 28 full-text articles were added. Thus, 92 full-text articles were assessed for eligibility and 18 met selection criteria. No additional studies were selected after the last literature search in January 2019. The study selection process is presented in the PRISMA flow diagram in Fig 1 [40].
Of these 18 articles, 12 different on-road tests were identified: the Performance-Based Driving Evaluation (PBDE) [41], the Washington University Road Test (WURT) [42], the New Haven [43], the Test Ride for Practical Fitness to Drive (TRIP) [44, 45], the Rhode Island Road Test (RIRT) [46, 47], the Sum of Manoeuvres Score (SMS) [48, 49], the Performance Analysis of Driving Ability (P-Drive) [25, 50–52], the Composite Driving Assessment Scale (CDAS) [47], the Nottingham Neurological Driving Assessment (NNDA) [53], the Driving Observation Schedule (DOS) [54], the Record of Driving Errors (RODE) [55], and the Western University’s On-road Assessment (UWO) [56, 57]. Their name, the citations of the related articles, and a summary of their characteristics are in Table 1.
All but two evaluation instruments have been developed for English-speaking countries. The articles were published between 1994 and 2016. The target populations are mainly people with dementia (5 of 12) and people aged 60 and over with different levels of cognitive functioning (4 of 12). One assessment tool was intended for people who have had a stroke, one for people with multiple sclerosis, and finally one for people with cognitive disorders of various etiologies. All tests take place on open roads and six of them start with a closed course. The route is predefined for ten on-road assessments and two take place in the participants’ ecological environment (DOS and CDAS). The test lasts between 45 and 60 minutes except for the DOS and the CDAS (31 minutes and 4 hours filmed over two weeks respectively) [47, 54]. This information is not available for two tests. The distance covered during the tests averaged 23.5 kilometers (ET = 10.5; 9.6–40) for nine of them. Three data were missing. A cut-off score is available for two instruments: It allows a dichotomization for the SMS [49] and a trichotomization for the P-Drive [50]. Details of items and scoring system are available on Zenodo (http://doi.org/10.5281/zenodo.3687014) under Appendix 1 (Items and scoring system).
The implementation characteristics (Table 2), with the exception of the acceptability of two on-road assessments [41, 54], required contact with the authors. Data for five instruments are not available in the absence of a response from the authors. Two instruments are available on the Internet (WURT and NNDA), one by contact with the author (TRIP), one after training (P-Drive) and one in a journal (CDAS). Access has not yet been specified for the remaining instruments. Three on-road assessments are free of charge (WURT, CDAS, and NNDA). The price has not been established for the TRIP and RODE, and is 800€ for the P-Drive (training included). An online training is being developed for another on-road test (RODE). Finally, it is recommended to have training as a specialist in driving rehabilitation for five on-road assessments (TRIP, CDAS, NNDA, RODE, and UWO) and as an occupational therapist for one (WURT).
The characteristics of the selected studies are presented in Table 3. Participants, whose average proportion of men is 64.28% (ET = 14.5; 40–87%), are on average 70.8 years old (ET = 8.7; 48–80.2). The average study sample size is 66 (SD = 50; 6–205). A dual control vehicle was used in nine studies, without specifying the type of gearbox. A dual control vehicle with automatic transmission was used in five studies. Participants were given a choice between their private vehicle and a dual control vehicle in one study. Three studies opted for the use of participants' vehicles. Information is missing in two studies.
Table 4 lists the psychometric properties investigated in the included articles: In addition to structural validity, internal consistency, inter-rater reliability, criterion validity, and construct validity, properties related to reassessment such as responsiveness or test-retest reliability were only poorly investigated (none and twice respectively). These psychometric properties were assessed for risk of bias using the COSMIN checklist and a detailed assessment of the quality of the evidence using the GRADE approach. Details of these assessments are available on Zenodo (http://doi.org/10.5281/zenodo.3687014) under Appendix 2 (Risk of bias) and Appendix 3 (Summary of Findings). Fig 2 illustrates the quality of the evidence of psychometric properties by assessment (colors) in a synthetic way, as well as the evaluation of the aggregated results by psychometric property against the criteria for good measurement properties defined in the COSMIN checklist (+, -, ±, ?) [38]. The aggregate results are sufficient for 18 psychometric properties, undetermined for eight, insufficient for three, and inconsistent for one. The inconsistency of the criterion validity for the P-Drive can be explained by the fact that the three studies exploring criterion validity include different target populations and different expertise level of the evaluators. No quality of evidence of a psychometric property was rated as high (A), four were rated as moderate (B), five as low (C) and 21 as very low (D). This low quality is partly explained by the sample size (-1 to the quality of the evidence if 100≥n≥50 and -2 if n<50) and by the indirectness. The latter refers to a different target population than the one of this systematic review. Indeed, most tests, except the P-Drive, have a relatively narrow target population.
1 +: sufficient; - : insufficient; ?: indeterminate; ±: inconsistent; 2 GRADE: .
Discussion
In this systematic review, 12 on-road tests were identified to assess the fitness-to-drive in people with cognitive impairment. These on-road tests are heterogeneous with regard to their components and few have been the subject of several validation studies. However, no test seems to really stand out from the others in terms of the quality of the evidence, particularly because of limited sample sizes. It is important to consider that this concerns all research on driving, because of the high costs in this field: Statistical power suffers from limited sample sizes [58].
As health conditions are potentially progressive, it is important not to decide on fitness-to-drive on a single assessment [20]. Health conditions can be transitory, episodic, or permanent. To carry out this reassessment, the test used should have good psychometric properties in terms of test-retest reliability, responsiveness, measurement stability over time, and change detection [59]. Among the tests identified in this systematic review, only test-retest reliability was assessed in two on-road tests (WURT and SMS) [42, 48].
On-road assessment allows for exploring the three dimensions of Michon's model: strategic, tactical, and operational [28]. Tactical and operational dimensions are systematically explored during an on-road assessment as they relate to vehicle maneuvering. However, depending on the construction of the test, the strategic dimension is not necessarily investigated. Only UWO, CDAS, and DOS explore the strategic dimension through a task of planning of a route [47, 54, 56, 57]. It seems important to investigate all dimensions of driving and not only those related to vehicle maneuvers. Indeed, the strategic dimension influences driving safely by answering the questions of when, where, with whom, and how, among others.
Combined with reassessment, the assessment of fitness-to-drive could allow the identification of self-regulatory behaviors. These represent changes in driving behaviors (e.g., highway avoidance) that promote safe driving in cases of functional limitations, discomfort, or lack of self-confidence [60]. Self-regulation may happen in tactical, strategic, or life-goal aspects (e.g., choice of place to live in relation to one's occupation or purchase of another vehicle). Thus, the identification of self-regulatory behaviors depends on the dimensions investigated by the on-road test (tactical and strategic according to Michon's model). Since the tactical dimension is systematically explored, the identification of self-regulatory strategies at this level is accessible. This makes it possible to identify the adoption of behaviors such as avoiding distracting elements when driving (e.g., radio), or a better gap acceptance [60]. Since only three on-road tests assess the strategic dimension, work is needed to facilitate the identification of self-regulatory strategies at this level.
A cut-off score is available for two on-road tests (P-Drive and SMS) [49–51]. However, only the P-Drive allows for trichotomization of the evaluation, i.e., the classification of the people assessed in three categories (fit / doubtful as to their fitness-to-drive / unfit): Two studies were conducted and showed two different cut-off scores allowing dichotomization; it was suggested to use them as limits defining the gray zone, thus allowing a trichotomization [50]. This gray area is important as it is a gateway for interventions [32, 33]. These may be aimed at maintaining driving performance and anticipating of managing a mobility transition following driving cessation [17].
Several studies have allowed participants to use their private vehicles [47, 52, 54] or have given them a choice [53], suggesting some ecological validity. In addition, two tests are conducted in an ecological environment (CDAS and DOS) [47, 54]. According to the Person-Environment-Occupation-Performance Model (PEOP), occupational performance emerges from the interaction between the person, their environment, and the occupation [61]. The use of standardized roads and the instructors' professional vehicle ignores the potential influence of the ecological environment on performance. In this context, the ecological validity of the assessment is questionable [54]. Indeed, familiarity with the environment while driving reduces cognitive load, particularly at the attention level, as suggested in a simulator study [62]. Familiarity also promotes spatial orientation [63]. In addition, the use of the instructors' professional vehicle results in a reduction of performance [64]. However, familiarity with the road increases distractibility while driving [65] as well as reaction time [66]. In addition, risk perception and ability to respect speed regulation are reduced [67, 68]. All ages combined, the majority of accidents occur on familiar roads [69]. However, these results are from studies that are not limited to participants with cognitive impairment. Studies suggest that tactical self-regulation is influenced by the level of familiarity with the environment in which the evaluation takes place. This could raise safety issues for evaluations that do not cover unknown environments.
Furthermore, the choice of the vehicle could have an influence on safety during the test. In the event of a serious error, the absence of dual control could impede on the evaluator’s ability to intervene quickly [33]. In this context, beginning the evaluation on a closed-course could allow an assessment of the operational dimension of driving beforehand. This ensures sufficient control over safety before entering traffic [28]. However, this component of the test was not identified in all the tests [44, 45, 48, 49, 53], including those conducted in an ecological setting [47, 54].
The involvement of two people, a driving instructor to ensure safety and usually an occupational therapist for the scoring, was very common. As confirmed by other studies, only three instruments, the CDAS, the New Haven, and the RIRT relied on a single person [43, 46, 47]. However, the added value of having a second evaluator was never properly studied. The shortage of qualified occupational therapists in some countries, such as Switzerland [70], could question this practice. Resources could be optimized by having occupational therapists focus on activity analysis, community mobility, and mobility transition [71]. Given the almost perfect inter-rater reliability between occupational therapists and instructors identified in two studies [42, 57], the latter appear to represent a potential way to address the shortage of therapists in this domain. According to Nilsen [72], another resource to consider in the implementation of an evaluation instrument is the accessibility to appropriate training. One training course is available for the P-Drive in Scandinavia and another is under development for the RODE. The WURT does not require any specific training, but simply to follow the protocol.
Even though systematic review protocols were searched before our study, a systematic review concerning the reliability and validity of on-road driving tests has recently been published [73]. They identified 21 on-road tests. This difference can be explained by the different eligibility criteria: They included studies not restricted to people with cognitive impairment, and studies in which the validation process couldn’t be identified in the title or abstract. The authors also found that validation studies mainly focused on the inter-rater reliability, that measurement error and responsiveness were not investigated despite their importance. However, these psychometric properties are of great importance, as explained previously and as mentioned as well by Sawada et al. [73].
Given the limitations of on-road tests concerning their psychometric properties and their components’ variability, these tests are not as promising as expected and costly in terms of financial and human resources. However, it is necessary to have reliable, valid and specific instruments in order to support decision making concerning the withdrawal of the driving licence, its retention or its restriction [74]. A less expensive and potentially effective alternative would be the use of cognitive tests to assess fitness-to-drive [75]. A systematic review exploring the relationship between cognitive tests and on-road driving performance in people with dementia has been conducted [75]. It appears that composite batteries of cognitive tests are more appropriate to predict on-road driving performance than cognitive tests focused on a single cognitive ability. Though, these composite batteries are not sufficiently validated: Cut-off scores enabling trichotomization would be useful. One specific test, the Useful Field Of View (UFOV©) appears to be a potential predictor of driving fitness-to-drive [76]. This instrument assesses cognitive domains (selective attention, divided attention and processing speed) in three tasks [76]. However, there are many different versions of this instrument and some authors have modified it for their studies [77]. For this reason, the interpretation of the results is more complex than it appears. In addition, the UFOV© possibly measures visual functions that are not directly related to driving. Indeed, in addition to measuring processing speed and different types of attention, it involves visual functions such as visual acuity. The latter is not a predictor of fitness-to-drive [77]. As visual acuity decreases with age, the UFOV© score decreases and this does not necessarily reflect a poorer driving performance. Thus, it seems important to have age-related normative values for UFOV© [78].
Another alternative would be the use of simulators: They allow to evaluate complex behaviors in a controlled environment when these behaviors might not be safe, practical nor ethical during an on-road test [79]. They must be immersive, sufficiently challenging and have complex scenarios in order to emulate accurately and consistently the real-world performance [79]. Technological advances have made it possible to make simulators much more dynamic by improving visual quality, for example, but also by being able to control traffic, include other road users and modify the driving environment in real time according to the behavior of the person being assessed [79]. These authors stress the importance of the simulator’s choice, as it is widely used in practice and research and, depending on the simulator, it can lead to biased interpretation. There are few validation studies on simulators. The authors suggest that the simulator does not systematically represent a valid assessment of driving performance [79]. Each simulation setting is unique and must be valid, which can lead to a very important financial investment: Simulators are therefore much more expensive than they seem.
Finally, according to Wynne et al. [79], the perception of risk in simulators is also reduced: The engagement in risky situations is higher and compliance with traffic regulations is reduced. Simulator’s driving performance is influenced by this absence of stress, which can be present during an on-road test. In addition, the simulator does not allow the evaluation of situations where familiarity with the environment could have an impact on on-road driving performance [79].
What does this study add
The choice of components (ecological environment, private vehicle, evaluation of the strategic dimension) when constructing the instrument is of great importance as they influence performance and safety during the evaluation.
- This study brings elements regarding implementability characteristics in addition to the psychometric properties of the instruments.
- This systematic review was specifically focused in on-road tests for people with cognitive impairment with a comparison of their psychometric properties and components.
- Familiarity’s influence must be explored in order to guide on-road tests’ elaboration.
Recommendations for practice and research
In conclusion, none of the methods for assessing fitness-to-drive appears to be ideal and none of them alone appears to be sufficient. Thus, these different tests could be combined in order to fuel the decision making process regarding the withdrawal, restriction or retention of the driving licence. The choice of an evaluation instrument remains an important concern: It is necessary to use valid instruments. In sum, off-road tests can be used to identify situations requiring further assessment. On-road or simulator-based performance tests could support decision making in ambiguous situations. In this regard, Sawada et al. consider WURT, P-Drive and TRIP as potential gold standards [73]. In the present study, WURT, P-Drive, TRIP, RIRT, SMS and New Haven seem to present the best results in terms of summarized results and quality of the evidence, the others being dismissed out-of-hand because of their weak results or quality of the evidence. Of these instruments, the P-Drive seems to stand out from the others: As mentioned previously, it allows trichotomization, a training is available, its target population is larger than in the other instruments and it is the most studied instrument among the articles selected in this systematic review. Nevertheless, limitations remain: The strategic dimension is not explored as well as the responsiveness, the measurement error and the test-retest reliability. When used in other languages than Swedish or English, a transcultural adaptation would be necessary to make this assessment (and the training) accessible. Finally, its implementation is limited by the shortage of occupational therapists. Specific interdisciplinary regional training programs for occupational therapists and driving instructors could facilitate the implementation of improved methods for evaluating fitness-to-drive in people with cognitive impairment.
There seems to be an association between environmental familiarity and driving performance. However, more studies on this subject are needed to determine the relevance of conducting a driving assessment in an ecological environment at the expense of road standardization and vice versa. In addition, this association should be explored in people with cognitive impairments. In this context, there is also a need for studies to evaluate the responsiveness of evaluation instruments to known changes in health conditions, and to develop methods to distinguish driving lapses and errors due to health conditions from those due to other causes. A possible solution could be to develop instruments that rely on at least three separate driving phases: evaluation, intervention, and re-evaluation.
Finally, it is relevant to provide evidence of the added value or necessity of the involvement of occupational therapists during the on-road assessment, as that has become the norm in many countries.
Limitations of the study
As some tests were initially developed on simulators (e.g. P-Drive), it is possible that some validation studies were not selected due to the defined selection criteria. Some data concerning the psychometric properties may therefore be missing. In addition, different steps of the systematic review were carried out jointly for educational purposes, which may limit the quality of the methodology.
As the choice of information sources is limited to five databases, it is possible that some on-road assessments available from institutional or government reports have not been included. The same applies to tests developed in doctoral theses, for example. Finally, test-retest and inter-rater reliability were not differentiated when assessing the risk of bias using the COSMIN checklist. The readability of the results may thus be affected.
Conclusion
This systematic review identified 12 on-road evaluation instruments adapted for people with cognitive impairment. When compared with recommendations from the scientific literature, these instruments do not comply to scientific standards for medical diagnosis procedures. Following a single-step evaluation procedure, risks of falsely recommending driving cessation, which might lead to important health consequences, could still be present. This is particularly the case for most evaluation methods that do not provide the opportunity to express uncertainty in the results. Trichotomization is of great importance as it favors interventions that could help maintain driving, or preconize a transition to anticipate driving cessation and facilitate mobility transition.
Acknowledgments
This article is based on the results of a Master Thesis conducted within the joint Master of Science (MSc) in Health Sciences HES-SO (University of Applied Sciences and Arts Western Switzerland) and University of Lausanne (UNIL), major in Occupational Therapy, at HES-SO Master.
References
- 1. Polgar J. Enabling Community Mobility is an Opportunity to Practice Social Occupational Therapy. Canadian Journal of Occupational Therapy. 2011;78(2):67–9. pmid:21560910
- 2. Mulry CM, Piersol CV. The Let’s Go Program for Community Participation: A Feasibility Study. Physical & Occupational Therapy In Geriatrics. 2014;32(3):241–54.
- 3. Fristedt S, Björklund A, Wretstrand A, Falkmer T. Continuing Mobility and Occupational Participation Outside the Home in Old Age Is an Act of Negotiation. Activities, Adaptation & Aging. 2011;35(4):275–97.
- 4. Buys L, Snow S, van Megen K, Miller E. Transportation behaviours of older adults: An investigation into car dependency in urban Australia: Transportation behaviours of older adults. Australasian Journal on Ageing. 2012;31(3):181–6. pmid:22950590
- 5. Eisenhandler SA. The Asphalt Identikit: Old Age and the Driver’s License. The International Journal of Aging and Human Development. 1990;30(1):1–14. pmid:2407663
- 6. Vrkljan BH, Polgar JM. Linking Occupational Participation and Occupational Identity: An Exploratory Study of the Transition from Driving to Driving Cessation in Older Adulthood. Journal of Occupational Science. 2007;14(1):30–9.
- 7. Boot WR, Stothart C, Charness N. Improving the Safety of Aging Road Users–A Mini-Review. Gerontology. 2014;60(1):90–6. pmid:23989044
- 8. Rosso AL, Taylor JA, Tabb LP, Michael YL. Mobility, disability, and social engagement in older adults. J Aging Health. 2013;25(4):617–637. pmid:23548944
- 9. Liddle J, Turpin M, Carlson G, McKenna K. The Needs and Experiences Related to Driving Cessation for Older People. British Journal of Occupational Therapy. 2008;71(9):379–88.
- 10. Marottoli RA, de Leon CFM, Glass TA, Williams CS, Cooney LM, Berkman LF. Consequences of Driving Cessation: Decreased Out-of-Home Activity Levels. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 2000;55(6):S334–40. pmid:11078110
- 11. Mezuk B, Rebok GW. Social integration and social support among older adults following driving cessation. J Gerontol B Psychol Sci Soc Sci. 2008;63(5):S298–303. pmid:18818450
- 12. Chihuri S, Mielenz TJ, DiMaggio CJ, Betz ME, DiGuiseppi C, Jones VC, et al. Driving Cessation and Health Outcomes in Older Adults. J Am Geriatr Soc. 2016;64(2):332–41. pmid:26780879
- 13. Edwards JD, Perkins M, Ross LA, Reynolds SL. Driving status and three-year mortality among community-dwelling older adults. J Gerontol A Biol Sci Med Sci. 2009;64(2):300–5. pmid:19181711
- 14. Freeman EE, Gange SJ, Muñoz B, West SK. Driving Status and Risk of Entry Into Long-Term Care in Older Adults. Am J Public Health. 2006;96(7):1254–9. pmid:16735633
- 15. Razani J, Kakos B, Orieta-Barbalace C, Wong JT, Casas R, Lu P, et al. Predicting Caregiver Burden from Daily Functional Abilities of Patients with Mild Dementia. J Am Geriatr Soc. 2007;55(9):1415–20. pmid:17767684
- 16. Mazer , Laliberté M, Hunt M, Lemoignan J, Gélinas I, Vrkljan B, et al. Ethics of Clinical Decision-Making for Older Drivers: Reporting Health-Related Driving Risk. Canadian Journal on Aging. 2016;35(S1):69–80. pmid:27117942
- 17. Ross LA, Schmidt EL, Ball K. Interventions to maintain mobility: What works? Accident Analysis & Prevention. 2013;61:167–96. pmid:23083492
- 18. Mathias JL, Lucas LK. Cognitive predictors of unsafe driving in older drivers: a meta-analysis. Int Psychogeriatr. 2009;21(4):637–53. pmid:19470197
- 19.
Michon JA. A Critical View of Driver Behavior Models: What Do We Know, What Should We Do? In: Evans L, Schwing RC, editors. Human Behavior and Traffic Safety [Online]. Boston, MA: Springer US; 1985. p. 485–524. http://link.springer.com/10.1007/978-1-4613-2173-6_19
- 20.
Canadian Council of Motor Transport Administrators (CCMTA). Determining driver fitness in Canada. [Online]. 2017. [cited 11 oct 2019]. http://ccmta.ca/images/publications/pdf/CCMTA_Medical_Standards_Dec_1_2015_final_clean_copy.pdf
- 21. Sebo P, Vaucher P, Haller D, Huber P, Mueller Y, Favrat B. What is the profile of older drivers considered medically unfit to drive? A cross-sectional survey in Western Switzerland. Swiss Medical Weekly. 2018;148:w14684. pmid:30449019
- 22. Bennett JM, Chekaluk E, Batchelor J. Cognitive Tests and Determining Fitness to Drive in Dementia: A Systematic Review. J Am Geriatr Soc. 2016;64(9):1904–17. pmid:27253511
- 23. Reger MA, Welsh RK, Watson GS, Cholerton B, Baker LD, Craft S. The Relationship Between Neuropsychological Functioning and Driving Ability in Dementia: A Meta-Analysis. Neuropsychology. 2004;18(1):85–93. pmid:14744191
- 24. Nef T, Bieri R, Müri RM, Mosimann UP. Non-illness-related factors contributing to traffic safety in older drivers: a literature review. Exp Aging Res. 2015;41(3):325–60. pmid:25978449
- 25. Patomella A-H, Tham K, Johansson K, Kottorp A. P-drive on-road: internal scale validity and reliability of an assessment of on-road driving performance in people with neurological disorders. Scand J Occup Ther. 2010;17(1):86–93. pmid:19565409
- 26. Iverson DJ, Gronseth GS, Reger MA, Classen S, Dubinsky RM, Rizzo M, et al. Practice parameter update: evaluation and management of driving risk in dementia: report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 2010;74(16):1316–24. pmid:20385882
- 27. Barco PP, Baum CM, Ott BR, Ice S, Johnson A, Wallendorf M, et al. Driving Errors in Persons with Dementia. J Am Geriatr Soc. 2015;63(7):1373–80. pmid:26140521
- 28. Fox GK, Bowden SC, Smith DS. On-road assessment of driving competence after brain impairment: review of current practice and recommendations for a standardized examination. Arch Phys Med Rehabil. 1998;79(10):1288–96. pmid:9779686
- 29. McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80. pmid:15685278
- 30. Cooper C, Booth A, Varley-Campbell J, Britten N, Garside R. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol. 2018;18:1–14.
- 31. Prinsen Ca. C, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–57. pmid:29435801
- 32. Korner-Bitensky N, Gélinas I, Man-Son-Hing M, Marshall S. Recommendations of the Canadian consensus conference on driving evaluation in older drivers. Physical & Occupational Therapy in Geriatrics. 2005;23(2/3):123–44.
- 33.
Kowalski K, Tuokko H. On-road driving assessment of older adults: a review of the literature. [Online]. 2007. [cited 11 oct 2019]. https://www-files.jibc.ca/research/pdf/On-RoadDrivingOlderAdults_LiteratureReview.pdf
- 34. Terwee CB, Prinsen Ca. C, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159–70. pmid:29550964
- 35. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Services Research. 2017;17(1):88. pmid:28126032
- 36. Mokkink LB, de Vet HCW, Prinsen Ca. C, Patrick DL, Alonso J, Bouter LM, et al. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures. Qual Life Res. 2018;27(5):1171–9. pmid:29260445
- 37. Collins NJ, Prinsen CAC, Christensen R, Bartels EM, Terwee CB, Roos EM. Knee Injury and Osteoarthritis Outcome Score (KOOS): systematic review and meta-analysis of measurement properties. Osteoarthritis and Cartilage. 2016;24(8):1317–29. pmid:27012756
- 38. Prinsen CAC, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”–a practical guideline. Trials. 2016;17(1):449. pmid:27618914
- 39. Verbeek J, Ruotsalainen J, Hoving JL. Synthesizing study results in a systematic review. Scandinavian Journal of Work, Environment & Health. 2012;38(3):282–90. pmid:22015561
- 40. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. pmid:19621072
- 41. Odenheimer G, Beaudet M, Jette A, Albert M, Grande L, Minaker K. Performance-Based Driving Evaluation of the Elderly Driver—Safety, Reliability, and Validity. J Gerontol. 1994;49(4):M153–9. pmid:8014389
- 42. Hunt LA, Murphy CF, Carr D, Duchek JM, Buckles V, Morris JC. Reliability of the Washington University road test—A performance-based assessment for drivers with dementia of the Alzheimer type. Arch Neurol. 1997;54(6):707–12. pmid:9193205
- 43. Richardson ED, Marottoli RA. Visual attention and driving behaviors among community-living older persons. Journal of Gerontology: Medical sciences. 2003;58(9):832–36.
- 44. Akinwuntan AE, DeWeerdt W, Feys H, Baten G, Arno P, Kiekens C. Reliability of a road test after stroke. Archives of Physical Medicine & Rehabilitation. 2003;84(12):1792–6.
- 45. Akinwuntan AE, De Weerdt W, Feys H, Baten , Arno P, Kiekens C. The validity of a road test after stroke. Archives of Physical Medicine & Rehabilitation. 2005;86(3):421–6. pmid:15759222
- 46. Brown LB, Ott BR, Papandonatos GD, Sui Y, Ready RE, Morris JC. Prediction of On-Road Driving Performance in Patients with Early Alzheimer’s Disease. J Am Geriatr Soc. 2005;53(1):94–8. pmid:15667383
- 47. Ott BR, Papandonatos GD, Davis JD, Barco PP, Ott BR, Papandonatos GD, et al. Naturalistic validation of an on-road driving test of older drivers. Human Factors. 2012;54(4):663–74. pmid:22908688
- 48. Justiss MD, Mann WC, Stav W, Velozo C. Development of a behind-the-wheel driving performance assessment for older adults. Topics in Geriatric Rehabilitation 2006;22(2):121–8.
- 49. Shechtman O, Awadzi KD, Classen S, Lanford DN, Joo Y. Validity and critical driving errors of on-road assessment for older drivers. American Journal of Occupational Therapy. 2010;64(2):242–251. pmid:20437911
- 50. Patomella A-H, Bundy A. P-Drive: Implementing an Assessment of On-Road Driving in Clinical Settings and Investigating Its Internal and Predictive Validity. American Journal of Occupational Therapy. 2015;69(4):1–8. pmid:26114466
- 51. Selander H, Lee HC, Johansson K, Falkmer T. Older drivers: On-road and off-road test results. Accid Anal Prev. 2011;43(4):1348–54. pmid:21545864
- 52. Vaucher P, Di Biase C, Lobsiger E, Margot-Cattin I, Favrat B, Patomella A-H. Reliability of P-drive in occupational therapy following a short training session: A promising instrument measuring seniors’ on-road driving competencies. Br J Occup Ther. 2015;78(2):131–9.
- 53. Lincoln NB, Taylor J, Radford KA. Inter-rater reliability of the Nottingham Neurological Driving Assessment for people with dementia—a preliminary evaluation. Clin Rehabil. 2012;26(9):836–9. pmid:22531717
- 54. Vlahodimitrakou Z, Charlton JL, Langford J, Koppel S, Di Stefano M, Macdonald W, et al. Development and evaluation of a driving observation schedule (DOS) to study everyday driving performance of older drivers. Accid Anal Prev. 2013;61:253–60. pmid:23639887
- 55. Barco PP, Carr DB, Rutkoski K, Xiong C, Roe CM. Interrater reliability of the record of driving errors (RODE). American Journal of Occupational Therapy. 2015;69(2):1–6.
- 56. Classen S, Krasniuk S, Alvarez L, Monahan M, Morrow SA, Danter T. Development and Validity of Western University’s On-Road Assessment. OTJR. 2016;37(1):14–29. pmid:27744272
- 57. Classen S, Krasniuk S, Knott M, Alvarez L, Monahan M, Morrow S, et al. Interrater reliability of Western University’s on-road assessment. Can J Occup Ther. 2016;83(5):317–25. pmid:28050933
- 58. Hird MA, Egeto P, Fischer CE, Naglie G, Schweizer TA. A Systematic Review and Meta-Analysis of On-Road Simulator and Cognitive Driving Assessment in Alzheimer’s Disease and Mild Cognitive Impairment. J Alzheimers Dis. 2016;53(2):713–29. pmid:27176076
- 59.
Polit DF, Beck CT. Nursing research: generating and assessing evidence for nursing practice. Philadelphia: Wolters Kluwer; 2017.
- 60.
Molnar LJ, Eby DW, Zhang L, Zanier N, St. Louis RM, Kostyniuk LP. Self-regulation of driving by older adults: a synthesis of the literature and framework for future research. [Online]. 2015. https://pdfs.semanticscholar.org/5288/c9fbd86d50ba580acdc47f5c53488f694323.pdf
- 61.
Baum CM, Christiansen CH, Bass JD. The person-environment-occupation-performance (PEOP) model. In: Christiansen CH, Baum CM, Bass JD, editors. Occupational therapy: performance, participation, and well-being 4th ed. Thorofare, NJ: SLACK Incorporated; 2015. p. 49–56.
- 62. Charlton SG, Starkey NJ. Driving on familiar roads: Automaticity and inattention blindness. Transportation Research Part F: Traffic Psychology and Behaviour. 2013;19:121–33.
- 63. Payyanadan RP, Sanchez FA, Lee JD. Influence of Familiarity on the Driving Behavior, Route Risk, and Route Choice of Older Drivers. IEEE Transactions on Human-Machine Systems. 2018;49(1):10–9.
- 64. Lundberg C, Hakamies-Blomqvist L. Driving tests with older patients: effect of unfamiliar versus familiar vehicle. Transportation Research Part F: Traffic Psychology and Behaviour. 2003;6(3):163–73.
- 65. Wu J, Xu H. The influence of road familiarity on distracted driving activities and driving operation using naturalistic driving study data. Transportation Research Part F: Traffic Psychology and Behaviour. 2018;52:75–85.
- 66. Yanko MR, Spalek TM. Route familiarity breeds inattention: A driving simulator study. Accident Analysis & Prevention. 2013;57:80–6. pmid:23643937
- 67. Colonna P, Intini P, Berloco N, Ranieri V. The influence of memory on driving behavior: How route familiarity is related to speed choice. An on-road study. Safety Science. 2016;82:456–68.
- 68. Rosenbloom T, Perlman A, Shahar A. Women drivers’ behavior in well-known versus less familiar locations. Journal of Safety Research. 2007;38(3):283–8. pmid:17617237
- 69. Burdett BRD, Starkey NJ, Charlton SG. The close to home effect in road crashes. Safety Science. 2017;98:1–8.
- 70.
IUMSP. Soutien aux proches aidants: analyse de la situation et mesures requises pour la Suisse. [Online]. 2019. https://www.iumsp.ch/fr/node/5895
- 71. Unsworth CA, Baker AM, So MH, Harries P, O’Neill D. A systematic review of evidence for fitness-to-drive among people with the mental health conditions of schizophrenia, stress/anxiety disorder, depression, personality disorder and obsessive compulsive disorder. BMC Psychiatry. 2017;78(2):109–20. pmid:28859696
- 72. Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science. 2015;10(1):53. pmid:25895742
- 73. Sawada T, Tomori K, Hamana H, Ohno K, Seike Y, Igari Y, et al. Reliability and validity of on-road driving tests in vulnerable adults: a systematic review. International Journal of Rehabilitation Research. 2019;42:289–99. pmid:31469673
- 74. Hird MA, Egeto P, Fischer CE, Naglie G, Schweizer TA. A systematic review and meta-analysis of on-road simulator and cognitive driving assessment in Alzheimer’s disease and mild cognitive impairment. Journal of Alzheimer’s Disease. 2016;53:713–29. pmid:27176076
- 75. Bennett JM, Chekaluk E, Batchelor J. Cognitive tests and determining fitness to drive in dementia: a systematic review. Journal of the American Geriatrics Society. 2016;64(9):1904–17. pmid:27253511
- 76. Ringer RV, Throneburg Z, Johnson AP, Kramer AF, Loschky LC. Impairing the useful field of view in natural scenes: tunnel vision versus general interference. Journal of Vision. 2019;16(2):1–25. pmid:27050950
- 77. Wood JM, Collins MJ, Chaparro A, Marszalek R, Carberry T, Lacherez P, et al. Differential effects of refractive blur on day and nighttime driving performance. Investigative Ophtalmology & Visual Science. 2014;55:2284–89. pmid:24618322
- 78. Woutersen K, van den Berg AV, Boonstra FN, Theelen T, Goossens J. Useful field of view test performance throughout adulthood in subjects without ocular disorders. 2018;13(5):1–18. pmid:29715316
- 79. Wynne RA, Beanland V, Salmon PM. Systematic review of driving simulator validation studies. Safety Science. 2019;117:138–51.