Accessible Data
Abstract
Background
Developmental disorders (DD), including autism spectrum disorder (ASD) and intellectual disability (ID), are a common group of clinical manifestations caused by a variety of genetic abnormalities. Genetic testing, including chromosomal microarray (CMA), plays an important role in diagnosing these conditions, but CMA can be limited by incomplete coverage of genetic abnormalities and lack of guidance for conditions rarely seen by treating physicians.
Methods
We conducted a longitudinal, randomized controlled trial investigating the impact of a higher resolution 2.8 million (MM) probe-CMA test on the quality of care delivered by practicing general pediatricians and specialists. To overcome the twin problems of finding an adequate sample size of multiple rare conditions and under/incorrect diagnoses, we used standardized simulated patients known as CPVs. Physicians, randomized into control and intervention groups, cared for the CPV pediatric patients with DD/ASD/ID. Care responses were scored against evidence-based criteria. In round one, participants could order diagnostic tests including existing CMA tests. In round two, intervention physicians could order the 2.8MM probe-CMA test. Outcome measures included overall quality of care and quality of the diagnosis and treatment plan.
Results
Physicians ordering CMA testing had 5.43% (p<0.001) higher overall quality scores than those who did not. Intervention physicians ordering the 2.8MM probe-CMA test had 7.20% (p<0.001) higher overall quality scores. Use of the 2.8MM probe-CMA test led to a 10.9% (p<0.001) improvement in the diagnosis and treatment score. Introduction of the 2.8MM probe-CMA test led to significant improvements in condition-specific interventions including an 8.3% (p = 0.04) improvement in evaluation and therapy for gross motor delays caused by Hunter syndrome, a 27.5% (p = 0.03) increase in early cognitive intervention for FOXG1-related disorder, and an 18.2% (p<0.001) improvement in referrals to child neurology for Dravet syndrome.
Figures
Citation: Peabody J, Martin M, DeMaria L, Florentino J, Paculdo D, Paul M, et al. (2016) Clinical Utility of a Comprehensive, Whole Genome CMA Testing Platform in Pediatrics: A Prospective Randomized Controlled Trial of Simulated Patients in Physician Practices. PLoS ONE 11(12): e0169064. https://doi.org/10.1371/journal.pone.0169064
Editor: Umberto Simeoni, Centre Hospitalier Universitaire Vaudois, FRANCE
Received: February 10, 2016; Accepted: December 12, 2016; Published: December 30, 2016
Copyright: © 2016 Peabody et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All files are available from figshare. The URL is https://figshare.com/articles/Clinical_utility_of_2_8MM_probe-CMA/3858246. The DOI is 10.6084/m9.figshare.3858246.
Funding: Lineagen, Inc. provided all funding for this study. The funder provided support in the form of salaries for authors MM, MP, RV, EBW, but did not have any additional role in designing the study design, the data collection and analysis, decision to publish, or preparation of the manuscript. QURE Healthcare, which was contracted by Lineagen, Inc. to perform the study, provided support in the form of salaries for authors JP, LD, JF, DP, TB, but QURE did not have any additional role in the study design, collection and analysis, decision to publish, or preparation of the manuscript. These were the responsibility of the authors, which are articulated in the ‘author contributions’ section.
Competing interests: The authors have the following interests: MM, MP, RV, and EBW are employees of Lineagen, Inc; LD, JF, DP, and TB are all employees of QURE Healthcare. JP is the President of CPV Technologies, which owns the CPV IP used in the study. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Approximately one in six U.S. children age three to 17-years have a reported developmental disability [1]. While clinically common, there is a remarkable number of genetic abnormalities known to be causal for developmental disorders (DD) that include autism spectrum disorder (ASD) and intellectual disability (ID). Some conditions, such as Down syndrome due to trisomy 21, are relatively straightforward when it comes to a genetic diagnosis, but others are not. Results from a number of studies, for example, suggest ASD is between 56–95% genetic in origin, [2–4] while another recent paper reported that 552 genes are implicated in ASD–a number that will increase as more genes are identified [5,6].
In recent years, these findings have led to the recommendation that genetics evaluations be incorporated into clinical practice guidelines for individuals with developmental disabilities. In 2010, the American College of Medical Genetics (ACMG) recommended chromosomal microarray (CMA) as the first-tier diagnostic genetic evaluation when investigating patients with DD, ASD or ID [7]. The American Academy of Pediatrics recommendation that CMA be considered as part of the initial diagnostic evaluation of patients with disorders of postnatal development including DD and ASD, was superseded by a formal guideline issued in 2014 clearly designating CMA as the first-line test, replacing the standard karyotype and fluorescent in situ hybridization tests for the child with intellectual disability of unknown etiology. These recommendations are mirrored by similar guidelines from the American Academy of Neurology and Society of Children Neurology, and the Academy of Child and Adolescent Psychiatry [8–10].
A confirmed genetic diagnosis means that associated life-threatening medical risks (seizures, tumors, renal disease, heart disease) can be better anticipated and managed, specific medications and therapies administered, and contraindicated treatments can be avoided with a predictability that was not possible before CMA testing [11–13]. Despite clear recognition of these new technologies and the clinical insights they provide, genetic testing likely remains underused [14–15]. This means fewer specific diagnoses are made and corresponding appropriate treatments offered.
Despite uniform opinion amongst the relevant societies as to the importance of CMA, a variety of CMA tests are currently on the market, which differ distinctly in their genomic coverage, both qualitatively and quantitatively, and the clinical interpretations which can be made from the various tests. This impacts the breadth of genetic detection and, ultimately, the clinical utility that testing offers to the patient and the provider.
We undertook a large-scale, experimental study to assess the clinical utility of CMA testing in general and a new 2.8MM probe-CMA test (FirstStepDx PLUS; Lineagen, Inc.), which was developed consistent with ACMG Guidelines on CMA design [16], through the addition of 88,435 probes of content validated to have sound clinical correlates with ASD and other ID/DD associated genes. The 2.8MM probe-CMA test has a higher overall detection rate than currently available CMA tests [17] and includes communication of test results to ordering physicians, and to families, via a comprehensive report that includes clear and actionable recommendations. The goal of this study was to determine the overall evidence-based practice changes that occur when a CMA test was ordered, as well as the specific impacts on diagnostic and treatment quality. This analysis is unique in that it overcomes the enormous practical problem of identifying and aggregating uncommon genetic conditions by using simulated patients to determine clinical utility. This approach allows us to look at the variability of clinical practice–what clinicians do—while holding constant the variability of (hard to find) patients with individually rare disorders.
Methods
Overview
We conducted a longitudinal randomized controlled study of practicing general and sub-specialty-trained pediatricians from across the U.S. without prior experience using the 2.8MM probe-CMA product (S1 Protocol). Using a “before and after” design, pediatricians were asked to care for online simulated patients, using Clinical Performance and Value (CPV®) vignettes via web-based interactive sessions [14, 18–20]. Physicians were randomized into one of two study arms: control and intervention. Physicians completed randomly assigned vignette cases at baseline (Round 1) and again 4 weeks later (Round 2). This study, which ran from August 2014 to January 2015, was conducted in accordance with ethical standards and approved on June 16, 2014, by the Chesapeake Institutional Review Board (IRB), Columbia, MD (S1 File). All participants provided written consent to participate. The trial was voluntarily listed in clinicaltrials.gov (Identifier: NCT02414438). Due to the brevity of the trial, the voluntary nature of the registration (studies not involving drugs, biologics, or devices are not required to register) and a clerical error, enrollment for this trial was underway before being logged in the registry. Notwithstanding, IRB approval was secured prior to conducting the study. The authors confirm that all ongoing and related trials for this intervention are registered.
Eligibility and Selection of Physicians
Recruitment of pediatrician participants was carried out by a mail campaign directed at a random subset of nationally representative lists of 25,000 general and 5,000 sub-specialty practice physicians. Recruit ran from August to September 2014, and letters were sent to 1,000 physicians randomly selected from the lists, explaining the purpose of the study and inviting physicians to participate. Eligibility for participation in the study was assessed using a physician questionnaire. Physicians had to be (1) either board-certified pediatric neurologists, developmental pediatricians or general pediatricians, (2) English-speaking, (3) practicing in a community/non-academic based setting, (4) accessible by Internet, and (5) have no prior experience with the 2.8MM probe-CMA test. Additionally, physicians who met the initial eligibility requirements were required to complete both a physician survey and two rounds of cases to remain in the study. In total, 232 respondents fulfilled the eligibility criteria and consented to participate (Fig 1). These physicians were randomized into control and intervention groups, using a 1:2 control to intervention ratio. Assignment into the arms was based on simple random allocation generated from Microsoft Excel’s random function. Between Rounds 1 and 2 of data collection, physicians in the intervention arm received a 25-minute webinar on the 2.8MM probe-CMA test and were provided with the 2.8MM probe-CMA test results specific to their patient if they chose to order and/or view the results in Round 2.
(139/147 (95%) physicians in the intervention arm confirmed viewing an educational webcast about the 2.8MM probe-CMA. No physicians in the control arm received any specific educational material.
Clinical Performance and Value® Vignettes
Clinical utility was measured at baseline (Round 1) and again after four weeks (Round 2) using CPV® vignettes for both groups. These vignettes are a widely-used, validated means for assessing differences in clinical practice and inherent variation in care, independent of case [14, 18–20]. In each vignette, the general and sub-specialist pediatricians were asked to provide open-ended responses regarding the clinical care they would provide for the simulated patient. Responses were scored in 5 domains: taking a medical history, performing a physical examination, ordering appropriate laboratory and imaging tests, assessing disease activity and physical function, and prescribing treatment. Explicit quality of care scoring criteria were established prior to study administration and were derived from the peer-reviewed medical literature and expert opinion by a physician board certified in Medical Genetics and Pediatrics in conjunction with a panel of clinical physicians. All participant responses were scored against these explicit scoring criteria by clinical physicians blinded to the participant’s identity. Overall CPV scores and scores with each domain represent the percentage of physician answers that addressed each of these evidence-based criteria, and range from 0 to 100%. CPVs have been linked to outcomes data in other studies, wherein a 3–5% increase in scores is associated with an improvement in outcomes and is the threshold used for clinical significance [21].
Cases
The vignettes used in this study simulated clinical encounters involving simulated pediatric patients presenting with an atypical phenotype and/or developmental or intellectual deficits. Patient vignettes in this analysis were developed around five clinical syndromes describing a range of abnormal phenotypes with underlying genetic diagnoses causal of developmental or intellectual delay. Attention was given to conditions in which an accurate diagnosis would lead to change in treatment.
These cases were used to evaluate the impact of CMA testing and the 2.8MM probe-CMA testing on patient diagnosis and management. The order of the cases was randomly assigned. To assess physician, patient and practice characteristics, as well as physicians’ familiarity and experience with genetic testing, we also administered a questionnaire to all physicians prior to baseline.
All participants in both rounds could order and receive results from existing CMA tests; provision of the 2.8MM probe-CMA test results was restricted to Round 2 intervention group physicians. CMA testing options available to the physicians included 180K array, high resolution CMA, and the 2.8MM probe-CMA test. The five cases comprised three groups for CMA testing results: the first category of cases yield similar results regardless of the CMA selected; the second yields normal (i.e., no abnormality) or incomplete results for the 180K test, but abnormal results (i.e., genetic findings) using high resolution CMA and the 2.8MM probe-CMA test; and the third group yields normal 180K and high resolution CMA results but the 2.8MM probe-CMA test detects the abnormality (Table 1).
The 2.8MM Probe-CMA Testing and Reporting Process
The 2.8MM probe-CMA test is a whole-genome CMA that is optimized for clinical use in the setting of DD/ID and ASD in part through the addition of an additional 88,435 probes in genes/regions documented in the clinical literature to be associated or causative of these conditions [23]. Compared to other CMA testing platforms, it has a two-fold increase in detection rate for variants that underlie ASD (5–7% on typical CMA platforms to 12–14% on the 2.8MM probe-CMA test), and an overall increase in detection rate from 15–20% to over 30% for all suspected DD/ID cases including ASD [17,24]. The 2.8MM probe-CMA test results are provided in a detailed report that includes both physician-specific and parent-specific sections with a clear description of the underlying abnormality, suggestions for medical management and/or further testing, and recurrence risk. The increased detection rate and resulting clinical management recommendations from Lineagen’s 2.8MM probe-CMA test should increase the clinical utility of CMA testing when this format is used.
Intervention
Before Round 2 data collection, subjects in the intervention arm were given an educational component describing CMA testing in general and the 2.8MM probe-CMA test specifically. This was a 25-minute web-based pre-recorded video, which participants watched at their convenience and provided an overview of genetic testing for ID/DD and described the 2.8MM probe-CMA test. Nearly all (138 of 149 intervention arm participants) confirmed viewing the intervention video. After viewing the short video, the intervention group completed three additional (randomly) administered cases and now had the opportunity to order and review the 2.8MM probe-CMA test results if they chose to do so. Physicians in the control arm also completed a second round of cases, but did not receive any information about the 2.8MM probe-CMA test and could not order or receive the 2.8MM probe-CMA test results.
Analysis
First, we compared the intervention and control groups, and summary statistics were assessed for baseline provider and practice characteristics. For continuous and interval data, we used an independent samples t-test to determine the p-value. For categorical data, Fisher’s exact test was used to compare the distribution of the two groups. We then tested the main hypothesis: Does CMA testing in general and/or the 2.8MM probe-CMA testing specifically improve clinical utility? We measured changes in clinical utility using quality of care scores from the CPV® vignettes (e.g., ordering the proper diagnostic work, making appropriate referrals), which quantify the percentage of pre-determined evidence-based care criteria which were appropriately addressed by the physician while caring for the virtual patient. We further hypothesized that the pediatricians who ordered CMA, including the 2.8MM probe-CMA test, were more likely to improve their diagnostic accuracy and deliver specific treatments that were appropriate for the patient’s underlying condition. To test this, we estimated the intervention’s effect by performing a logistic regression and modeling the marginal effects for the specific diagnosis or treatment option of interest.
We compared the change (differences) in the outcome scores between Rounds 1 and 2 for both the intervention and the control groups. Univariate regression models using physician and practice characteristics were performed (S1 Table). Our a priori assumption was that all physician and practice characteristics would be of interest in the final model, and thus, our decision was to include all these variables in the final model, regardless of significance in the univariate model. After including these variables, we added in the relevant CMA, round, and intervention variables. 2.8MM is the explanatory variable describing those who ordered the 2.8MM probe in Round 2, and CMA is an explanatory variable for those who ordered CMA testing (other than the 2.8MM probe-CMA test) in either round. The 2.8MM variable measures the utility of the 2.8MM probe-CMA testing after its introduction in Round 2. We also analyzed the interaction of ROUND x CMA, to measure the differential change in the utility from CMA testing in Round 2 versus Round 1 and a possible learning effect spillover from the 2.8MM probe-CMA test to all CMA testing. The interaction terms between the 2.8MM probe-CMA test x Group 2 and the 2.8MM probe-CMA test x Group 3 tells us the differential effects on cases types of the 2.8MM probe-CMA test compared to any other CMA tests.
Results
Provider Characteristics
After randomization, there were 147 participants in the intervention arm and 85 in the control arm. Ultimately, 202 physicians completed both rounds of data collection from September 2014 through January 2015, the predefined study period. 30 participants were dropped from the study (22 from the controls and 8 from the intervention arm) because they were determined to have not met initial inclusion criteria (n = 5), asked to be withdrawn from the study (n = 3), or did not complete the additional study requirements (n = 22) (Fig 1). There was no significant difference between the control and intervention groups by provider characteristic (Table 2). Overall, the majority of the pediatricians were part of a hospital/health care system (61%). Over 90% were employed by their practice. 59% were pediatric specialists and 41% were general pediatricians.
Practice characteristics, particularly as they relate to genetic testing, were varied (Table 3). On familiarity with genetic testing, the average score was 2.9 on a scale of 1 to 5 and 44% rated themselves a 3. However, 85% felt that a geneticist was readily accessible to them if they needed consultation. Their patients were almost evenly split between commercial and public insurance patients (mostly Medicaid) and their practices were active with more than 58% of respondents reporting seeing over 50 general pediatric patients per week, while 30% reported seeing more than 20 patients with developmental disabilities per week.
Ordering CMA Diagnostic Testing
Overall 36% of all physicians ordered CMA testing of any kind in Round 1 of the study and 44% ordered CMA testing in Round 2. Not surprisingly, almost no physicians in any of the study arms ordered the 2.8MM probe-CMA test in Round 1 (n = 5 or 1.1% of the sample) a pattern which persisted in the control arm in Round 2, where only 1 physician ordered the 2.8MM probe-CMA test. If the 2.8MM probe-CMA test was ordered in Round 1 or by a control arm physician in Round 2, the 2.8MM probe-CMA test results were delivered. Almost all of the increase in CMA testing in Round 2 came from physicians in the intervention group ordering specifically the 2.8MM probe-CMA test, where 20% (n = 60) of physicians ordered the test (Table 4).
Link between Diagnostic Testing and CPV® Vignette Scores
CMA testing was associated with improved medical management, taken here as a surrogate for clinical utility. Those that ordered CMA testing had overall CPV quality scores 5.4% higher (p<0.001) than those that did not, indicative of clinically significant improvements in medical management (Table 5). This difference did not change, in either magnitude or significance, in the second round of testing. However, the 2.8MM probe-CMA testing, which was only available in Round 2, resulted in CPV quality scores 7.2% higher than cases in which CMA was not ordered. This represents an additional 1.8% improvement in quality of care over cases in which other CMA tests were ordered, which is statistically significant (p<0.001).
We looked at the three clinical groups detailed in Table 1 to determine if this improvement was from the additional clinical information provided by the 2.8MM probe-CMA test. By design, the 2.8MM probe-CMA test would provide similar diagnostic information as other CMA tests in Group 1, a more specific diagnosis compared to the 180k test in Group 2, and a more specific diagnosis compared to both the 180k and high resolution CMA in Group 3. A regression model including the interaction of the 2.8MM probe-CMA test with Groups 2 and 3 confirmed that overall CPV scores increased by 10.9% in Group 2 and 8.7% in Group 3 CPVs, compared to Group 1 (p = <0.001 for each Group). Two additional covariates were also associated with higher scores: generalist physicians (p = 0.008) and female physicians (p = 0.003).
For the five cases where the 2.8MM probe-CMA test offered a specific diagnosis, this was also associated with a statistically significant, 10.9% greater accuracy (p<0.001) for combined diagnosis and treatment scores (Table 6). This difference accounted for the majority of overall improvement in clinical utility noted in Table 5. Conversely, other CMA testing (defined as 180K, regular or high resolution CMA, but excluding the 2.8MM probe-CMA test) was not linked to significantly higher scores in the combined diagnosis and treatment domain, showing an increase of only 2.7% in the diagnosis and treatment scores (p = 0.122). In this analysis, neither covariate of generalist physician or gender of physician was associated with a significant increase in the combined score. However, those physicians who did not own their practice were more likely to accurately treat and diagnose their patients (p = 0.001). The effect of using the 2.8MM probe-CMA test was strong and positive in both Group 2 and Group 3.
Finally, we completed a secondary analysis, conducting a case-by-case evaluation of diagnostic and treatment practice changes associated with more accurate the 2.8MM probe-CMA test diagnosis (Table 7). Despite the lack of statistical significance, the disaggregated data were directionally suggestive of several benefits. For two out of the five conditions (Hunter syndrome and mosaic Turner syndrome), the 2.8MM probe-CMA test led to a statistically higher rate of accurate diagnosis, as seen in the improved diagnosis scores in the intervention group. Correct secondary diagnoses, such as gross motor delay for Hunter syndrome and uncontrolled myoclonic seizures for Dravet syndrome, were also more likely to be made for four of the five conditions (Table 7). For almost every condition studied there were some statistically significant differences in treatment. Examples of these treatment differences include: evaluation and therapy for gross motor delays (Hunter syndrome); anticipatory guidance and screening for heart disease (Turner syndrome); recommending early cognitive intervention (FOXG1-related disorder); referral to child neurologist and commencement of neuroleptics (Dravet syndrome). Many other treatments were directionally positive but not statistically significant and thus not causally established. However, in the Turner syndrome case, one important treatment that showed there was no difference between the intervention and control groups in either ordering or discussing the performing a prophylactic gonadectomy, despite the risk of a gonadoblastoma in these patients.
Discussion
Appropriate assessment of new health care interventions, particularly diagnostic technologies, requires high quality information on the clinical utility, economic value, affordability, and population health benefits of the technology in question [25]. As a reflection of this, assessing clinical utility—demonstrating the usefulness of a test in clinical practice—is now a significant hurdle required by payors before offering insurance coverage for most new technologies [26]. As government and commercial payors struggle to make decisions regarding reimbursement, demonstrating that additional clinical information is useful and leads to changes in what a provider does is critically important for any vulnerable population [27].
Developmental delays, including autism spectrum disorder, have both a common phenotypic expression and are a reflection of myriad different genetic conditions, each with its own clinical characteristics that require unique therapeutic interventions. Assessing these patients clinically, however, is time consuming and difficult, leading to missed diagnoses for thousands of children every year [28,29]. To overcome this, professional societies recommend that genetic testing, specifically CMA, be done as first tier diagnostic evaluation of these conditions [7–10].
We sought to clarify the evidence around the utility of CMA testing for patients with developmental delay associated with five specific genetic copy number variants (CNVs) detected by CMA testing. We did this by using an innovative measurement tool, CPVs, which controls for patient case mix, overcomes the problem of collecting an adequate number of patients with a specific (rare) genetic disorder and allows us to conduct a randomized controlled experiment of related physician behavior and patient management. CPVs are a validated measure of clinical practice that has been widely used to serially measure clinical performance [20,30]. The tool has proven particularly valuable in clinical utility studies determining whether new technologies change clinical decision making and associated spending [18]. Using this tool, we examined whether CMA testing in general, and the 2.8MM probe-CMA test testing specifically, changed provider practice for five different patient genotypes.
We found that CMA testing did indeed improve the quality of care and clinical decision making among more than 200 general and sub-specialty pediatricians. When physicians used the 2.8MM probe-CMA test, an optimized whole-genome high resolution CMA, we found an increased causal relationship between testing and improved clinical utility. Physician users of the 2.8MM probe-CMA test had a statistically and clinically significant increase of 7.2% in their overall utility scores over those physicians who use no CMA testing at all and a 1.8% marginally higher score over those who use standard CMA testing. The effect of the 2.8MM probe-CMA test is even more notable on diagnosis and treatment scores, arguably the most critical part of the care process. Scores of the 2.8MM probe-CMA test users for the combined domain score for diagnosis and treatment were 10.9 percentage points higher than those who used no CMA testing at all. The 2.8MM probe-CMA test was particularly useful in individuals who harbor a pathogenic CNV in regions covered by the custom probes contained on that CMA to optimize clinical interpretation in cases of DD, ASD and ID, including but not limited to the genes for GAMT deficiency (GAMT) and congenital Rett syndrome (FOXG1-related disorder). Most of the commercially available CMA platforms cannot detect these, and other genetic abnormalities targeted by the 2.8MM probe-CMA test. The increased clinical utility of the 2.8MM probe-CMA test also extends to individuals affected by syndromes such as Mosaic Turner syndrome with an XY cell line and Dravet syndrome, which can be diagnosed by some other CMA tests available on the market. We hypothesize that this increase in clinical utility is driven by the comprehensiveness and clarity of the 2.8MM probe-CMA test report, which includes specific treatment recommendations.
We observed opportunities to improve the quality of care delivered to all patients with DD/ID and ASD. At baseline, only 37% of physicians ordered any CMA for their patients. Following the addition of the 2.8MM probe-CMA test, after providing a simple intervention, CMA testing increased significantly to 44%. CMA testing at baseline was surprisingly low suggesting that professional medical guidelines are either not widely known or are not used.
By analyzing the specific aspects of care provided to these patients, we found that the 2.8MM probe-CMA test provided benefits in many areas such as identifying the primary diagnosis of the DD/ID syndrome, managing secondary diagnoses, and defining treatment. The treatment improvements led to instances of proper treatment and presumably earlier therapy. Indeed, successful management of ID/DD/ASD is predicated on recognition of any underlying syndrome. Other studies will be needed to determine if earlier, more precise, diagnosis from CMA testing leads to less unnecessary testing or less testing overall.
This study has several limitations. The control group received neither orientation on genetic testing nor the 2.8MM probe-CMA test results, thus it is difficult to tease apart the effect from the 2.8MM probe-CMA test results from that of the educational intervention on genetic disorders and testing. Second, the five diseases that were the focus of the CPVs are each relatively rare like most DD, ASD and ID syndromes, thus results here may not necessarily be extrapolated to all underlying syndromes. We do suspect however that regardless of the syndrome, the challenges of arriving at a genetic diagnosis and developing an appropriate therapeutic plan for the individual are common across a number of disease types. Thirdly, this study did not address issues related to variants of unknown significance, additional secondary possibly actionable variants, or point mutations necessitating genomic sequencing approaches to detect. As genetic tests become increasingly detailed and complex, these issues will benefit from future follow-up work. Finally, this analysis focused on CMA testing, which in general will detect cases of these conditions due to larger deletions and duplications, not point mutations. Future studies of this type will be needed to address the contribution of point mutations by emerging next-generation sequencing technology.
Studies to determine the clinical utility and impact of diagnostics and interventions for rare diseases are particularly challenged by the difficulty in locating and identifying an adequate number of patients for study. Those who attempt to undertake assessments of clinical utility using patient level data are additionally challenged by difficulty controlling for case mix which is a routine issue in any patient level study. This challenge is further compounded when dealing with rare diseases and small numbers of study participants. The CPV vignettes provide a standardized method to analyze how physicians with different backgrounds and from various settings assess and treat the same patient. While the CPV approach does not use actual patient level data, they uniquely provide a standardized method to analyze how physicians with different backgrounds and from various settings would assess and treat the same patient. Our results show that CMA in general and specifically the 2.8MM probe-CMA test provides improved medical management in children with developmental delay, and therefore meet criteria for clinical utility.
Supporting Information
S1 Table. Univariate regression model linking CMA and Utility: Overall CPV Scores.
https://doi.org/10.1371/journal.pone.0169064.s001
(DOCX)
S1 Checklist. CONSORT 2010 checklist of information to include when reporting a randomized trial.
https://doi.org/10.1371/journal.pone.0169064.s002
(DOC)
S1 File. Protocol approval with modifications.
https://doi.org/10.1371/journal.pone.0169064.s004
(PDF)
Author Contributions
- Conceptualization: JP MM MP.
- Data curation: JF DP.
- Formal analysis: JF DP.
- Funding acquisition: MP RV ERW.
- Investigation: JP LD TB.
- Methodology: JP LD JF.
- Project administration: MM LD.
- Resources: JP MP RV.
- Software: JP LD.
- Supervision: JP MP RV ERW.
- Validation: JP DP.
- Writing – original draft: JP MM LD MP RV.
- Writing – review & editing: JP MM LD JF DP MP RV ERW TB.
References
- 1.
Boyle CA, Boulet S, Schieve LA, Cohen RA, Blumberg SJ, Yeargin-Allsopp M, et al. Trends in the Prevalence of Developmental Disabilities in US Children, 1997–2008. Pediatrics. 2011 May 23;peds.2010–989.
- 2. Ronald A, Hoekstra RA. Autism spectrum disorders and autistic traits: A decade of new twin studies. Am J Med Genet B Neuropsychiatr Genet. 2011 Apr 1;156(3):255–74.
- 3. Colvert E, Tick B, McEwen F, Stewart C, Curran SR, Woodhouse E, et al. Heritability of Autism Spectrum Disorder in a UK Population-Based Twin Sample. JAMA Psychiatry. 2015 May 1;72(5):415–23. pmid:25738232
- 4. Deng W, Zou X, Deng H, Li J, Tang C, Wang X, et al. The Relationship Among Genetic Heritability, Environmental Effects, and Autism Spectrum Disorders: 37 Pairs of Ascertained Twin Study. J Child Neurol. 2015 Apr 14;
- 5. Warrier V, Chee V, Smith P, Chakrabarti B, Baron-Cohen S. A comprehensive meta-analysis of common genetic variants in autism spectrum conditions. Mol Autism. 2015 Aug 28;6:49. pmid:26322220
- 6. Miles JH. Autism spectrum disorders—A genetics review. Genet Med. 2011 Apr;13(4):278–94. pmid:21358411
- 7. Manning M, Hudgins L, Professional Practice and Guidelines Committee. Array-based technology and recommendations for utilization in medical genetics practice for detection of chromosomal abnormalities. Genet Med Off J Am Coll Med Genet. 2010 Nov;12(11):742–5.
- 8. Michelson DJ, Shevell MI, Sherr EH, Moeschler JB, Gropman AL, Ashwal S. Evidence report: Genetic and metabolic testing on children with global developmental delay: report of the Quality Standards Subcommittee of the American Academy of Neurology and the Practice Committee of the Child Neurology Society. Neurology. 2011 Oct 25;77(17):1629–35. pmid:21956720
- 9. Moeschler JB, Shevell MI, and COMMITTEE ON GENETICS. Comprehensive Evaluation of the Child With Intellectual Disability or Global Developmental Delays. Pediatrics. Sept 2014; 134(3):903–18.
- 10. Volkmar F, Siegel M, Woodbury-Smith M, King B, McCracken J, State M. Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. J Am Acad Child Adolesc Psychiatry. 2014 Feb;53(2):237–57. pmid:24472258
- 11. Coulter ME, Miller DT, Harris DJ, Hawley P, Picker J, Roberts AE, et al. Chromosomal microarray testing influences medical management. Genet Med. 2011 Sep;13(9):770–6. pmid:21716121
- 12. Ellison JW, Ravnan JB, Rosenfeld JA, Morton SA, Neill NJ, Williams MS, et al. Clinical utility of chromosomal microarray analysis. Pediatrics. 2012 Nov;130(5):e1085–95. pmid:23071206
- 13. Riggs ER Wain KE, Riethmaier D, Smith-Packard B, Faucett WA, Hoppman N, Thorland EC, Patel VC, Miller DT. Chromosomal microarray impacts clinical management. Clinical Genetics. 2014; 85(2):147–53. pmid:23347240
- 14. Peabody JW, DeMaria LM, Tamandong-LaChica D, Florentino JF, Burgon T. Low Rates of Genetic Testing in Children with Developmental Delays, Intellectual Disability and Autism Spectrum Disorders. Global Pediatric Health. 2015;
- 15. Tarini BA, Zikmund-Fisher BJ, Saal HM, Edmondson L, Uhlmann WR. Primary Care Providers' Initial Evaluation of Children with Global Developmental Delay: A Clinical Vignette Study. J Pediatr. 2015 Dec;167(6):1404–1408.e1. pmid:26477869
- 16. Kearney HM, South ST, Wolff DJ, Lamb A, Hamosh A, Rao KW; Working Group of the American College of Medical Genetics. American College of Medical Genetics recommendations for the design and performance expectations for clinical genomic copy number microarrays intended for use in the postnatal setting for detection of constitutional abnormalities. Genet Med. 2011;13(7):676–9. pmid:21681105
- 17.
Ho KS, Vanzo RJ, Peiffer A, Hensel CH. Chromosomal microarrary results and medical management for children with autism spectrum disorder and other developmental conditions. Poster presented at: International Society for Autism Research; 2015 May 13–16; Salt Lake City, UT.
- 18. Peabody JW, Strand V, Shimkhada R, Lee R, Chernoff D. Impact of rheumatoid arthritis disease activity test on clinical practice. PloS One. 2013;8(5):e63215. pmid:23667587
- 19. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000 Apr 5;283(13):1715–22. pmid:10755498
- 20. Peabody JW, Luck J, Glassman P, Jain S, Hansen J, Spell M, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med. 2004 Nov 16;141(10):771–80. pmid:15545677
- 21. Peabody JW, Shimkhada R, Quimbo S, Solon O, Javier X, McCulloch C. The impact of performance incentives on child health outcomes: results from a cluster randomized controlled trial in the Philippines. Health Policy Plan. 2014 Aug;29(5):615–21. pmid:24134922
- 22. Sdano M, Vanzo RJ, Martin MM, Baldwin EE, South ST, Rope AF, et al. Clinical utility of chromosomal microarray analysis of DNA from buccal cells: detection of mosaicism in three patients. 2014. J Genetic Counsel.
- 23. Matsunami N, Hadley D, Hensel CH, Christensen GB, Kim C, Frackelton E, et al. Identification of rare recurrent copy number variants in high-risk autism families and their prevalence in a large ASD population. PLoS ONE. 2013;8(1):e52239. pmid:23341896
- 24. Miller DT, Adam MP, Aradhya S, Biesecker LG, Brothman AR, Carter NP, et al. Consensus statement: chromosomal microarray is a first-tier clinical diagnostic test for individuals with developmental disabilities or congenital anomalies. Am J Hum Genet. 2010 May 14;86(5):749–64. pmid:20466091
- 25. Phillips KA, Sakowski JA, Trosman J, Douglas MP, Liang S-Y, Neumann P. The economic value of personalized medicine tests: what we know and what we need to know. Genet Med Off J Am Coll Med Genet. 2014 Mar;16(3):251–7.
- 26. Peabody JW, Shimkhada R, Tong KB, Zubiller MB. New thinking on clinical utility: hard lessons for molecular diagnostics. Am J Manag Care. 2014 Sep;20(9):750–6. pmid:25365750
- 27. ACMG Board of Directors. Clinical utility of genetic and genomic services: a position statement of the American College of Medical Genetics and Genomics. Genet Med Off J Am Coll Med Genet. 2015 Mar 12;
- 28. Vande Wydeven K, Kwan A, Hardan AY, Bernstein JA. Underutilization of genetics services for autism: the importance of parental awareness and provider recommendation. J Genet Couns. 2012 Dec;21(6):803–13. pmid:22415587
- 29. Too many children with autism miss genetics evaluations: parents and other medical specialties and professionals need education, study says. Am J Med Genet A. 2012 Aug 1;158A(8):vii–viii. pmid:22821836
- 30. Dresselhaus TR, Peabody JW, Luck J, Bertenthal D. An evaluation of vignettes for predicting variation in the quality of preventive care. J Gen Intern Med. 2004 Oct; 19(10):1013–8. pmid:15482553
