Correction
10 Mar 2016: Wang G, Sun Y, Zhang K, Jia T, Hao M, et al. (2016) Correction: External Quality Assessment of Molecular Detection of Ebola Virus in China. PLOS ONE 11(3): e0151695. https://doi.org/10.1371/journal.pone.0151695 View correction
Figures
Abstract
In 2014, Ebola hemorrhagic fever broke out in West Africa. As contact between China and West Africa is frequent, the possibility that Ebola virus would enter China was high. Thus, an external assessment of the quality of Ebola virus detection was organized by the National Center for Clinical Laboratories in China. Virus-like particles encapsulating known sequences of epidemic strains of Ebola virus from 2014 were prepared as positive quality controls. The sample panel, which was composed of seven positive and three negative samples, was dispatched to 19 laboratories participating in this assessment of Ebola virus detection. Accurate detection was reported at 14 of the 19 participating laboratories, with a sensitivity of 91.43% and a specificity of 100%. Four participants (21.05%) reported false-negative results and were classified as “acceptable.” One participant (5.26%) did not detect any positive samples and was thus classified as “improvable.” Based on the results returned, the ability to detect weakly positive Ebola specimens should be improved. Furthermore, commercial assays and the standard primers offered by the Chinese Centers for Disease Control and Prevention were found to be most accurate and dependable for Ebola detection. A two-target detection approach is recommended for Ebola screening; this approach could reduce the probability of false-negative results. Additionally, standardization of operations and punctual adjustment of instruments are necessary for the control and prevention of Ebola virus.
Citation: Wang G, Sun Y, Zhang K, Jia T, Hao M, Zhang D, et al. (2015) External Quality Assessment of Molecular Detection of Ebola Virus in China. PLoS ONE 10(7): e0132659. https://doi.org/10.1371/journal.pone.0132659
Editor: Jens H. Kuhn, Division of Clinical Research, UNITED STATES
Received: May 7, 2015; Accepted: June 17, 2015; Published: July 15, 2015
Copyright: © 2015 Wang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by the National High Technology Research and Development Program of China 863 program (grant No. 2011AA02A116) and “AIDS and hepatitis, and other major infectious disease control and prevention” Program of China (No. 2013ZX10004805). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
From 2013 to 2015, the outbreak of Ebola hemorrhagic fever (EHF) caused over 22,000 cases of infection, including nearly 9,000 deaths (as of February 4, 2015) [1] in West Africa. The Ebola virus has an 18–19 kb, non-segmented, single-stranded negative-RNA genome that encodes seven structural proteins (nucleoprotein (NP), virion structural protein (VP)35, VP40, glycoprotein (GP), VP30, VP24, and RNA-dependent RNA polymerase (L)) [2]. Virological research and analysis recently identified the epidemic agent as Zaire Ebola virus (EBOV) [3]. The characteristics of infection with EBOV are high mortality [4] and catastrophic clinical syndromes. Owing to a lack of effective intervention and treatment, early diagnosis and isolation are the primary methods to resist EBOV infection [5]. For early detection of EBOV [6], real-time reverse transcription polymerase chain reaction (RT-PCR) is the method most commonly and frequently used [7, 8].
On August 8, 2014, the World Health Organization (WHO) announced the Ebola outbreak in West Africa as a global public health event. In fact, although the EBOV epidemic was concentrated in West Africa, cases of infection were also reported in other areas, such as the USA, Spain, and Scotland. Although EBOV showed a sporadic distribution, the possibility of Ebola infections appearing outside West Africa could not be neglected. China is one of the countries in closest contact with West Africa, and a large number of people entered and exited China, especially in harbor cities, from the areas affected by EBOV by airline. As a result, there was a risk that EBOV would enter China. Screening of high-risk populations from EBOV-affected regions was therefore essential not only for control and prevention of Ebola in Chinese native, but also as part of the global task of fighting the plague.
Hence, it was important to evaluate the EBOV detection capability in key port cities in China, along with the reagents employed by the testing agencies. Therefore, the National Center for Clinical Laboratories (NCCL) organized an external quality assessment (EQA) of EBOV detection in China.
Materials and Methods
Preparation and composition of panel
Nucleotide sequences of the epidemic strain of Zaire Ebola virus from West Africa were obtained from GenBank (KJ660346, KJ660347, and KJ660348) [3]. Synthesis of the target segments of the EBOV genome was outsourced to Thermo Fisher Scientific Inc. (Beijing, China), including the full sequence of the 3’-untranslated region (UTR) and the NP fragment, and portions of the sequence of the GP, L, and 5’-UTR regions. Synthesis of the target sequences was divided into two parts: the first encompassed nearly 2400 bp (the 3′-UTR and a large part of the NP region), and the other encompassed approximately 2250 bp (the remaining part of the NP region, and selected parts of the GP, L, and 5′-UTR regions). The primers used are listed in S1 Table. The two synthesized fragments were subcloned into the pACYC-MS2 vector at the KpnI/PacI and BglII/KpnI restriction sites, respectively, to form the recombinant plasmids pACYC-MS2-NP and pACYC-MS2-GP/L. The expression and purification of the two types of MS2 virus-like particles (VLPs) was performed according to previously published protocols [9, 10].
Two different types of EBOV MS2 VLPs were used as positive samples. The sample panel was comprised of three negative and seven positive samples, ranging from 8 × 102 to 5 × 107 copies/mL, in accordance with previously reported clinical outcomes [11]. Three negative samples and dilution buffer for positive ones were composed of 30% normal human serum and 70% normal saline.
Of the seven positive samples, No. 1402 contained the highest concentration of EBOV VLPs (Table 1). No. 1408 also contained hepatitis C virus (HCV) and measles virus (MV) VLPs, to verify whether EBOV detection is impacted by the presence of other viruses. Repeatability was evaluated by the use of two replicate specimens (No. 1403 and No. 1407). No.1410 was prepared to evaluate the ability to detect slightly positive samples. Additionally, one specimen (No. 1405) contained only a very weak concentration of the NP fragment and a moderate concentration of the GP/L EBOV fragment. Prior to distribution, each sample panel was confirmed using three commercial kits manufactured in China (Daan Gene Co., Ltd., Guangzhou; ZJ Bio-Tech Co., Ltd., Shanghai; and Puruikang Bio-Tech Co., Ltd., Shenzhen).
Stability assessment
The stability of two types of EBOV MS2 VLPs with the dilution buffer of 30% normal human serum and 70% normal saline was evaluated at different times and temperatures. Two different concentrations of VLPs were used: 105 copies/mL and 103 copies/mL. For each of the concentrations, 15 samples were included. Three samples were stored at -80°C as a control. The remaining 12 samples were divided into three groups and incubated at different temperatures: room temperature, 4°C, and -20°C. In each group, two samples were removed and stored at -80°C for 10 days or 20 days. After 20 days, all 15 samples for each concentration were simultaneously assessed using EBOV commercial assays (real-time RT-PCR kit, Daan Gene Co., Ltd., Guangzhou, China).
The stability of the samples during distribution and transport was monitored. A sample panel was sent unmarked by mail to a deliberately non-existent destination and returned to NCCL. The returned sample panel was assessed simultaneously with another that had been stored at -80°C, using an EBOV RT-PCR assay manufactured by Daan Gene Co., Ltd. In all assays, stability was designated as a reduction of < 0.5 log10 copies/mL, compared with the results for samples stored at -80°C.
Organization and Transport
All qualified laboratories, institutions for inspection and quarantine, and manufacturers of kits for EBOV detection in mainland China were invited to participate in this investigation in November 2014. All participating organizations were identified by early December (Fig 1). All samples were distributed by NCCL on December 10, 2014. Qualitative results, threshold cycle (Ct) values, RNA extraction and detection methods, and primer sources were all required to be reported to NCCL by December 30, 2014. The full EQA process and related time points are shown in Fig 1.
Appraisal and statistical analysis
The results were classified into three categories: competent (100% correct responses), acceptable (≤ 2 incorrect results), or improvable (> 2 incorrect results). All data were analyzed using SPSS, version 19.0; the variances of different groups were compared using Fisher’s exact test or Pearson’s chi-square test, when appropriate.
Results
Preparation, stability, and evaluation of the sample panel
Construction of the recombinant plasmids pACYC-MS2-NP and pACYC-MS2-GP/L was successful. A series of experiments was conducted to verify the effects of VLP expression and packaging capability. Target protein purification in approximately 60 minutes (S1A Fig) and observation of a 14-kD band following sodium dodecyl sulfate polyacrylamide gel electrophoresis (SDS-PAGE) (S1B Fig) were preliminary evidence for successful VLP expression. After digestion with RNase A and DNase I at 37°C, observation of a single band over 1 kb in length in 1% agarose gel electrophoresis (S1C Fig) indicated the stability of the EBOV VLPs. RT-PCR results showing a target band between 2–2.5 kb (S1D Fig) served as further evidence of successful encapsulation of the target EBOV segments in VLPs after extraction of nucleotides. The concentrations of the two types of EBOV VLPs were each approximately 5 × 1011 copies/mL.
In the stability evaluation, the variations of two concentrations of samples in different temperatures were presented in form of mean Ct±standard deviation (SD), which indicated in S2 Table. Samples stored at different conditions showed less than 0.5 log10 variations in concentration, compared with samples incubated at -80°C. Meanwhile, the comparisons of sample panels saved in -80°C and from blind mail were showed in S2 Fig. No statistical difference was observed between test results for sample panels sent unmarked through the mail and those stored at -80°C.
The sample panel was verified before distribution using the three commercial kits mentioned above; the results were concordant with the results shown in Table 1. The results for No. 1403 and No. 1407 were similar. No obvious interference of HCV or MV with EBOV detection was observed for No. 1408. The slightly positive samples (No. 1405 and No. 1410) could be detected.
Participating agencies and methodologies
A total of 19 agencies took part in the assessment of the quality of molecular detection of EBOV, including three provincial Centers for Disease Control and Prevention (CDC) (Shanghai, Zhejiang, and Jiangsu), six Entry-Exit Inspection and Quarantine agencies in port cities (Fujian, Hainan, Hebei, Yunnan, Zhejiang, and Zhuhai), one public health platform in Shanghai, one fixed point hospital qualified for EBOV detection(Chongqing), and eight manufacturers of EBOV detection kits(Daan Gene Co., Ltd., ZJ Bio-Tech Co., Ltd., HuaDa BioTech Co., Ltd., KeHua Bio-Engineering Co., Ltd., TianLong Bio-Tech Co., Ltd., BioPerfectus Technologies, Sansure Bio-Tech Co., Ltd., and Puruikang Bio-Tech Co., Ltd.). All participating organizations had certification in China for EBOV screening and diagnosis.
At the 19 participating agencies, both magnetic bead and spin column methods were used for nucleic acid extraction. The paramagnetic particle method was preferred by nine organizations, the spin column method by the other ten (Table 2). All of the laboratories that used the spin column method used manual extraction, whereas 2/3 of the agencies using the paramagnetic particle method used instruments. For detection, only one agency used RT-PCR and agarose gel electrophoresis, whereas most chose real-time PCR. Among the 18 agencies that selected quantitative real-time PCR, 83.33% used commercial kits. The specified primers offered by the national CDC [12] (Table 3), one of which returned results for both the NP and GP sites, were preferred by two provincial CDCs. Additionally, one agency used a real-time PCR assay developed in-house for EBOV diagnosis.
Laboratory performance
The performance of the 19 participating agencies is shown in Table 2. The performance of 14 of the 19 agencies (73.68%) was found to be competent (100% correct responses). Three participants (15.79%) reported one false-negative and one (5.26%) reported two false-negatives; these were classified as acceptable (21.05%) according to the criteria stated above. One participant (5.26%) was unable to obtain any positive results in this assessment, and was thus included in the improvable group.
In all, among the 200 results returned, 12 (6%) were incorrect, including 8.57% false-negative results and no false-positives. Seven of these incorrect results were returned by one agency. No. 1405 was not detected by five different laboratories, which reported very weakly positive results for the NP (8 × 102 copies/mL) fragment. The pair of duplicate specimens (No. 1403 and No. 1407) with similar Ct values was reported correctly by 18 institutions. No. 1410 (NP: 8 × 103 copies/mL, GP/L: 2 × 103 copies/mL), with a Ct value close to the declared lower limit (1 × 103 copies/mL) of several kits, was detected by 17 participants. In addition, target sequences of other viruses (No. 1408 with HCV and MV VLP) did not produce cross-reactivity with the EBOV fragments.
We next evaluated the performance of different assays and participants. The commercially available assay from Daan Gene was the most frequently used and produced 100% accurate results at five agencies. Several participants selected the kits manufactured by Shanghai ZJ Bio-Tech Co., Ltd., one of which failed to detect the weakly positive specimens (No. 1405 and No. 1410). The primers offered by the Chinese CDC were selected by two participants, which achieved “competent” results. One participant used the RT-PCR and agarose gel electrophoresis method, and returned qualitative results (100% accurate rate) without a Ct value. However, another used an assay developed in-house and achieved only 30% accuracy, failing to detect any positive specimens. Additionally, no significant differences (P = 0.179) were observed between results obtained using commercial kits and those obtained using assays developed in-house.
Discussion
From 2014 to 2015, the number of people suffering from Ebola hemorrhagic fever (EHF) increased continually in West Africa [13], and at one point the situation was out of control [14]. As the epidemic attracted international concern, many countries provided the required support for Ebola-affected areas. A Chinese medical team and public health experts [15] were assigned to the worst-hit countries, such as Sierra Leone, Liberia, and Guinea [16, 17]. Along with these personnel, several commercial real-time RT-PCR assays, including the kits manufactured by Daan Gene, ZJ Bio-Tech, Puruikang, Huada BioTech, and Tianlong, were used in EBOV detection in West Africa; these kits were also assessed in this EQA. Additionally, the assay manufactured by ZJ BioTech had attained the certification both from the Conformite Europeenne (CE) and WHO. The medical staff who worked in Ebola-affected regions and then returned to China needed to be isolated and to be examined for infection after a certain period of time. Given the presence of high-risk groups among Chinese citizens, adequate capacity for detection of and emergency response to EBOV had vital significance not only for China itself, but also for plague prevention at a global scale. Consequently, it is necessary to conduct this EQA for Ebola detection in China.
The majority of laboratories and commercial assays (73.68%) tested could detect EBOV with 100% accuracy. Notably, all negative specimens were identified correctly. However, efforts should be made to improve detection sensitivity. Apart from the seven positive samples that were not detected by the “improvable” participants, five positive samples were not detected. The conditions of the false-negative results should be considered carefully, to avoid errors in detection of weakly positive samples. In clinical practice, false-negative results could cause misdiagnosis and delay timely treatment and isolation of patients. In fact, PCR results are often negative at 2 or 3 days after the onset of clinical symptoms [18]. The earlier EBOV is diagnosed, the more timely measures can be taken, which has advantages for the control and prevention of EBOV.
In this assessment, we confirmed the dilution buffer composed of 30% normal human serum and 70% normal saline as sample matrix. In fact, three kinds of different proportions of dilution buffers (100% normal human serum, 30% normal human serum and 70% normal saline and 100% normal saline) were detected with the EBOV MS2 VLPs at the concentration of 105copies/mL before finally determined. It was regarded as no difference because the variations among three types of dilution buffer were all less than 0.5 log10 copies/mL. In the dilution buffer, we mixed human serum for simulated EBOV clinical specimens and considering the influence of sample matrix in nucleic acid detection. Meanwhile, the stability of samples in the confirmed dilution buffer was further proved. Hence, it was feasible and rational for the sample panel with 30% normal human serum and 70% normal saline as dilution buffer.
The effects of different detection targets and methods should be further considered. Among the 19 participating agencies, Puruikang Bio-Tech Co., Ltd. and two provincial CDC offices used the GP/L site for detection, whereas the others chose the NP site. However, no significant differences (P = 0.462) in detection were found between the NP and GP/L sites. With respect to the various detection methods, the procedures for nucleic acid extraction and qualitative detection should be considered separately. First, we compared the paramagnetic particle and spin column methods for RNA extraction. Of the five participants that reported erroneous results, three used paramagnetic particle methods and two used spin column methods. No significant differences (P = 0.767) in overall detection were observed between the two methods for RNA extraction.
Second, the qualitative detection assays should be mentioned. Commercial detection assays were employed by 15 of the 19 participants, with an accuracy rate of 96.67%. Among them, the kits from Daan Gene and ZJ Bio-Tech were chosen with high frequency. Due to the small number of participating agencies, deep analysis of the various commercial reagents could not be conducted. In-house-developed real-time PCR assays using primers from the China CDC were adopted with 100% accuracy. Although institutions using common RT-PCR also reported 100% accurate results, this is not recommended, due to the importance of rapid diagnosis for EBOV control [14]. Notably, one participant failed to detect any of the positive samples in this evaluation. The primers used by this participant were designed for multiple types of EBOV, which likely decreased their analytic capability for single-type detection. For further identification of problems with detection, internal controls should be used, to exclude faults in the nucleic acid extraction process. The use of conventional and sensitive primers and probes for EBOV diagnosis is recommended.
Further improvement and optimization should be implemented, based on the feedback provided here. First, normalized detection assays provide accurate EBOV diagnosis. Commercial assays and the standard primers provided by the Chinese CDC were shown to be most trustworthy. Additionally, for all testing laboratories, it is advisable to implement a two-target assay approach, which could decrease the probability of false-negative results for viral mutations. Second, standardization of operations and punctual adjustment of instruments are also indispensable. In summary, effective and reliable EBOV detection requires recognition of problems and corresponding amendment of methods.
Supporting Information
S2 Fig. Comparison of sample panels stored at -80°C and through blind mail.
https://doi.org/10.1371/journal.pone.0132659.s002
(DOC)
Acknowledgments
We thank all the laboratories that participated in this EQA for their cooperation. We also thank the reagent manufacturers that provided the real-time RT-PCR kits for EBOV detection in this study.
Author Contributions
Conceived and designed the experiments: JML GJW. Performed the experiments: GJW DZ. Analyzed the data: GJW YS JML. Contributed reagents/materials/analysis tools: KZ TTJ LC LZ RZ GGL RXP MJH. Wrote the paper: GJW YS JML.
References
- 1.
World Health Organization. Ebola response roadmap situation report. 2015. 2. 4. Available: http://www.who.int/csr/disease/ebola/situation-reports/archive/en.
- 2. Leroy EM, Baize S, Mavoungou E, Apetrei C. Sequence analysis of the GP, NP, VP40 and VP24 genes of Ebola virus isolated from deceased, surviving and asymptpmatically infected individuals during the 1996 outbreak in Gabon: comparative studies and phylogenetic characterization. J Gen Virol. 2002;83: 67–73. pmid:11752702
- 3. Baize S, Pannetier D, Oestereich L, Rieger T, Koivogui L, Magassouba N, et al. Emergence of Zaire Ebola virus disease in Guinea—Preliminary Report. N Engl J Med. 2014;371: 1418–1425. pmid:24738640
- 4. Goodman JL. Studying “secret serums”–toward safe, effective Ebola treatments. N Engl J Med. 2014;371: 1086–1089. pmid:25140857
- 5. Briand S, Bertherat E, Cox P, Formenty P, Kieny MP, Myhre JK, et al. The international Ebola emergency. N Engl J Med. 2014;371: 1180–1183. pmid:25140855
- 6. Dhillon RS, Srikrishna D, Garry RF, Chowell G. Ebola control: rapid diagnostic testing. Lancet Infect Dis. 2015;15: 147–148.
- 7. Feldmann H. Ebola—a growing threat? N Engl J Med. 2014;371: 1375–1378. pmid:24805988
- 8. Frieden TR, Damon I, Bell BP, Kenyon T, Nichol S. Ebola 2014—new challenges new global response and responsibility. N Engl J Med. 2014;371: 1177–1180. pmid:25140858
- 9. Sun Y, Jia T, Sun Y, Han Y, Wang L, Zhang R, et al. External quality assessment for Avian Influenza A (H7N9) Virus detection using armored RNA. J Clin Microbiol. 2013;51: 4055–4059. pmid:24088846
- 10. Song L, Sun S, Li B, Pan Y, Li W, Zhang K, et al. External quality assessment for enterovirus 71 and coxsackievirus A16 detection by reverse transcription-PCR using armored RNA as a virus surrogate. J Clin Microbiol. 2011;49: 3591–3595. pmid:21865426
- 11. Schieffelin JS, Shaffer JG, Goba A, Gbakie M, Gire SK, Colubri A, et al. Clinical illness and outcomes in patients with Ebola in Sierra Leone. N Engl J Med. 2014;371: 2092–2100. pmid:25353969
- 12.
Chinese Center for disease control and prevention. The laboratory detection scheme for Ebola haemorrhagic fever (Third edition). Available: http://www.chinacdc.cn/jkzt/crb/ablcxr/jszl_2273.
- 13. WHO Ebola response team. Ebola virus disease in West Africa—the first 9 months of the epidemic and forward projections. N Engl J Med. 2014;371: 1481–1495. pmid:25244186
- 14. Drazen JM, Campion EW, Rubin EJ, Morrissey S, Baden LR. Ebola in West Africa at One Year—From Ignorance to Fear to Roadblocks. N Engl J Med. 2015;372: 563–564. pmid:25539445
- 15. Gao GF, Feng Y. On the ground in Sierra Leone. Science. 2014;346: 666. pmid:25359978
- 16. Zhang B, Gao GF. A new chapter for China's public health security-aids offered to Africa to combat Ebola. Sci China Life Sci. 2015;58: 114–116. pmid:25514849
- 17. Fan HJ, Gao HW, Ding H, Zhang BK, Hou SK. The Ebola threat: China's response to the West African epidemic and national development of prevention and control policies and infrastructure. Disaster Med Public Health Prep. 2015;9: 64–65. pmid:25563862
- 18. Drazen JM, Kanapathipillai R, Campion EW, Rubin EJ, Hammer SM, Morrissey S, et al. Ebola and quarantine. N Engl J Med. 2014;371: 2029–2030. pmid:25347231