Artificial intelligence to predict the BRAFV600E mutation in patients with thyroid cancer

Jiyoung Yoon; Eunjung Lee; Ja Seung Koo; Jung Hyun Yoon; Kee-Hyun Nam; Jandee Lee; Young Suk Jo; Hee Jung Moon; Vivian Youngjean Park; Jin Young Kwak

doi:10.1371/journal.pone.0242806

Abstract

Purpose

To investigate whether a computer-aided diagnosis (CAD) program developed using the deep learning convolutional neural network (CNN) on neck US images can predict the BRAF^V600E mutation in thyroid cancer.

Methods

469 thyroid cancers in 469 patients were included in this retrospective study. A CAD program recently developed using the deep CNN provided risks of malignancy (0–100%) as well as binary results (cancer or not). Using the CAD program, we calculated the risk of malignancy based on a US image of each thyroid nodule (CAD value). Univariate and multivariate logistic regression analyses were performed including patient demographics, the American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TIRADS) categories and risks of malignancy calculated through CAD to identify independent predictive factors for the BRAF^V600E mutation in thyroid cancer. The predictive power of the CAD value and final multivariable model for the BRAF^V600E mutation in thyroid cancer were measured using the area under the receiver operating characteristic (ROC) curves.

Results

In this study, 380 (81%) patients were positive and 89 (19%) patients were negative for the BRAF^V600E mutation. On multivariate analysis, older age (OR = 1.025, p = 0.018), smaller size (OR = 0.963, p = 0.006), and higher CAD value (OR = 1.016, p = 0.004) were significantly associated with the BRAF^V600E mutation. The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAF^V600E mutation, while the multivariable model yielded an AUC of 0.706 (95% CI: 0.576, 0.716). The multivariable model showed significantly better performance than the CAD value alone (p = 0.004).

Conclusion

Deep learning-based CAD for thyroid US can help us predict the BRAF^V600E mutation in thyroid cancer. More multi-center studies with more cases are needed to further validate our study results.

Citation: Yoon J, Lee E, Koo JS, Yoon JH, Nam K-H, Lee J, et al. (2020) Artificial intelligence to predict the BRAF^V600E mutation in patients with thyroid cancer. PLoS ONE 15(11): e0242806. https://doi.org/10.1371/journal.pone.0242806

Editor: Oded Cohen, Kaplan Medical Center, ISRAEL

Received: July 29, 2020; Accepted: November 9, 2020; Published: November 25, 2020

Copyright: © 2020 Yoon et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting information files.

Funding: This study was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) by the Ministry of Education (2016R1D1A1B03930375) and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2019R1A2C1002375). ). This study was also supported by a CMB-Yuhan research grant of Yonsei University College of Medicine (6-2017-0170). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: ACR, American College of Radiology; AI, Artificial intelligence; AJCC, American Joint Committee on Cancer; CAD, computer-aided diagnosis; CNN, convolutional neural network; FNA, fine needle aspiration; PTC, papillary thyroid cancer; ROI, region-of-interest; TIRADS, Thyroid Imaging, Reporting and Data System; US, ultrasonography; VIF, variance inflation factors

Introduction

The BRAF^V600E mutation is the most commonly detected oncogene in thyroid cancer and is highly specific for papillary thyroid cancer (PTC) [1, 2]. Because of this high specificity, the mutation has been used in diagnostic methods adjunctive to fine needle aspiration (FNA) for thyroid nodules, especially those with indeterminate cytology results [1–5]. Also, the BRAF^V600E mutation is a known predictor of aggressive PTCs as it has been associated with higher cancer stage and a higher rate of extrathyroidal extension and lymph node metastases [6–8]. Considering that ultrasonography (US) features have also been associated with the BRAF^V600E mutation [3–5, 9], we can assume that the mutation test is a cost-effective tool for thyroid nodules with suspicious US features. However, US itself is inherently limited by its subjectivity, leading to the low reproducibility of its results [10, 11].

Artificial intelligence (AI) is being widely studied in the medical field with various applications being thought feasible in actual healthcare systems [12]. When diagnosing thyroid nodules on US, AI has shown comparable diagnostic accuracies to radiologists, with publications reporting the accuracies to be from 83% to 98% [13–15]. Compared to how diagnostic performances can vary according to the operator’s level of experience, AI can provide more objective results for US.

Several previous studies have focused on US features to predict BRAF status, and these studies have found that US characteristics associated with malignancy (marked hypoechogenicity, taller-than-wide shape, etc.) are also associated with BRAF positivity [9, 16, 17]. However, to the best of our knowledge, no studies have associated the BRAF^V600E mutation with US features using a computer-aided diagnosis (CAD) program. Therefore, we investigated whether a CAD program that was recently developed using the deep convolutional neural network (CNN) to diagnose thyroid cancer from US images of thyroid nodules could also help predict the BRAF^V600E mutation in thyroid nodules.

Materials & methods

This retrospective study was approved by the Severance Hospital Institutional Review Board, with a waiver for patient consent (Approval number: 4-2019-1223). Signed informed consent was obtained from all patients prior to biopsy or surgical procedures.

Patients

We collected patient data from Feburary 2019 to September 2019, during which 1817 patients were referred to our institution for thyroid surgery after undergoing biopsies at outside clinics and preoperative staging US at our institution, a tertiary referral center. Among them, 1348 patients were excluded because they were younger than 19 years old (n = 8), had nodules smaller than 10mm on US (n = 993), did not proceed with surgery (n = 230), had a final pathology of benignity (n = 87), and did not have the BRAF test performed on a pathologic specimen (n = 30). Finally, 469 thyroid cancers in 469 patients were included. Mean age of the patients was 42.3 years ± 12.8 (range, 19–86 years). Mean size of the tumors was 16.9 mm ± 8.3 (range, 10–62 mm). There were 353 (75.3%) women and 116 (24.7%) men.

US examinations

Preoperative staging US was performed with a 5–12 MHz linear array transducer (iU22; Philips Medical Systems or EPIQ 7; Philips Medical Systems, Bothell, WA, USA) by one of nine radiologists specializing in thyroid imaging. All physicians were informed of previous cytology results and the location of the index lesion that underwent FNA or biopsy at outside clinics. Since June 2012, the physicians in our institution who perform staging US have filled out a report format for index nodules that includes nodule size and US features such as composition (solid, predominantly cyst, predominantly solid, spongiform, cyst), echogenicity (hyper-, iso-, hypo- and marked hypoechoic), margin (well, microlobulated, irregular), shape (parallel, not parallel) and calcification (eggshell, micro-, macro- and mixed calcification) [18]. One radiologist (K.J.Y.) who had 19 years of experience in thyroid imaging re-assigned the nodules according to the American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TIRADS) using the prospectively recorded US features. Suspicious lymph nodes on US were also evaluated at the time of staging US. Finally, clinical T and N stages were reported according to the 8th American Joint Committee on Cancer (AJCC) cancer staging system [19].

Image acquisition and CNN evaluation

One radiologist (K.J.Y) who had 19 years of experience in thyroid imaging reviewed the preoperative US images, selected a representative image for each thyroid tumor, saved it as a JPEG file in the picture archiving and communication system, and drew square region-of-interests (ROIs) to cover the whole nodule using the Microsoft Paint program (version 6.1; Microsoft Corporation, Redmond, WA, USA).

Recently, we developed a CAD program to diagnose thyroid cancer on US from 13,560 US images of thyroid nodules using the deep CNN (see S1 File for details on the CAD program). The CAD program provides risks of malignancy (0–100%) as well as binary results (cancer or not). Using the CAD program, we calculated the risks of malignancy from US images of the thyroid nodules (CAD value) (Figs 1 and 2).

Download:

Fig 1. A 33-year-old woman with a thyroid nodule confirmed as suspicious for malignancy on US guided-fine needle aspiration (US-FNA).

https://doi.org/10.1371/journal.pone.0242806.g001

Download:

Fig 2. A 40-year-old woman with a thyroid nodule confirmed as suspicious for malignancy on US-FNA.

https://doi.org/10.1371/journal.pone.0242806.g002

US showed a 13 mm-sized solid, hypoechoic, and taller-than-wide shaped nodule with lobulated margin and punctate echogenic foci, which was categorized as “highly suspicious” on the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TIRADS) (a). The computer-aided diagnosis (CAD) program calculated the risk of malignancy as 98.86% (CAD value). (b). The patient underwent total thyroidectomy and final pathology was conventional papillary thyroid carcinoma. Pyrosequencing was positive for the BRAF^V600E mutation.

US shows a 34 mm-sized predominantly solid, hypoechoic, and wider-than-tall shaped nodule with smooth margin and without echogenic foci, which was categorized as “mildly suspicious” on the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TIRADS). (a). The computer-aided diagnosis (CAD) program calculated the risk of malignancy as 3.71% (CAD value) (b). The patient underwent left lobectomy and the final pathology was the follicular variant of papillary thyroid carcinoma. Pyrosequencing was negative for the BRAF^V600E mutation.

BRAF^V600E mutation analysis

BRAF V600E mutants were evaluated using pyrosequencing per the manufacturer’s instructions. PCR amplification was performed with a forward primer (5′- biotin- TTCTTCATGAAGACCTCACAGTAA-3′) and a reverse primer (5′- CCAGACAACTGTTCAAACTGATG-3′) on a C1000 thermal cycler (BIO-RAD, California, USA). The pyrosequencing reaction was performed with a sequencing primer (5′- GGACCCACTCCCATCGAGATTT-3′) on a Pyromark Q24 instrument (Qiagen). The produced pyrogram was analyzed with the PyroMark Q24 software (Qiagen) to distinguish mutant versus wild-type alleles by relative peak height.

Statistical analysis

We compared the demographics, ACR TIRADS categories and risks of malignancy from CAD between patients with and without the BRAF^V600E mutation using the independent two-sample t-test for continuous data and the Chi-square test or Fisher’s exact test for categorical data. Univariate and multivariate logistic regression analyses were performed to identify independent factors for predicting the BRAF^V600E mutation in thyroid cancer. As the ACR TIRADS and CAD values were assessed from the US images of thyroid nodules which could be correlated with each other, multicollinearity between the ACR TIRADS and CAD values was assessed with variance inflation factors (VIF). Multicollinearity was considered high when the VIF was > 10.

The multivariable model was only constructed with factors independently associated with the BRAF^V600E mutation. The predictive power of the CAD value and the multivariable model for the BRAF^V600E mutation in thyroid cancer were measured and compared with the area under the receiver operating characteristic (ROC) curves. The optimal cutoff for the CAD value to predict the BRAF^V600E mutation was identified using the Youden index. The performances of the CAD value and the multivariable model were compared using Delong’s method. All statistical analyses in this study were performed with SPSS statistical software (SPSS for Windows, version 25.0; IBM Corporation, Armonk, NY) and R (version 4.0.2.; R Foundation for Statistical Computing, Vienna, Austria). P-values of less than 0.05 were considered to indicate statistical significance.

Results

In this study, 380 (81%) patients were positive and 89 (19%) patients were negative for the BRAF^V600E mutation. There were 422 (90%) conventional papillary thyroid carcinomas (PTCs), 26 (5.6%) follicular variant PTC, 9 (1.9%) diffuse sclerosing variant PTCs, 4 (0.9%) solid variant PTCs, 2 (0.4%) tall cell variant PTCs, 2 (0.4%) oncocytic variant PTCs, 1 (0.2%) Hobnail variant PTC, 1 (0.2%) Warthin-like variant PTC, 1 (0.2%) minimally invasive follicular carcinoma, and 1 (0.2%) medullary carcinoma.

Table 1 summarizes patient demographics and clinical characteristics according to the BRAF^V600E mutation. Patients with the BRAF^V600E mutation were older and had tumors of smaller size, higher ACR TIRADS scores, and higher CAD values than ones without the mutation.

Download:

Table 1. Comparison of patient demographics, ACR TIRADS categories and CAD values according to the BRAF^V600E mutation status.

https://doi.org/10.1371/journal.pone.0242806.t001

To determine potential predictors directly related to the BRAF^V600E mutation, univariate and multivariate logistic regression were performed. On univariate analysis, the BRAF^V600E mutation was associated with older age, smaller size, higher ACR TIRADS scores, and higher CAD values (Table 2). Multivariate logistic regression analysis was performed to assess the independent associations of the BRAF^V600E mutation with clinical factors. As the ACR TIRADS and CAD values did not show multicollinearity in the model (VIF was 1.366), we used both parameters in the regression model. On multivariate analysis, older age, smaller size, and higher CAD values were significantly associated with the BRAF^V600E mutation (Table 2).

Download:

Table 2. Univariate and multivariate logistic regression analysis for clinical factors based on the presence or absence of the BRAF^V600E mutation in thyroid carcinoma patients.

https://doi.org/10.1371/journal.pone.0242806.t002

The multivariable logistic model was constructed using age, size, and the CAD value to predict the BRAF^V600E mutation. The ROC curves of the CAD value and the multivariable model were plotted to show the performances of the CAD value and the multivariable model for predicting the BRAF^V600E mutation in thyroid cancer patients. The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAF^V600E mutation. The cutoff for the CAD value to obtain maximum accuracy for the diagnosis of the BRAF^V600E mutation was 57.7. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the CAD value for predicting the BRAF^V600E mutation was 85.3%, 41.6%, 59.4%, and 73.9%, respectively. The multivariable model yielded an AUC of 0.706 (95% CI: 0.644, 0.767) for predicting the BRAF^V600E mutation. The multivariable model showed significantly better performance than the CAD value only (p = 0.004) (Fig 3).

Download:

Fig 3. Receiver operating characteristic curves of the CAD value and the multivariable logistic model for predicting the BRAF^V600E mutation.

https://doi.org/10.1371/journal.pone.0242806.g003

The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAF^V600E mutation, while the multivariable model yielded an AUC of 0.706 (95% CI: 0.576, 0.716). The multivariable model showed significantly better performance than the CAD value only (p = 0.004).

Discussion

The prevalence of the BRAF mutation in PTC ranges from 29% to 83% and almost all of the mutations are the BRAF^V600E form [20]. As the BRAF^V600E mutation is highly specific for PTC [21], a combination of cytology and the BRAF^V600E mutation has been considered a good diagnostic approach for thyroid nodules with indeterminate results on cytology [1]. Furthermore, the BRAF^V600E mutation is a known predictor of aggressive PTC; previous studies have reported that thyroid malignancies with the BRAF^V600E mutation present with higher cancer stage and higher rate of extrathyroidal extension and lymph node metastases than those without the mutation [6–8]. As the BRAF^V600E mutation can be used in both diagnosing thyroid cancer and predicting its aggressiveness, preoperative detection of the mutation may also help optimize the initial operative approach in patients with thyroid nodules [22–24]. However, the mutation analysis requires invasive procedures such as biopsy or surgical resection to retrieve specimens.

Several previous studies have tried to identify US features that can predict BRAF status [9, 16, 17]. A recent review article concluded that US features of PTCs correlate with their particular molecular mutations, and BRAF^V600E mutation is known to be associated with suspicious US findings, such as hypoechogenicity, non-parallel orientation, taller-than-wide shape, spiculated/microlobulated margins, and the presence of microcalcifications [25]. However, interpretation of US image is operator-dependent and inter-observer variability is moderate to substantial [10, 11, 26]. Radiomics is one of the emerging methods for predicting molecular characteristics of tumors, using quantitative imaging features extracted using data-characterization algorithms [27, 28]. A recent study evaluated the value of US-based radiomics for predicting BRAF^V600E mutation in pathologically proven PTCs, but radiomics features extracted from US had limited value [29].

In our study, the ACR TIRADS category assessed by radiologists was associated with the BRAF^V600E mutation in univariate analysis, but the correlation disappeared in multivariable analysis. In contrast, the CAD value, which was calculated from a program that we developed using deep CNN, was associated with the BRAF^V600E mutation in multivariable analysis. Our findings with the AUCs of the ROC curve indicate that the CAD value can help predict the BRAF^V600E mutation in thyroid cancer. Furthermore, the multivariable model which was composed with patient age, tumor size, and the CAD value showed significantly increased predictability. Considering that a deep learning algorithm allows consistent prediction, we can avoid or even overcome problems arising from inter-observer variability in US evaluations with the CAD value [30]. Therefore, the CAD program used in this study is expected to be a non-invasive and objective biomarker for prediction of the BRAF^V600E mutation in PTCs.

A number of CAD algorithms have been developed and implemented for the accurate diagnosis of disease, including binary logistic regression, support vector machines, and artificial neural networks [31]. Among them, deep learning with CNN has recently gained attention for its high performance in image recognition, and other researchers have developed deep learning CNN models by using radiological images for lesion detection, lesion evaluation, estimation of patient survival, etc. [32]. Many CNN models have already benefited clinical practice and this is especially so for thyroid malignancies, as deep learning CNN models developed for the detection, segmentation, and classification of thyroid nodules using neck US images have shown good performances [13, 14, 33–36]. A recent study reported that a deep learning CNN model developed to discriminate the BRAF^V600E and RAS mutations of PTCs from histological images showed 95% accurate predictions [37]. However, ours is the first study using deep learning-based CAD to predict the BRAF^V600E mutation in patients diagnosed with thyroid cancer. Previous studies reported associations between suspicious US features (such as marked hypoechogenicity and taller-than-wide shape) and the BRAF mutation, but most of these features showed weak correlation and the study results were inconsistent, probably due to intra- and inter-observer variability [9, 10, 16, 17]. As the CAD program provides objective probabilities of malignancy, it can potentially help diagnose BRAF^V600E mutations with more accuracy.

There are several limitations to this study. First, our study was of retrospective design and patient data were collected from a single tertiary referral center, which means a selection bias was inevitable. Second, an experienced radiologist retrospectively re-assigned categories to thyroid nodules according to ACR TIRADS using US images prospectively recorded by 9 radiologists. The re-assigned categories might not have fully reflected the prospectively recorded US features. Third, the correlation between the CAD value and cancer prognosis could not be confirmed due to the lack of a follow-up period. Further studies are needed to confirm whether the CAD value can be a predictor of cancer prognosis. Last, the prevalence of BRAF^V600E mutations among PTCs differs from country to country. The Korean population is known for its high mutation prevalence and 81% of the PTCs in our study showed the BRAF^V600E mutation [38]. Results might differ when the same study is conducted on populations from other countries.

In conclusion, the CAD program developed with deep learning can help predict the BRAF^V600E mutation in thyroid cancer. More multi-center studies with more cases are needed to further validate our study results.

Supporting information

S1 File. Computer-aided diagnosis (CAD) program using the convolutional neural network (CNN).

https://doi.org/10.1371/journal.pone.0242806.s001

(DOCX)

References

1. Seo JY, Kim EK, Baek JH, Shin JH, Han KH, Kwak JY. Can ultrasound be as a surrogate marker for diagnosing a papillary thyroid cancer? Comparison with BRAF mutation analysis. Yonsei medical journal. 2014;55(4):871–8.
- View Article
- Google Scholar
2. Moon HJ, Kwak JY, Kim EK, Choi JR, Hong SW, Kim MJ, et al. The role of BRAFV600E mutation and ultrasonography for the surgical management of a thyroid nodule suspicious for papillary thyroid carcinoma on cytology. Ann Surg Oncol. 2009;16(11):3125–31. pmid:19644722
- View Article
- PubMed/NCBI
- Google Scholar
3. Rho M, Kim EK, Moon HJ, Yoon JH, Park VY, Han K, et al. Clinical Parameter for Deciding the BRAFV600E Mutation Test in Atypia of Undetermined Significance/Follicular Lesion of Undetermined Significance Thyroid Nodules: US Features According to TIRADS. Ultrasound Q. 2017;33(4):284–8. pmid:28877096
- View Article
- PubMed/NCBI
- Google Scholar
4. Moon HJ, Kim EK, Chung WY, Choi JR, Yoon JH, Kwak JY. Diagnostic value of BRAF(V600E) mutation analysis of thyroid nodules according to ultrasonographic features and the time of aspiration. Ann Surg Oncol. 2011;18(3):792–9. pmid:20945104
- View Article
- PubMed/NCBI
- Google Scholar
5. Lee EJ, Song KH, Kim DL, Jang YM, Hwang TS, Kim SK. The BRAF(V600E) mutation is associated with malignant ultrasonographic features in thyroid nodules. Clin Endocrinol (Oxf). 2011;75(6):844–50. pmid:21707687
- View Article
- PubMed/NCBI
- Google Scholar
6. Xing M. BRAF mutation in papillary thyroid cancer: pathogenic role, molecular bases, and clinical implications. Endocrine reviews. 2007;28(7):742–62. pmid:17940185
- View Article
- PubMed/NCBI
- Google Scholar
7. Park JY, Yi JW, Park CH, Lim Y, Lee KH, Lee KE, et al. Role of BRAF and RAS Mutations in Extrathyroidal Extension in Papillary Thyroid Cancer. Cancer genomics & proteomics. 2016;13(2):171–81. pmid:26912807
- View Article
- PubMed/NCBI
- Google Scholar
8. Lee KC, Li C, Schneider EB, Wang Y, Somervell H, Krafft M, et al. Is BRAF mutation associated with lymph node metastasis in patients with papillary thyroid cancer? Surgery. 2012;152(6):977–83. pmid:23062653
- View Article
- PubMed/NCBI
- Google Scholar
9. Kabaker AS, Tublin ME, Nikiforov YE, Armstrong MJ, Hodak SP, Stang MT, et al. Suspicious ultrasound characteristics predict BRAF V600E-positive papillary thyroid carcinoma. Thyroid. 2012;22(6):585–9. pmid:22524468
- View Article
- PubMed/NCBI
- Google Scholar
10. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ. Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid. 2010;20(2):167–72. pmid:19725777
- View Article
- PubMed/NCBI
- Google Scholar
11. Kim HG, Kwak JY, Kim EK, Choi SH, Moon HJ. Man to man training: can it help improve the diagnostic performances and interobserver variabilities of thyroid ultrasonography in residents? European journal of radiology. 2012;81(3):e352–6. pmid:22137098
- View Article
- PubMed/NCBI
- Google Scholar
12. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC medicine. 2019;17(1):195. pmid:31665002
- View Article
- PubMed/NCBI
- Google Scholar
13. Ma J, Wu F, Zhu J, Xu D, Kong D. A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics. 2017;73:221–30. pmid:27668999
- View Article
- PubMed/NCBI
- Google Scholar
14. Ko SY, Lee JH, Yoon JH, Na H, Hong E, Han K, et al. Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound. Head Neck. 2019;41(4):885–91. pmid:30715773
- View Article
- PubMed/NCBI
- Google Scholar
15. Li X, Zhang S, Zhang Q, Wei X, Pan Y, Zhao J, et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. The Lancet Oncology. 2019;20(2):193–201. pmid:30583848
- View Article
- PubMed/NCBI
- Google Scholar
16. Kwak JY, Kim EK, Chung WY, Moon HJ, Kim MJ, Choi JR. Association of BRAFV600E mutation with poor clinical prognostic factors and US features in Korean patients with papillary thyroid microcarcinoma. Radiology. 2009;253(3):854–60. pmid:19710001
- View Article
- PubMed/NCBI
- Google Scholar
17. Hwang J, Shin JH, Han BK, Ko EY, Kang SS, Kim JW, et al. Papillary thyroid carcinoma with BRAFV600E mutation: sonographic prediction. AJR American journal of roentgenology. 2010;194(5):W425–30. pmid:20410389
- View Article
- PubMed/NCBI
- Google Scholar
18. Yoon JH, Lee HS, Kim EK, Moon HJ, Kwak JY. Malignancy Risk Stratification of Thyroid Nodules: Comparison between the Thyroid Imaging Reporting and Data System and the 2014 American Thyroid Association Management Guidelines. Radiology. 2016;278(3):917–24. pmid:26348102
- View Article
- PubMed/NCBI
- Google Scholar
19. Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, et al. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more "personalized" approach to cancer staging. CA: a cancer journal for clinicians. 2017;67(2):93–9. pmid:28094848
- View Article
- PubMed/NCBI
- Google Scholar
20. Trovisco V, Soares P, Sobrinho-Simoes M. B-RAF mutations in the etiopathogenesis, diagnosis, and prognosis of thyroid carcinomas. Human pathology. 2006;37(7):781–6. pmid:16784975
- View Article
- PubMed/NCBI
- Google Scholar
21. Cohen Y, Xing M, Mambo E, Guo Z, Wu G, Trink B, et al. BRAF mutation in papillary thyroid carcinoma. Journal of the National Cancer Institute. 2003;95(8):625–7. pmid:12697856
- View Article
- PubMed/NCBI
- Google Scholar
22. Melck AL, Yip L, Carty SE. The utility of BRAF testing in the management of papillary thyroid cancer. The oncologist. 2010;15(12):1285–93. pmid:21147872
- View Article
- PubMed/NCBI
- Google Scholar
23. O’Neill CJ, Bullock M, Chou A, Sidhu SB, Delbridge LW, Robinson BG, et al. BRAF(V600E) mutation is associated with an increased risk of nodal recurrence requiring reoperative surgery in patients with papillary thyroid cancer. Surgery. 2010;148(6):1139–45; discussion 45–6. pmid:21134544
- View Article
- PubMed/NCBI
- Google Scholar
24. Yip L, Nikiforova MN, Carty SE, Yim JH, Stang MT, Tublin MJ, et al. Optimizing surgical treatment of papillary thyroid carcinoma associated with BRAF mutation. Surgery. 2009;146(6):1215–23. pmid:19958951
- View Article
- PubMed/NCBI
- Google Scholar
25. Lewinski A, Adamczewski Z, Zygmunt A, Markuszewski L, Karbownik-Lewinska M, Stasiak M. Correlations between Molecular Landscape and Sonographic Image of Different Variants of Papillary Thyroid Carcinoma. Journal of clinical medicine. 2019;8(11). pmid:31717363
- View Article
- PubMed/NCBI
- Google Scholar
26. Park CS, Kim SH, Jung SL, Kang BJ, Kim JY, Choi JJ, et al. Observer variability in the sonographic evaluation of thyroid nodules. J Clin Ultrasound. 2010;38(6):287–93. pmid:20544863
- View Article
- PubMed/NCBI
- Google Scholar
27. Gevaert O, Echegaray S, Khuong A, Hoang CD, Shrager JB, Jensen KC, et al. Predictive radiogenomics modeling of EGFR mutation status in lung cancer. Sci Rep. 2017;7:41674. pmid:28139704
- View Article
- PubMed/NCBI
- Google Scholar
28. Braman NM, Etesami M, Prasanna P, Dubchuk C, Gilmore H, Tiwari P, et al. Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI. Breast cancer research: BCR. 2017;19(1):57. pmid:28521821
- View Article
- PubMed/NCBI
- Google Scholar
29. Yoon JH, Han K, Lee E, Lee J, Kim EK, Moon HJ, et al. Radiomics in predicting mutation status for thyroid cancer: A preliminary study using radiomics features for predicting BRAFV600E mutations in papillary thyroid carcinoma. PloS one. 2020;15(2):e0228968. pmid:32053670
- View Article
- PubMed/NCBI
- Google Scholar
30. Buda M, Wildman-Tobriner B, Hoang JK, Thayer D, Tessler FN, Middleton WD, et al. Management of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists. Radiology. 2019;292(3):695–701. pmid:31287391
- View Article
- PubMed/NCBI
- Google Scholar
31. Lim KJ, Choi CS, Yoon DY, Chang SK, Kim KK, Han H, et al. Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Academic radiology. 2008;15(7):853–8. pmid:18572120
- View Article
- PubMed/NCBI
- Google Scholar
32. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O. Deep learning with convolutional neural network in radiology. Japanese journal of radiology. 2018;36(4):257–72. pmid:29498017
- View Article
- PubMed/NCBI
- Google Scholar
33. Song W, Li S, Liu J, Qin H, Zhang B, Zhang S, et al. Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition. IEEE journal of biomedical and health informatics. 2019;23(3):1215–24. pmid:29994412
- View Article
- PubMed/NCBI
- Google Scholar
34. Ma J, Wu F, Jiang T, Zhao Q, Kong D. Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. International journal of computer assisted radiology and surgery. 2017;12(11):1895–910. pmid:28762196
- View Article
- PubMed/NCBI
- Google Scholar
35. Li H, Weng J, Shi Y, Gu W, Mao Y, Wang Y, et al. An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images. Sci Rep. 2018;8(1):6600. pmid:29700427
- View Article
- PubMed/NCBI
- Google Scholar
36. Ma J, Wu F, Jiang T, Zhu J, Kong D. Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Medical physics. 2017;44(5):1678–91. pmid:28186630
- View Article
- PubMed/NCBI
- Google Scholar
37. Tsou P, Wu CJ. Mapping Driver Mutations to Histopathological Subtypes in Papillary Thyroid Carcinoma: Applying a Deep Convolutional Neural Network. Journal of clinical medicine. 2019;8(10). pmid:31614962
- View Article
- PubMed/NCBI
- Google Scholar
38. Kim KH, Kang DW, Kim SH, Seong IO, Kang DY. Mutations of the BRAF gene in papillary thyroid carcinoma in a Korean population. Yonsei medical journal. 2004;45(5):818–21. pmid:15515191
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Seo JY, Kim EK, Baek JH, Shin JH, Han KH, Kwak JY. Can ultrasound be as a surrogate marker for diagnosing a papillary thyroid cancer? Comparison with BRAF mutation analysis. Yonsei medical journal. 2014;55(4):871–8.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Moon HJ, Kwak JY, Kim EK, Choi JR, Hong SW, Kim MJ, et al. The role of BRAFV600E mutation and ultrasonography for the surgical management of a thyroid nodule suspicious for papillary thyroid carcinoma on cytology. Ann Surg Oncol. 2009;16(11):3125–31. pmid:19644722
View Article
PubMed/NCBI
Google Scholar

[5] View Article

[6] PubMed/NCBI

[7] Google Scholar

[ref3] 3. Rho M, Kim EK, Moon HJ, Yoon JH, Park VY, Han K, et al. Clinical Parameter for Deciding the BRAFV600E Mutation Test in Atypia of Undetermined Significance/Follicular Lesion of Undetermined Significance Thyroid Nodules: US Features According to TIRADS. Ultrasound Q. 2017;33(4):284–8. pmid:28877096
View Article
PubMed/NCBI
Google Scholar

[9] View Article

[10] PubMed/NCBI

[11] Google Scholar

[ref4] 4. Moon HJ, Kim EK, Chung WY, Choi JR, Yoon JH, Kwak JY. Diagnostic value of BRAF(V600E) mutation analysis of thyroid nodules according to ultrasonographic features and the time of aspiration. Ann Surg Oncol. 2011;18(3):792–9. pmid:20945104
View Article
PubMed/NCBI
Google Scholar

[13] View Article

[14] PubMed/NCBI

[15] Google Scholar

[ref5] 5. Lee EJ, Song KH, Kim DL, Jang YM, Hwang TS, Kim SK. The BRAF(V600E) mutation is associated with malignant ultrasonographic features in thyroid nodules. Clin Endocrinol (Oxf). 2011;75(6):844–50. pmid:21707687
View Article
PubMed/NCBI
Google Scholar

[17] View Article

[18] PubMed/NCBI

[19] Google Scholar

[ref6] 6. Xing M. BRAF mutation in papillary thyroid cancer: pathogenic role, molecular bases, and clinical implications. Endocrine reviews. 2007;28(7):742–62. pmid:17940185
View Article
PubMed/NCBI
Google Scholar

[21] View Article

[22] PubMed/NCBI

[23] Google Scholar

[ref7] 7. Park JY, Yi JW, Park CH, Lim Y, Lee KH, Lee KE, et al. Role of BRAF and RAS Mutations in Extrathyroidal Extension in Papillary Thyroid Cancer. Cancer genomics & proteomics. 2016;13(2):171–81. pmid:26912807
View Article
PubMed/NCBI
Google Scholar

[25] View Article

[26] PubMed/NCBI

[27] Google Scholar

[ref8] 8. Lee KC, Li C, Schneider EB, Wang Y, Somervell H, Krafft M, et al. Is BRAF mutation associated with lymph node metastasis in patients with papillary thyroid cancer? Surgery. 2012;152(6):977–83. pmid:23062653
View Article
PubMed/NCBI
Google Scholar

[29] View Article

[30] PubMed/NCBI

[31] Google Scholar

[ref9] 9. Kabaker AS, Tublin ME, Nikiforov YE, Armstrong MJ, Hodak SP, Stang MT, et al. Suspicious ultrasound characteristics predict BRAF V600E-positive papillary thyroid carcinoma. Thyroid. 2012;22(6):585–9. pmid:22524468
View Article
PubMed/NCBI
Google Scholar

[33] View Article

[34] PubMed/NCBI

[35] Google Scholar

[ref10] 10. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ. Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid. 2010;20(2):167–72. pmid:19725777
View Article
PubMed/NCBI
Google Scholar

[37] View Article

[38] PubMed/NCBI

[39] Google Scholar

[ref11] 11. Kim HG, Kwak JY, Kim EK, Choi SH, Moon HJ. Man to man training: can it help improve the diagnostic performances and interobserver variabilities of thyroid ultrasonography in residents? European journal of radiology. 2012;81(3):e352–6. pmid:22137098
View Article
PubMed/NCBI
Google Scholar

[41] View Article

[42] PubMed/NCBI

[43] Google Scholar

[ref12] 12. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC medicine. 2019;17(1):195. pmid:31665002
View Article
PubMed/NCBI
Google Scholar

[45] View Article

[46] PubMed/NCBI

[47] Google Scholar

[ref13] 13. Ma J, Wu F, Zhu J, Xu D, Kong D. A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics. 2017;73:221–30. pmid:27668999
View Article
PubMed/NCBI
Google Scholar

[49] View Article

[50] PubMed/NCBI

[51] Google Scholar

[ref14] 14. Ko SY, Lee JH, Yoon JH, Na H, Hong E, Han K, et al. Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound. Head Neck. 2019;41(4):885–91. pmid:30715773
View Article
PubMed/NCBI
Google Scholar

[53] View Article

[54] PubMed/NCBI

[55] Google Scholar

[ref15] 15. Li X, Zhang S, Zhang Q, Wei X, Pan Y, Zhao J, et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. The Lancet Oncology. 2019;20(2):193–201. pmid:30583848
View Article
PubMed/NCBI
Google Scholar

[57] View Article

[58] PubMed/NCBI

[59] Google Scholar

[ref16] 16. Kwak JY, Kim EK, Chung WY, Moon HJ, Kim MJ, Choi JR. Association of BRAFV600E mutation with poor clinical prognostic factors and US features in Korean patients with papillary thyroid microcarcinoma. Radiology. 2009;253(3):854–60. pmid:19710001
View Article
PubMed/NCBI
Google Scholar

[61] View Article

[62] PubMed/NCBI

[63] Google Scholar

[ref17] 17. Hwang J, Shin JH, Han BK, Ko EY, Kang SS, Kim JW, et al. Papillary thyroid carcinoma with BRAFV600E mutation: sonographic prediction. AJR American journal of roentgenology. 2010;194(5):W425–30. pmid:20410389
View Article
PubMed/NCBI
Google Scholar

[65] View Article

[66] PubMed/NCBI

[67] Google Scholar

[ref18] 18. Yoon JH, Lee HS, Kim EK, Moon HJ, Kwak JY. Malignancy Risk Stratification of Thyroid Nodules: Comparison between the Thyroid Imaging Reporting and Data System and the 2014 American Thyroid Association Management Guidelines. Radiology. 2016;278(3):917–24. pmid:26348102
View Article
PubMed/NCBI
Google Scholar

[69] View Article

[70] PubMed/NCBI

[71] Google Scholar

[ref19] 19. Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, et al. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more "personalized" approach to cancer staging. CA: a cancer journal for clinicians. 2017;67(2):93–9. pmid:28094848
View Article
PubMed/NCBI
Google Scholar

[73] View Article

[74] PubMed/NCBI

[75] Google Scholar

[ref20] 20. Trovisco V, Soares P, Sobrinho-Simoes M. B-RAF mutations in the etiopathogenesis, diagnosis, and prognosis of thyroid carcinomas. Human pathology. 2006;37(7):781–6. pmid:16784975
View Article
PubMed/NCBI
Google Scholar

[77] View Article

[78] PubMed/NCBI

[79] Google Scholar

[ref21] 21. Cohen Y, Xing M, Mambo E, Guo Z, Wu G, Trink B, et al. BRAF mutation in papillary thyroid carcinoma. Journal of the National Cancer Institute. 2003;95(8):625–7. pmid:12697856
View Article
PubMed/NCBI
Google Scholar

[81] View Article

[82] PubMed/NCBI

[83] Google Scholar

[ref22] 22. Melck AL, Yip L, Carty SE. The utility of BRAF testing in the management of papillary thyroid cancer. The oncologist. 2010;15(12):1285–93. pmid:21147872
View Article
PubMed/NCBI
Google Scholar

[85] View Article

[86] PubMed/NCBI

[87] Google Scholar

[ref23] 23. O’Neill CJ, Bullock M, Chou A, Sidhu SB, Delbridge LW, Robinson BG, et al. BRAF(V600E) mutation is associated with an increased risk of nodal recurrence requiring reoperative surgery in patients with papillary thyroid cancer. Surgery. 2010;148(6):1139–45; discussion 45–6. pmid:21134544
View Article
PubMed/NCBI
Google Scholar

[89] View Article

[90] PubMed/NCBI

[91] Google Scholar

[ref24] 24. Yip L, Nikiforova MN, Carty SE, Yim JH, Stang MT, Tublin MJ, et al. Optimizing surgical treatment of papillary thyroid carcinoma associated with BRAF mutation. Surgery. 2009;146(6):1215–23. pmid:19958951
View Article
PubMed/NCBI
Google Scholar

[93] View Article

[94] PubMed/NCBI

[95] Google Scholar

[ref25] 25. Lewinski A, Adamczewski Z, Zygmunt A, Markuszewski L, Karbownik-Lewinska M, Stasiak M. Correlations between Molecular Landscape and Sonographic Image of Different Variants of Papillary Thyroid Carcinoma. Journal of clinical medicine. 2019;8(11). pmid:31717363
View Article
PubMed/NCBI
Google Scholar

[97] View Article

[98] PubMed/NCBI

[99] Google Scholar

[ref26] 26. Park CS, Kim SH, Jung SL, Kang BJ, Kim JY, Choi JJ, et al. Observer variability in the sonographic evaluation of thyroid nodules. J Clin Ultrasound. 2010;38(6):287–93. pmid:20544863
View Article
PubMed/NCBI
Google Scholar

[101] View Article

[102] PubMed/NCBI

[103] Google Scholar

[ref27] 27. Gevaert O, Echegaray S, Khuong A, Hoang CD, Shrager JB, Jensen KC, et al. Predictive radiogenomics modeling of EGFR mutation status in lung cancer. Sci Rep. 2017;7:41674. pmid:28139704
View Article
PubMed/NCBI
Google Scholar

[105] View Article

[106] PubMed/NCBI

[107] Google Scholar

[ref28] 28. Braman NM, Etesami M, Prasanna P, Dubchuk C, Gilmore H, Tiwari P, et al. Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI. Breast cancer research: BCR. 2017;19(1):57. pmid:28521821
View Article
PubMed/NCBI
Google Scholar

[109] View Article

[110] PubMed/NCBI

[111] Google Scholar

[ref29] 29. Yoon JH, Han K, Lee E, Lee J, Kim EK, Moon HJ, et al. Radiomics in predicting mutation status for thyroid cancer: A preliminary study using radiomics features for predicting BRAFV600E mutations in papillary thyroid carcinoma. PloS one. 2020;15(2):e0228968. pmid:32053670
View Article
PubMed/NCBI
Google Scholar

[113] View Article

[114] PubMed/NCBI

[115] Google Scholar

[ref30] 30. Buda M, Wildman-Tobriner B, Hoang JK, Thayer D, Tessler FN, Middleton WD, et al. Management of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists. Radiology. 2019;292(3):695–701. pmid:31287391
View Article
PubMed/NCBI
Google Scholar

[117] View Article

[118] PubMed/NCBI

[119] Google Scholar

[ref31] 31. Lim KJ, Choi CS, Yoon DY, Chang SK, Kim KK, Han H, et al. Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Academic radiology. 2008;15(7):853–8. pmid:18572120
View Article
PubMed/NCBI
Google Scholar

[121] View Article

[122] PubMed/NCBI

[123] Google Scholar

[ref32] 32. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O. Deep learning with convolutional neural network in radiology. Japanese journal of radiology. 2018;36(4):257–72. pmid:29498017
View Article
PubMed/NCBI
Google Scholar

[125] View Article

[126] PubMed/NCBI

[127] Google Scholar

[ref33] 33. Song W, Li S, Liu J, Qin H, Zhang B, Zhang S, et al. Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition. IEEE journal of biomedical and health informatics. 2019;23(3):1215–24. pmid:29994412
View Article
PubMed/NCBI
Google Scholar

[129] View Article

[130] PubMed/NCBI

[131] Google Scholar

[ref34] 34. Ma J, Wu F, Jiang T, Zhao Q, Kong D. Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. International journal of computer assisted radiology and surgery. 2017;12(11):1895–910. pmid:28762196
View Article
PubMed/NCBI
Google Scholar

[133] View Article

[134] PubMed/NCBI

[135] Google Scholar

[ref35] 35. Li H, Weng J, Shi Y, Gu W, Mao Y, Wang Y, et al. An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images. Sci Rep. 2018;8(1):6600. pmid:29700427
View Article
PubMed/NCBI
Google Scholar

[137] View Article

[138] PubMed/NCBI

[139] Google Scholar

[ref36] 36. Ma J, Wu F, Jiang T, Zhu J, Kong D. Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Medical physics. 2017;44(5):1678–91. pmid:28186630
View Article
PubMed/NCBI
Google Scholar

[141] View Article

[142] PubMed/NCBI

[143] Google Scholar

[ref37] 37. Tsou P, Wu CJ. Mapping Driver Mutations to Histopathological Subtypes in Papillary Thyroid Carcinoma: Applying a Deep Convolutional Neural Network. Journal of clinical medicine. 2019;8(10). pmid:31614962
View Article
PubMed/NCBI
Google Scholar

[145] View Article

[146] PubMed/NCBI

[147] Google Scholar

[ref38] 38. Kim KH, Kang DW, Kim SH, Seong IO, Kang DY. Mutations of the BRAF gene in papillary thyroid carcinoma in a Korean population. Yonsei medical journal. 2004;45(5):818–21. pmid:15515191
View Article
PubMed/NCBI
Google Scholar

[149] View Article

[150] PubMed/NCBI

[151] Google Scholar

Figures

Abstract

Purpose

Methods

Results

Conclusion

Introduction

Materials & methods

Patients

US examinations

Image acquisition and CNN evaluation

BRAFV600E mutation analysis

Statistical analysis

Results

Discussion

Supporting information

S1 File. Computer-aided diagnosis (CAD) program using the convolutional neural network (CNN).

References

BRAF^V600E mutation analysis