https://orcid.org/0000-0003-2181-3505
Figures
Abstract
Objectives
This comparative study aimed to evaluate intraoral digital photography (IODP) as assessment-tool for DMFT and number of implants (IMPL) compared to clinical diagnosis (CLIN) in an elderly population with high restorative status. Secondary research questions were whether an additional evaluation of panoramic radiographs (PAN-X) or raters’ clinical experience influence the agreement.
Methods
Fifty patients (70.98±7.60 years) were enrolled for standardized CLIN and IODP. The clinical reference examiner and ten blinded raters evaluated the photographs without and with a PAN-X regarding DMFT and IMPL. CLIN were used as reference standard and differences to IODP and IODP-PAN-X findings were analysed descriptively. To assess intra-rater agreement, pairwise Gwet’s AC1s of the three diagnostic methods CLIN, IODP and IODP+PAN-X were calculated.
Results
Compared to a DMFT of 22.10±3.75 (CLIN), blinded raters evaluated a DMFT of 21.54±3.40 (IODP) and 22.12±3.45 (IODP+PAN-X). Mean values for “Decayed” were 0.18±0.52 (CLIN), 0.45±0.46 (IODP) and 0.48±0.47 (IODP-PAN-X), while 11.02±5.97 (CLIN), 10.66±5.78 (IODP) and 10.93±5.91 (IODP+PAN-X) were determined for “Missing” and 10.90±5.61 (CLIN), 10.43±4.85 (IODP) and 10.71±5.11 (IODP+PAN-X) for “Filled”. IMPL were 0.78±2.04 (CLIN), 0.58±1.43 (IODP), 0.78±2.04 (IODP+PAN-X). Gwet’s AC1 using the mode of the blinded raters’ assessment of "Decayed", "Missing" and IMPL compared to CLIN ranged from 0.81 to 0.89 (IODP) and 0.87 to 1.00 (IODP+PAN-X), while for "Filled" and DMFT they were 0.29 and 0.36 (IODP) as well as 0.33 and 0.36 (IODP+PAN-X), respectively. Clinical experience did not influence the agreement.
Conclusions
Assessment of “Decayed”, “Missing” and IMPL by IODP showed almost perfect agreement, whereas of “Filled” and DMFT revealed fair to moderate agreement with clinical findings. Additional PAN-X-evaluation increased agreement compared to IODP-diagnostics alone. IODP for the assessment of DMFT and IMPL might be a suitable method in large-scale epidemiological studies, considering high agreement in total values and miscellaneous agreement at patient-level.
Citation: Ciardo A, Sonnenschein SK, Simon MM, Ruetters M, Spindler M, Ziegler P, et al. (2022) Remote assessment of DMFT and number of implants with intraoral digital photography in an elderly patient population – a comparative study. PLoS ONE 17(5): e0268360. https://doi.org/10.1371/journal.pone.0268360
Editor: Kelvin I. Afrashtehfar, University of Bern: Universitat Bern, SWITZERLAND
Received: December 16, 2021; Accepted: April 27, 2022; Published: May 17, 2022
Copyright: © 2022 Ciardo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Caries in permanent teeth remains the most prevalent condition worldwide and has a serious health and economic burden on individuals and society [1, 2]. In the prevention of caries, an early detection of lesions or caries-risk patients is of great importance [3]. In order to investigate the prevalence of untreated or treated caries in clinical or epidemiological studies, the DMFT index proposed by the WHO has been applied for over 80 years and is still frequently used. The index indicates the number of decayed, missing and filled teeth [4, 5]. Traditionally, this index is carried out as a direct clinical assessment with the advantages of visual-tactile inspection of the tooth and restoration surfaces. However, this approach has some limitations. In large-scale epidemiological studies, the inability to blind examiners or the subjective impact of different examiners can represent a risk of bias. Also, a high level of personnel on site and therefore financial effort must be expected. In the context of increasing multi-center, often multi-national studies, these factors are of growing importance. Data aquired by digital imaging could overcome some of these obstacles [6].
Consequently, the prerequisite for using such data, especially in epidemiological studies, is high standardization and reliability of the collected data compared to clinical data.
Digital methods and imaging are already an integral part of routine patient care and research in dentistry and will gain further importance. Digital imaging for on-site diagnostics and treatment planning as well as for remote data acquisition in large-scale epidemiological studies, will become more relevant as artificial intelligence (AI) advances [7–11]. In addition, dental imaging will increasingly be used in teledentistry [12]. With the purpose of overcoming limited access to dental care due to global rural disadvantages and patients’ immobility as a result of illness or old age, technical approaches are emerging: the assessment of medical or dental history via smartphone is already used to carry out preventive measures, dental diagnostics or monitoring, as well as to identify the specific need for dental treatment [12–17]. However, the validation of such approaches is complicated and further concerns, e.g. about data security, remain an additional hurdle [11].
In dentistry, intraoral digital photography is used for diagnostics, documentation, patient communication and education as well as research. Caries diagnostics by dental photographs showed high validity compared to clinical diagnostics as the reference standard [18–20]. Also tooth decay, dental trauma, tooth wear, plaque scores, esthetic scores around implants and orthodontic procedures have been observed via intraoral digital photography [21–28]. Though, many of these comparative studies have been performed in a healthy or young patient population, often in children and adolescents [18, 19, 21–23, 25, 28].
Therefore, the primary objective of this study was to evaluate to what extent an assessment of the DMFT index, i.e. the recording of “decayed”, “missing” and “filled” teeth and the number of implants by means of digital dental photographs agrees with clinical diagnosis in an elderly patient population with high restorative status. Secondary research questions were to what extent the additional evaluation of panoramic radiographs provide a further benefit and whether the clinical experience of the investigators influence the agreement of the assessment methods.
Materials and methods
This comparative study was approved by the ethics committee of the Medical Faculty of Heidelberg (# S-630/2019) and is in accordance with the ethical standards of the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The trial was registered at the US National Institute of Health (ClinicalTrials.gov, # NCT04192188). This study is being reported using the STARD statement for diagnostic accuracy studies and the STROBE statement for cross-sectional studies [29, 30].
Study participants
The study participants represent a subpopulation of a larger case-control study on the effects of antiresorptive medication in supportive periodontal therapy ("SPTunderART"). In the original study, 100 patients from the Section of Periodontology of the Department of Conservative Dentistry at Heidelberg University Hospital were included. For the present study, the available data of the first 50 participants were analyzed. Written informed consent was obtained from all individual participants included in the study.
Reference standard: Clinical examination
The clinical examination of the patients for the present study was performed by one clinical examiner (AC) with experience in oral diagnostics in the context of epidemiological studies between December 2019 and March 2020. The examination results served as a reference standard.
First, a professional tooth cleaning was performed on all participants. The subsequent intraoral examination for the recording of the number of teeth, restorations and teeth affected by caries was performed by tactile inspection of the teeth with a probe (EXS3A6, Hu-Friedy Mfg. Co., LLC., Frankfurt am Main, Germany), a mouth mirror (DA036R, Aesculap, B. Braun Melsungen AG, Melsungen, Germany) and air blowing. Intraoral illumination was provided by the operating light of the dental chair (Teneo/LED View Plus, Dentsply Sirona Inc., York, USA). A panoramic radiograph was screened and if the most recent one was older than five years, an updated radiograph was acquired in a standardized manner. Patients were positioned in the Frankfurt horizontal plane, a standard bite block was used and the “P1 program” (radiation time: 13,930 ms, tube current: 16 mA, tube voltage: 69 kV) was followed as specified by the manufacturer (Orthophos SL 3D, Dentsply Sirona Inc., York, USA). The findings were documented manually on a sheet and the DMFT index was recorded digitally afterwards.
Intraoral digital photography
A complete standardized photographic status comprised of five single photographs was taken of all 50 study participants by the clinical examiner with the help of additional assistance. All participants were positioned on the dental chair in such a way that a perpendicular image of the dentition was possible. All photographs were taken in the same order for every participant. For the first photograph, cheek retractors (Mirahold®, Hager & Werken, Duisburg, Germany) were inserted into the corners of the mouth to allow a frontal photograph to be taken with the dentition in habitual occlusion. The second and third images comprised the buccal representation of the lateral teeth. For this purpose, the cheek retractor was placed into the opposite corner of the mouth and a preheated mouth mirror (“T1”, dps digital-photo-systeme, Enzklösterle, Germany) was inserted into the buccal side to be imaged. The mirror was angulated in a way that the occlusal plane in habitual occlusion was positioned in the center of the image and the lateral teeth could be captured as perpendicular as possible. The fourth image was an occlusal view of the upper jaw. For this purpose, the upper lip was held off by two dental mouth mirrors and a preheated mirror (“XXLH”, dps digital-photo-systeme, Enzklösterle, Germany) was positioned in a way that the occlusal surfaces of all maxillary teeth could be imaged. Analogously, a photograph of the lower jaw was taken as the fifth image. For this purpose, the tongue was moved towards the throat with the mirror. For all intraoral images, the teeth, the gingiva, the oral mucosa of the vestibule and, if necessary, the base of the mouth were cleared of saliva with suction and air blowing. If a removable prosthesis was present, it was additionally photographed extraorally. Fig 1 shows an exemplary intraoral digital photography status and the corresponding panoramic radiograph.
Exemplary intraoral digital photography status (A) and corresponding panoramic radiograph (B).
The photographs were taken with a digital single-lens reflex camera (Nikon D90, Nikon Corporation, Minato, Japan) with an 18 to 150 mm lens (AF-S DX Nikkor 18–105 mm f/3.5–5.6G ED VR, Nikon Corporation, Minato, Japan) and a ring flash (EM-140 DG, Sigma Corporation, Kawasaki, Japan). A professional photographer specialized in dental photography was consulted for setting the camera for intraoral shots. Thus, the camera was set with a focal length of 105 mm and an aperture of F/20 to F/22 and an exposure time of 1/125 seconds. The resolution of the images was 4,288 x 2,848 with 300 dpi. All images were pseudonymized and stored in an electronic database. Brightness, color or contrast of the images were not processed afterwards. However, the images of the buccal aspects and of the occlusal surfaces taken indirectly were mirrored to reflect the real positional relationship (Microsoft Photos 2020, Microsoft Corporation, Redmond, USA).
Index test: Evaluation of digital dental photographs without and with panoramic radiographs
The pseudonymized photographs and panoramic radiographs were provided digitally to the clinical reference examiner and ten additional blinded raters for evaluation (index test). The blinded raters were seven dentists from the Department of Conservative Dentistry and three dental students from Heidelberg University. It was made sure to include assessors with different amount of clinical experience in oral diagnostics (Table 1).
All examiners were trained in advance regarding the assessment and DMFT index reporting as proposed by the WHO using a separate case example. For the assessment of “Decayed”, all active carious lesions requiring treatment, e.g. primary or secondary caries, had to be considered. All the tooth surfaces had to be screened for lesions, including pit and fissures and smooth surfaces. It was noted that wisdom teeth were also included for this evalution. Assessments took place over a period of one week and without consultation between the blinded raters. In a first step, only the photographs were analysed (IODP). In a second step, the photographs were evaluated with additional consideration of the panoramic radiographs (IODP+PAN-X). The evaluation of the photographs and panoramic radiographs could be performed on the investigator’s own notebook or computer. The individual computer monitors and image viewer software used by the investigators are listed in S1 Table. For closer examination, individual processing of the photographs and the radiographs in terms of brightness and contrast as well as zooming in could be carried out independently. The documentation of the DMFT index and the number of implants was done on an evaluation sheet for IODP and IODP+PAN-X separately.
Statistical analysis
Due to the exploratory nature of this pilot study on remote dental assessment in an elderly patient population, a sample size calculation has not been performed. In order to investigate the influence of the raters’ age on the agreement with clinical findings, ten blinded raters with differently amount of clinical experience have been chosen for remote assessment.
Patients’ demographic and clinical examination data were expressed descriptively. Continuous variables were expressed as mean ± standard deviation and categorical variables as absolute and relative frequencies.
In order to investigate the extent of agreement of the DMFT index between IODP with clinical diagnosis (primary endpoint), the clinical examination results were used as reference standard and hence, differences between the clinical diagnosis and the IODP and IODP+PAN-X assessment results were calculated and descriptively analysed using mean ± standard deviation and median (Q1-Q3).
Furthermore, to assess intra-rater agreement, pairwise Gwet’s AC1s of the three diagnostic methods “Clinical”, “IODP” and “IODP+PAN-X” were calculated separately for all examiners. In addition, overall pairwise Gwet’s AC1s were calculated using the mode of the ten blinded raters. Gwet’s AC1s were calculated because the widely used KAPPA statistics for testing the extent of agreement between raters yield unexpected low values in high agreement cases, which is present here [31, 32]. Gwet’s AC1 can be interpreted as follows: 0–0.2: slight agreement; 0.2–0.4: fair agreement; 0.4–0.6: moderate agreement; 0.6–0.8: substantial agreement; 0.8–1.0: almost perfect agreement [32, 33].
The secondary objective, whether the clinical experience of the raters influence the agreement of the assessment methods, was investigated using linear mixed effect models of the outcome “DMFT”. As dependent variable the difference between the clinical assessment and IODP or IODP+PAN-X of “DMFT” was calculated, e.g. a difference of zero indicates that the assessment methods led to the same DMFT values. As fixed effect (depend variable) the clinical experience of the raters (experienced/novices/students) was added to the model. Furthermore, random intercept effects for patient and examiner were added to the model to account for patient and examiner heterogeneity.
Statistical analyses were conducted using the statistic software R (version 4.0.2, R Core Team, Auckland, New Zealand) using the packages “lme4” and “lmerTest” for linear mixed effect models, “irrCAC” for calculation of Gwet’s AC1 and “ggplot” for data illustrations. Analyses were carried out at the Institute for Medical Biometry (IMBI) at Heidelberg University Hospital.
Results
Patients’ demographic and clinical examination data
The 50 study participants had a mean age of 70.98±7.60 years. Forty of them were female (80%). Clinical examination revealed a DMFT index of mean 22.10±3.75, “Decayed” of 0.18±0.52, “Missing” of 11.02±5.97, and Filled of “10.90±5.61”. The mean number of implants was 0.78±2.04.
DMFT and implant assessment via intraoral digital photography
The IODP diagnosis by the clinical reference examiner and the ten blinded raters in relation to the DMFT index, “decayed”, “missing” and “filled” teeth as well as the number of implants are shown in Table 2 for each investigator and in the distinction between IODP alone and with PAN-X evaluation. The blinded raters’ value for DMFT was 21.54±3.40 by IODP and 22.12±3.45 by IODP+PAN-X compared to clinical diagnosis of 22.10±3.75. For “Decayed”, clinical examination revealed a value of 0.18±0.52 while the blinded raters assessed 0.45±0.46 (IODP) and 0.48±0.47 (IODP-PAN-X). Respectively, the means for “Missing” were 11.02±5.97 (clinical) compared to 10.66±5.78 (IODP) and 10.93±5.91 (IODP+PAN-X) whereas for “Filled”, they were 10.90±5.61 (clinical) versus 10.43±4.85 (IODP) and 10.71±5.11 (IODP+PAN-X). The assessment of number of implants revealed a mean of 0.78±2.04 by clinical examination and 0.58±1.43 (IODP) and 0.78±2.04 (IODP+PAN-X) by the blinded raters.
Agreement between clinical examination and IODP/IODP+PAN-X diagnostics on the individual patient level is shown with pairwise Gwet’s AC1 for the clinical reference examiner and for each of the blinded raters (Table 3).
When measuring the agreement of the blinded raters’ assessment of "Decayed", "Missing" and "Implants" with the clinical reference findings, overall Gwet’s AC1 using the mode of the blinded raters ranged from 0.81 to 0.89 (IODP) and 0.87 to 1.00 (IODP+PAN-X), while for "Filled" and "DMFT" they were 0.29 and 0.36 (IODP) as well as 0.33 and 0.36 (IODP+PAN-X), respectively. For comparative evaluation of the reference examiner’s assessment, Gwet’s AC1s for "Decayed", "Missing" and "Implants" ranged from 0.83 to 0.98 (IODP) and 0.83 to 1.00 (IODP+PAN-X). For "Filled" and "DMFT", Gwet’s AC1s ranged from 0.36 to 0.39 (IODP) and 0.54 to 0.58 (IODP+PAN-X), respectively.
Fig 2 highlights the benefit of the additional evaluation of the panoramic radiographs to IODP for almost every comparison and investigator.
(A) Decayed, (B) Missing, (C) Filled, (D) DMFT, (E) number of implants, Abbreviations: IODP = intraoral digital photography, PAN-X = panoramic radiograph, RE = reference examiner, BR = blinded raters, Exp = Experienced Dentist, Nov = Novice Dentist, Stud = Dental Student.
Investigator’s clinical experience
When comparing the differences between clinical results as reference standard and the IODP/IODP+PAN-X asssessment by the blinded raters, the linear mixed effect models for “DMFT” show no statistically significant differences between experienced dentists, novice dentists and dental students (S2 Table).
Discussion
In this elderly study population with advanced restorative and periodontal status, assessment of the DMFT index as well as the number of implants by means of digital dental photographs, showed miscellaneous agreement compared with clinical diagnosis. Comparisons of the parameters "Decayed", "Missing" and "Implants" show almost perfect agreement by IODP without and with the aid of the panoramic radiographs—this applies to the distinction between the clinical reference examiner alone and between the reference examiner and the blinded raters. For "Filled" and accordingly "DMFT", there is moderate agreement between the clinical examination and the IODP/IODP+PAN-X diagnosis by the reference examiner supported by the radiographic diagnostics, whereas without radiographic evaluation there is fair agreement. With respect to the IODP/IODP+PAN-X evaluation by the blinded raters, only fair agreement on these two parameters is found, regardless of whether the photodiagnostic was performed without or with radiographic evaluation.
In the overall patient cohort, mean and median values in all the examined diagnostic parameters were very similar between clinical diagnosis and the assessment of the blinded raters, while the agreement between IODP/IODP+PAN-X-diagnosis on an individual patient level was lower for “Filled” and “DMFT” compared to “Decayed”, “Missing” and “Implants”. The additional assessment of the panoramic radiographs led to higher agreement in most parameters and in almost all investigators.
While the clinical experience of the investigators showed no influence on the agreement, slightly higher agreement of the clinical examiner compared to the blinded raters was found in each parameter. This may be due to several reasons. Given the high number of photographs and patients, the effect of unblinding is unlikely. It is more plausible that the individual decision for the need for treatment may also play a role, for example in caries diagnostics. It has been shown that regardless of the clinical experience, the subjective assessment can be more or less conservative [34]. This also illustrates the assumption that–even with strict criteria for defining caries or insufficient restorations–for multi-center epidemiological studies, a bias in the results can be reduced if one designated examiner alone makes the distinct decisions. For the rather objective parameters “Missing”, “Filled” and “Implants”, this should play a minor role.
The lower agreement for "Filled" and "DMFT" despite similar mean values in the descriptive analysis can be explained by the greater variance, as can be seen from the higher standard deviations and the first and third quartiles. One reason for the somewhat higher scatter of the findings for "Filled" could be the lack of tactile inspection of the tooth surfaces for restoration margins and the reduced visualization on intraoral photographs of proximal and lingual areas. The use of bitewing instead of panoramic radiographs could result in a more precise assessment of “filled” teeth. Also, the additional analysis of lateral photographs taken with an open bite could help detecting restorations or carious lesions in the coronal or incisal aspects of the teeth. The difficulty in detecting esthetic restorations should also be highlighted. Signori et al. investigated photodiagnostics with regard to the assessment of repair and replacement needs or failure of restorations compared to clinical diagnostics and showed higher agreement in the posterior region compared to the anterior region with more esthetic restorations [35].
Other studies, however, emphasize that the assessment of restorations or tooth defects via intraoral photographs could be more accurate compared to clinical examination, taking advantage of digital post-processing and magnification of the photographs [28, 36].
Compared to other studies that investigated oral diagnostics by means of dental photographs, the subjects of the present study were older and had a higher restorative status [18, 19, 21–23, 25, 28]. This increases the external validity of the study and the generalizability of the findings to other age groups. For example, the current Fifth German Oral Health Study (DMS V) showed a DMFT of 17.7, "Decayed" of 0.5, "Missing" of 11.1 and "Filled" of 6.1 in seniors aged 65–74 years, respectively [37]. Our cohort showed similar values for "Missing" (11.02±5.97) and for "Decayed" (0.18±0.52). However, "Filled" showed higher values of 10.90±5.61 and the overall DMFT index was 22.10±3.75. In contrast to DMS V, wisdom teeth were included in the present study. The mean number of implants in this cohort was 0.78±2.04, whereas in the DMS V 0.22 implants were determined in the corresponding age group [37]. It should be noted that the patients in the present study were recruited in a university department specialized in conservative dentistry.
Strengths and limitations
In addition to the elderly patient population, another strength of this study is the standardized clinical examination and photographs taken by one experienced investigator. Further, ten blinded examiners with varying levels of clinical experience participated in data acquisition. Thus, a comparison between blinded raters and the clinical reference examiner as well as of the reference examiner alone could be investigated. The study design also allows to draw conclusions on the diagnostic performance of intraoral digital photography alone and in combination with the evaluation of panoramic radiographs.
The blinded examiners were able to rate the digital images on their own monitors. The photographs could be adjusted in brightness and contrast or zoomed in according to individual needs. On the one hand, this represents a limitation of internal validity; on the other hand, it increases the generalizability of the findings for large-scale epidemiological study projects in which different terminal devices are also used for data collection. An in vitro study on caries diagnosis using intraoral photographs also demonstrated no significant differences in the results when distinguishing between standardized and non-standardized viewing conditions [38].
Since this study cohort demonstrates a low number of teeth with caries or restorations in need of renewal, a valid statement on the detection of “Decayed” cannot be provided.
Conclusions
For large-scale epidemiological studies, this method of diagnostics by intraoral digital photography in an elderly patient population with advanced restorative status might be suitable, keeping in mind some limitations of the present study. Further studies including patients with a higher incidence of caries or restorations in need of replacement would be necessary. A further investigation of different additional radiological evaluation tools as bitewing radiographs would also be helpful in order to increase the accuracy of “filled” teeth.
Supporting information
S1 Table. Computer monitor and image viewer software used for image assessment.
https://doi.org/10.1371/journal.pone.0268360.s001
(PDF)
S2 Table. Linear mixed effect model for “DMFT” (n = 500).
https://doi.org/10.1371/journal.pone.0268360.s002
(PDF)
S2 Checklist. STROBE statement—Checklist of items that should be included in reports of cross-sectional studies.
https://doi.org/10.1371/journal.pone.0268360.s004
(DOC)
Acknowledgments
The authors thank Kirsten Stoik, professional photographer at Heidelberg University Hospital for her expertise and support in intraoral digital photography.
References
- 1. Kassebaum NJ, Bernabé E, Dahiya M, Bhandari B, Murray CJ, Marcenes W. Global burden of untreated caries: a systematic review and metaregression. Journal of dental research. 2015;94(5):650–8. Epub 2015/03/06. pmid:25740856.
- 2. Peres MA, Macpherson LMD, Weyant RJ, Daly B, Venturelli R, Mathur MR, et al. Oral diseases: a global public health challenge. Lancet (London, England). 2019;394(10194):249–60. Epub 2019/07/23. pmid:31327369.
- 3. Machiulskiene V, Campus G, Carvalho JC, Dige I, Ekstrand KR, Jablonski-Momeni A, et al. Terminology of Dental Caries and Dental Caries Management: Consensus Report of a Workshop Organized by ORCA and Cariology Research Group of IADR. Caries Research. 2020;54(1):7–14. pmid:31590168
- 4. Pitts NB, Fyffe HE. The effect of varying diagnostic thresholds upon clinical caries data for a low prevalence group. Journal of dental research. 1988;67(3):592–6. Epub 1988/03/01. pmid:3049719.
- 5. Klein H, Palmer CE. Studies on Dental Caries: V. Familial Resemblance in the Caries Experience of Siblings. Public Health Reports (1896–1970). 1938;53(31):1353–64.
- 6. Hogan R, Goodwin M, Boothman N, Iafolla T, Pretty IA. Further opportunities for digital imaging in dental epidemiology. Journal of dentistry. 2018;74 Suppl 1:S2–s9. Epub 2018/06/23. pmid:29929584.
- 7. Askar H, Krois J, Rohrer C, Mertens S, Elhennawy K, Ottolenghi L, et al. Detecting white spot lesions on dental photography using deep learning: A pilot study. Journal of dentistry. 2021;107:103615. Epub 2021/02/23. pmid:33617941.
- 8. Schwendicke F, Golla T, Dreher M, Krois J. Convolutional neural networks for dental image diagnostics: A scoping review. Journal of dentistry. 2019;91:103226. Epub 2019/11/11. pmid:31704386.
- 9. Schwendicke F, Krois J. Data Dentistry: How Data Are Changing Clinical Care and Research. Journal of dental research. 2021:220345211020265. Epub 2021/07/10. pmid:34238040.
- 10. Schwendicke F, Samek W, Krois J. Artificial Intelligence in Dentistry: Chances and Challenges. Journal of dental research. 2020;99(7):769–74. Epub 2020/04/22. pmid:32315260; PubMed Central PMCID: PMC7309354.
- 11. Mörch CM, Atsu S, Cai W, Li X, Madathil SA, Liu X, et al. Artificial Intelligence and Ethics in Dentistry: A Scoping Review. Journal of dental research. 2021:220345211013808. Epub 2021/06/02. pmid:34060359.
- 12. Fernández CE, Maturana CA, Coloma SI, Carrasco-Labra A, Giacaman RA. Teledentistry and mHealth for Promotion and Prevention of Oral Health: A Systematic Review and Meta-analysis. Journal of dental research. 2021;100(9):914–27. Epub 2021/03/27. pmid:33769123.
- 13. Kohara EK, Abdala CG, Novaes TF, Braga MM, Haddad AE, Mendes FM. Is it feasible to use smartphone images to perform telediagnosis of different stages of occlusal caries lesions? PLoS One. 2018;13(9):e0202116. Epub 2018/09/07. pmid:30188900; PubMed Central PMCID: PMC6126822.
- 14. AlShaya MS, Assery MK, Pani SC. Reliability of mobile phone teledentistry in dental diagnosis and treatment planning in mixed dentition. J Telemed Telecare. 2020;26(1–2):45–52. Epub 2018/08/24. pmid:30134778.
- 15. Estai M, Bunt S, Kanagasingam Y, Kruger E, Tennant M. Diagnostic accuracy of teledentistry in the detection of dental caries: a systematic review. J Evid Based Dent Pract. 2016;16(3):161–72. Epub 2016/11/20. pmid:27855831.
- 16. Inês Meurer M, Caffery LJ, Bradford NK, Smith AC. Accuracy of dental images for the diagnosis of dental caries and enamel defects in children and adolescents: A systematic review. J Telemed Telecare. 2015;21(8):449–58. Epub 2015/09/18. pmid:26377122.
- 17. Irving M, Stewart R, Spallek H, Blinkhorn A. Using teledentistry in clinical practice as an enabler to improve access to clinical care: A qualitative systematic review. J Telemed Telecare. 2018;24(3):129–46. Epub 2017/01/17. pmid:28092220.
- 18. Boye U, Willasey A, Walsh T, Tickle M, Pretty IA. Comparison of an intra-oral photographic caries assessment with an established visual caries assessment method for use in dental epidemiological studies of children. Community Dent Oral Epidemiol. 2013;41(6):526–33. Epub 2013/04/10. pmid:23566100.
- 19. Boye U, Pretty IA, Tickle M, Walsh T. Comparison of caries detection methods using varying numbers of intra-oral digital photographs with visual examination for epidemiology in children. BMC Oral Health. 2013;13:6. Epub 2013/01/15. pmid:23312001; PubMed Central PMCID: PMC3549278.
- 20. Guo S, Chen Y, Mallineni SK, Huang S, Liu B, Zhang S, et al. Feasibility of oral health evaluation by intraoral digital photography: a pilot study. J Int Med Res. 2021;49(2):300060520982841. Epub 2021/02/04. pmid:33530811; PubMed Central PMCID: PMC7871079.
- 21. Wong HM, McGrath C, King NM. Diffuse opacities in 12-year-old Hong Kong children—four cross-sectional surveys. Community Dent Oral Epidemiol. 2014;42(1):61–9. Epub 2013/07/31. pmid:23889509.
- 22. Cochran JA, Ketley CE, Sanches L, Mamai-Homata E, Oila AM, Arnadóttir IB, et al. A standardized photographic method for evaluating enamel opacities including fluorosis. Community Dent Oral Epidemiol. 2004;32 Suppl 1:19–27. Epub 2004/03/16. pmid:15016113.
- 23. Soto-Rojas AE, Martínez-Mier EA, Ureña-Cirett J, Jackson RD, Stookey GK. Development of a standardisation device for photographic assessment of dental fluorosis in field studies. Oral Health Prev Dent. 2008;6(1):29–36. Epub 2008/04/11. pmid:18399305.
- 24. Wohlrab T, Flechtenmacher S, Krisam J, Saure D, Wolff D, Frese C. Diagnostic Value of the Basic Erosive Wear Examination for the Assessment of Dental Erosion on Patients, Dental Photographs, and Dental Casts. Oper Dent. 2019;44(6):E279–e88. Epub 2019/04/30. pmid:31034351.
- 25. Skalsky Jarkander M, Grindefjord M, Carlstedt K. Dental erosion, prevalence and risk factors among a group of adolescents in Stockholm County. Eur Arch Paediatr Dent. 2018;19(1):23–31. Epub 2018/01/13. pmid:29327216; PubMed Central PMCID: PMC5807473.
- 26. Jackson TH, Kirk CJ, Phillips C, Koroluk LD. Diagnostic accuracy of intraoral photographic orthodontic records. J Esthet Restor Dent. 2019;31(1):64–71. Epub 2018/11/02. pmid:30381885.
- 27. Barwacz CA, Stanford CM, Diehl UA, Qian F, Cooper LF, Feine J, et al. Electronic assessment of peri-implant mucosal esthetics around three implant-abutment configurations: a randomized clinical trial. Clinical oral implants research. 2016;27(6):707–15. Epub 2015/06/23. pmid:26096162.
- 28. Golkari A, Sabokseir A, Pakshir HR, Dean MC, Sheiham A, Watt RG. A comparison of photographic, replication and direct clinical examination methods for detecting developmental defects of enamel. BMC Oral Health. 2011;11:16. Epub 2011/04/23. pmid:21510890; PubMed Central PMCID: PMC3103494.
- 29. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Bmj. 2015;351:h5527. Epub 2015/10/30. pmid:26511519; PubMed Central PMCID: PMC4623764.
- 30. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet (London, England). 2007;370(9596):1453–7. Epub 2007/12/08. pmid:18064739.
- 31. Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol. 1990;43(6):551–8. Epub 1990/01/01. pmid:2189948.
- 32. Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61(Pt 1):29–48. Epub 2008/05/17. pmid:18482474.
- 33. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. Epub 1977/03/01. pmid:843571.
- 34. Wilson N, Lynch CD, Brunton PA, Hickel R, Meyer-Lueckel H, Gurgan S, et al. Criteria for the Replacement of Restorations: Academy of Operative Dentistry European Section. Oper Dent. 2016;41(S7):S48–s57. Epub 2016/10/01. pmid:27689930.
- 35. Signori C, Collares K, Cumerlato CBF, Correa MB, Opdam NJM, Cenci MS. Validation of assessment of intraoral digital photography for evaluation of dental restorations in clinical research. Journal of dentistry. 2018;71:54–60. Epub 2018/02/14. pmid:29438796.
- 36. Moncada G, Silva F, Angel P, Oliveira OB Jr., Fresno MC, Cisternas P, et al. Evaluation of dental restorations: a comparative study between clinical and digital photographic assessments. Oper Dent. 2014;39(2):E45–56. Epub 2013/08/14. pmid:23937403.
- 37.
Jordan AR, Micheelis W. Fünfte Deutsche Mundgesundheitsstudie-(DMS IV): Deutscher Zahnärzte Verlag DÄV Köln; 2016.
- 38. Boye U, Walsh T, Pretty IA, Tickle M. Comparison of photographic and visual assessment of occlusal caries with histology as the reference standard. BMC Oral Health. 2012;12:10. Epub 2012/05/01. pmid:22540771; PubMed Central PMCID: PMC3444379.