Measuring instrument

Three questionnaires have been developed for the OHO. Table 1 shows the questions posed per questionnaire. The first questionnaire is aimed at collecting background patient data and is completed by the dentist or a member of staff in the practice, before the patient concerned visits the practice (Table 1A). The second questionnaire concerns the patient’s oral health and the dentist completes this after the patient’s visit (Table 1B). Finally, the third questionnaire is completed by the patient and involves them providing background data on themselves and their oral health (Table 1C). In the case of younger patients, it was completed by a parent or carer. The third questionnaire, which is based on the WHO research method, can be completed before or after a visit to the dentist [12].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Questions of the questionnaires of the proof of concept study of the oral health observatory.

https://doi.org/10.1371/journal.pone.0191385.t001

These three questionnaires were developed into a mobile application (app). By creating a specific app, the changes of misuse of the data are minimized. The app can be used on tablets with an Android operating system: Samsung Galaxy Tab 4.10.1 (model number SM-T530 and Android version 5.0.2). The questionnaires were completed in the practice with or without access to the internet. Synchronization of the data with the centralized database could be done in real-time, upon internet access availability. In circumstances where there was no network connection, all data were stored until they could be synchronized when a network connection was available. Since the three questionnaires concerning an individual patient are linked to each other, a cross analysis can be made between the background characteristics of the patient, their oral care behaviour and well-being, and the clinical status reported by the dentist.

Ethics statement

Dentists who consented to take part in the study were visited and instructed how to recruit patients for the study. The purpose of the study and the nature of participation was introduced to potentially participating patients and parents of young patients at a regular dentist visit. After this introduction they were asked to participate. Those who consented participation completed a questionnaire and gave the dentist permission to fill out two questionnaires about their (child’s) background and oral health status. This study did not contain any other interventions or additional examinations. The study protocol was approved by an independent review board of the KNMT.

Set-up of the first phase of the proof of concept study

The OHO proof of concept study has been conducted in two phases. Five dental practices in the Netherlands participated in the first phase, which took place in September 2014. The Netherlands was a suitable country for this purpose since dental practices in this country are familiar with surveys conducted by the dental professional organization. As it happens, the Royal Dutch Dental Association (KNMT) has had its own research programme for 20 years which focuses on periodic data collection in dental practices about practising the dental profession [20][21]. Five dental practices selected from the participants in KNMT-surveys, were initially approached by telephone. All five practices that agreed participation and were visited for the delivery of a tablet with the OHO app installed and given oral instructions on how to use it. During the first phase of the proof of concept study, the participating practices were in regular phone or email contact with the KNMT-office about how to collect the data.

In the first phase a questionnaire in English was used as translating the questionnaire would have been a timely process. Besides, the intention was not so much focused on collecting usable data on oral health and the patients at this stage, but rather to test whether or not data collection via an app on a tablet in the dental practice would work and could be reliable. In addition, it is reported that a large majority of Dutch people say they have sufficient command of the English language [22]. In the five practices, data was collected via the app for a maximum of about 10 patients from various age groups. The collected data was sent from the tablet via the internet to a database managed by FDI. Patients remained anonymous in the database. Dentists could do this at any given moment, for example immediately after the patient consultation, at the end of each day, or after seeing all the patients.

Both dentists and patients were enthusiastic about this method of data collection. Some questions, however, still proved to be somewhat unclear and the dentists found that the app reacted slowly. The data collected in this first phase has not been used in the results as described below.

Set-up of the second phase of the proof of concept study

Where possible, the comments received in the first phase were incorporated into the questionnaires and the app. After this the questionnaires were translated into German, Spanish and Dutch for the second phase of the proof of concept study, which was conducted more or less simultaneously in Germany, Mexico and the Netherlands. In each of these three countries, 20 dentists were prepared to participate in the second phase. The second phase started in the Netherlands at the end of March 2015 and lasted until mid-2015. In the Netherlands the five dentists who had already participated in the first phase took part. Following the same procedure as used in the first phase, 15 other dentists were recruited from a group of 20 dentists, five dentist refused participation. During the selection process, attention was paid to ensuring there was a proportional representation of gender, age and region.

The 20 dentists were first sent an online questionnaire in order to collect some general data about their practice organization. Subsequently, each one of them was visited in person to deliver the tablet and explain first-hand how to use it. All the information and instructions were also made available in writing, partly in English and partly in Dutch. On the basis of this material one of the dentists made an information and instruction poster for the patients. This was subsequently offered by the KNMT to several of his colleagues and was thus used in more practices.

The dentists were asked to collect data from a total of 38 patients via the OHO app: six patients, three male and three female, from each age group of youths (ages: 2-≤5 years, 6-≤11 years and 12-≤17 years), and 20 adult patients (aged ≥18 years), 10 male and 10 female. As in the first phase of thestudy, the patients remained anonymous. When selecting the patients, dentists were asked to pay attention to securing an even spread of age and gender.

Statistical analysis

From the FDI database the data collected from the Netherlands was made available for further analysis to the KNMT via an Excel file. After conversion of this file the data was analysed with the aid of IBM SPSS Statistics, version 20. Thus, an overview of frequencies and means was first obtained from the straight counts, in which a distinction was made as regards age. Subsequently, for two aspects of oral health, viz. the presence of visually detectable caries and gingivitis, the bivariate cohesion with some other characteristics was shown (Chi-quadrate test, t-test, ANOVA).

Assessment of the proof of concept study by dentists

During the course of the proof of concept study there was regular contact, by telephone and e-mail, on the survey’s progress between the researchers and the participating dentists. During such contacts interviews were conducted on matters such as app user-friendliness, general experiences or any other issues the dentists or patients had were discussed.

Oral health

The survey results as shown in Table 3, demonstrate that dentists visually detected cavitated caries after the consultation in 16% of the patients and that in 63% they saw one or more oral restorations. Furthermore, the dentists noted in 40% of the patients that at least one tooth had been extracted. In 27% they saw sealants and in 32% there was gingivitis. Bone loss was detected in 19% of the patients and according to the dentists 10% turned out to be smokers. Furthermore, it appeared that virtually all the characteristics of oral health were age-related.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Clinical evaluation of the patients’ oral health, provided by the dentist.

https://doi.org/10.1371/journal.pone.0191385.t003

Background characteristics and oral health according to the patients themselves

As is shown in Table 4, the vast majority of the patients in the survey (92%) had also seen their dentist in the year prior to this present visit. A total 84% of the patients also stated that they brushed their teeth at least twice a day. With regards the questions concerning the influence of oral problems on their functioning, only a small minority (15% or less) stated that they faced this regularly.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Results of the OHO-questionnaire about background characteristics and oral health, provided by the patient.

https://doi.org/10.1371/journal.pone.0191385.t004

Cohesion with caries and gingivitis

Table 5 shows that in the survey patients in whom cavitated caries were visually detected by the dentist, they had on average more eating and drinking moments per day than those in whom cavitated caries had not been visually detected (6.4 versus 5.7). Furthermore, a smaller proportion of this group brushed their teeth at least twice a day (70% versus 87%), a larger proportion had eating problems due to issues in or with their mouth (19% versus 7%) and a larger proportion of patients also had problems with aching teeth, molars and/or painful gums (27% against 12%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Aspects of oral self-care and oral heath related quality of life in cohesion with the presence of cavitated caries.

https://doi.org/10.1371/journal.pone.0191385.t005

In Table 6 the same aspects of oral care, behaviour, and the results of oral problems have been linked to gingivitis whose presence had been detected by the dentist. This shows that patients with gingivitis, in comparison with patients without, brushed their teeth at least twice a day in fewer cases (78% versus 87%). Moreover, a larger proportion had eating problems resulting from issues in, or with their mouth (17% versus 5%) and a larger proportion also had aching teeth, molars and/or painful gums (24% versus 10%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Aspects of oral self-care and oral heath related quality of life in cohesion with the presence of gingivitis.

https://doi.org/10.1371/journal.pone.0191385.t006

Assessment of the proof of concept study by dentists

The dentists in this study were very positive about answering the questions through the app and appreciated the ability to do this jointly during the patient’s visit to the practice. According to them also patients were positive about the OHO instrument. Furthermore, the dentists found that the instructions for the instalment of the app were good and they also valued the oral instructions they received though the direct contact hence there was no criticism on this side. The app was, however, found to be slow and was lacking visual response to input. It was recommended to increase the app reactivity. In addition, several dentists said that parents of children aged 6–11 years thought it strange they should be asked the question whether or not their child smokes. The suggestion that this was possible was sometimes considered to be insulting.

Advantages

In this proof of concept pilot, the testing of the research methodology of the OHO was central as was the collection of reliable data. A comparison between the results derived from this study and the results from other surveys shows similarities. Thus, the outcomes with regard to brushing teeth can be compared to those of earlier research in the Netherlands [25][26]. The results with regards to dental visits show a slight over estimation, but this can be explained by the fact that this proof of concept study was conducted in dental practices, resulting in an automatic over-representation of dental visitors [27]. The internationally recognized relation between brushing teeth and caries and periodontal aberrations was also confirmed [16]. The effect of fluoride toothpaste cannot be proven in this study, since nearly all patients in the Netherlands are using such toothpaste. It is true that for the collection of clinical data an independent and calibrated oral inspection is preferred, but doing this is very time and labour consuming [28]. The set-up of the OHO makes the oral inspection for the collection of clinical data less time consuming and in comparison takes no extra effort, making it less costly. After all, the dentist will in any case perform an oral inspection with patients who come for treatment or a regular dental check-up. A precondition is, however, that the intention is to make a global assessment of oral health, on the basis of aspects that can be determined as objectively as possible and must also be visible to the naked eye.

In the Netherlands finding 20 dentists across the country, who were prepared to take part in the proof of concept study, proved to be easy. In addition, these dentists generally had little difficulty in recruiting patients for the survey. It did appear to be important that the researchers remained in close contact with the dentists in the study by means of a visit to the practice, oral instructions and regular contact. This limited any uncertainties and gave dentists the feeling that they were not ‘left to their own devices’ in case of problems. Perhaps an environmental advantage was that dentists in the Netherlands are used to research being conducted by the national professional organization [20].

Collecting data on a tablet by means of an app is done relatively quickly because the information is directly sent to a central database. In a systematic literature study of different ways of recording Patient-Reported Outcome Measures, no differences in results between questionnaires submitted in writing, or electronically (for example via tablets) were found [29]. The collection of data via an app may therefore, be considered to be reliable.

Challenges

The difficulty with international comparisons is that all kinds of national circumstances can influence a phenomenon to be measured. This will also apply when using an app. Thus, for example, it appeared during the study that parents of young children were sometimes irritated by the question whether or not their child was smoking. However, in some countries this may be a relevant question. In short, compiling a questionnaire which should be applicable for each country is not easy due to such cultural and social influences. Thus, certain questions may be subject to interpretation depending on the survey country. Take for example the travel time to the practice as an indicator of access to dental care. From the study it appeared that in the Netherlands 9% of patients had to travel for more than half an hour to the dental practice. However, the Netherlands is a densely populated country with one dentist in approximately every 1,900 inhabitants [30][31]. If a patient has to travel for more than half an hour to go to the dentist, then this is relatively far for the Netherlands and possibly a deliberate choice. In a different country travelling for half an hour may mean a relatively short distance. Therefore, one has to be alert and choose the right context when interpreting the outcomes.

Although collecting clinical data within the OHO by the dentist offers practical advantages there are also reservations. A first concern is that there may be biases present in that only those who visit the dentist are involved in the survey. This does not generate a complete picture, as the oral health of non-attendants is therefore not taken into account. In the Netherlands this distortion can be considered minor, as a vast majority of the population visits a dentist [32]. But we certainly speak of a distortion, which will be greater in countries with fewer dental visits.

A second point concerns the multi-level character of the data to be collected [33].

Within the OHO the collection of patient data takes place within dental practices. Therefore, the measurements at a patient level are not automatically independent, but dependent on the dentist and/or the practice. The average number of restorations per patient, for example, may differ as a result of variations in treatment [34].

The app developed for the OHO is undeniably attractive to participants in the survey. But then its fast and accurate functionality is indeed a precondition. Especially, since the speed of the app sometimes proved to be a problem in this survey.

Recommendations

In August 2016, FDI announced the deployment phase of the OHO [35]. From the proof of concept study in the Netherlands it was highlighted that the OHO is a practical instrument to gain general insight from dental practices into the oral health and oral care of a particular country, specifically with a view to make international comparisons between countries. To make this possible a standardized and valid questionnaire containing is required. The questionnaire used in this study did not meet this criteria. Furthermore, as far as content is concerned this instrument has its limitations since only those who visit the dentist are involved. With respect to the methodology the bias that can occur because of social differences between countries should be taken into account. Bias, since dentists ‘score’ their own care and in connection with this the multi-level character of the clinical data [36]. In addition, it is advisable to further develop the OHO app. A new version of the app should be faster, more user-friendly and more flexible. As for the latter point it is of particular importance to National Dental Associations (NDA’s) that they themselves, or in collaboration with stakeholders, have the possibility to add their own questions about specific national issues (e.g. man power planning and quality policies). Thus FDI offers its members a project that can be deployed not just internationally, but also nationally. This may have a positive influence on an NDA’s willingness to participate in the OHO.

There is no doubt that the proof of concept study in the Netherlands has proven that the OHO can be used well, and there is also great value in the information content the instrument offers. The extent however, to which this information can be compared in an acceptable way to the information collected via the OHO in other countries needs further investigation. A further analysis of the Dutch data in relation to the data collected in Germany and Mexico offers every possibility to do so. This could be the next step in the process to achieve, in time, an international comparison between countries on oral health and oral care, which is content relevant and methodologically reliable. The FDI initiative of the OHO offers a very promising perspective.