Ethics statement

Ethical approval was obtained from the ethic commission of the medical faculty of the University of Duisburg-Essen.

Survey design and study populations

We performed a cross-sectional, web-based questionnaire survey among a random and a convenience sample of primary care physicians in North Rhine-Westphalia, Germany’s largest federal state. The survey included general practitioners (with and without board certification) and general internists, paediatricians were excluded. Primary care physicians in North Rhine-Westphalia are organized in the two regional Statutory Health Care Administrations North Rhine and Westphalia-Lippe. We draw a 10% random sample with a total of 946 of 9533 primary care physicians: 576 of 5757 physicians in North Rhine and 378 of 3776 physicians in Westphalia-Lippe. In addition, all primary care teaching physicians of the University of Duisburg-Essen, Essen (n = 211), were invited to participate in the survey.

Survey instrument

The online questionnaire survey was constructed using LimeSurvey (www.limesurvey.org). The questionnaire consisted of 30 questions addressing immunization aspects relevant for daily primary care. The items were developed and selected based on three practice visits and a systematic comparison of four national guidelines (United Kingdom [2], United States of America [3], [4], Australia [5], [6], Germany [7]–[10]). The first part of the questionnaire addressed issues of vaccination management, the second part errors and near-errors while the third part requested social demographic data. The questionnaire was pre-tested on the research team and three external academic primary care physicians.

Errors and near-errors in vaccination management

In the second part of the questionnaire, physicians’ experiences with 15 vaccination errors were addressed. Items were based on the anonymous German critical incident reporting system “Jeder Fehler Zaehlt” (ie, “every error counts”; www.jeder-fehler-zaehlt.de). Answer options for each item were: not experienced, experienced as near-error or experienced as actual error.

Data collection

The survey was hosted on the LimeSurvey platform. Participants were invited by email or - if email address was missing - letter and provided with login information. Participants were asked to complete the anonymous questionnaire within six weeks and received a reminder after three weeks.

Statistical analysis

Analyses were conducted using SPSS Statistics, version 20. For the descriptive statistics, the frequencies of various answers were calculated. The quality of vaccination management was assessed using one quality indicator each for patient-, vaccine- and personnel and storage-related aspects. Each indicator consisted of the above mentioned three items and scoring positive on all these served as cut-off point for good quality. Also, a summarizing quality indicator called “good vaccination management” was developed through the summation of scores above the cut-off points on the four indicators; scoring positive on all 12 indicators was considered good quality. Chi-square tests were used to analyse if independent parameters (practice and physician characteristics) influenced the frequencies of scoring on the four quality indicators and the overall quality indicator for good vaccination management. A statistical significance was assigned at a level of p<0.05. Logistic regression analysis was used to determine predictors for good vaccination management. Factors associated with good vaccination management quality in bivariate analysis were included in the final model.

Study participation

A total of 247 of 1157 physicians participated in the survey (response rate: 21%). The sub-analyses showed that 127 of 211 teaching physicians and 120 of 946 physicians in the random sample participated. The questionnaire was completed by 74% of the random sample and 65% of the university sample. Formal comparison showed that completion of the questionnaire was positively correlated with a larger vaccine spectrum in the random sample (Spearman r_s = 0.191, p = 0.037), in the teaching sample no correlation was identified. Analyses included completed questionnaires only (n = 172). The two samples showed no significant differences in their structural characteristics except that the teaching practice sample showed more practices with two or more physicians (65% vs. 46%, p<0.01), more practices with more than 1500 patients (59% vs. 34%, p<0.01) and more physicians with palliative care training (29 vs. 16%, p = 0.04). For details see Table 1 and Table S1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Physician and practice characteristics of survey respondents.

https://doi.org/10.1371/journal.pone.0105119.t001

Physician and practice characteristics

The majority of the physicians was male with an average age of 51 years. 63% were GPs, 30% held a specialty degree in general internal medicine, and 5% were practitioners without degree. Almost half of the physicians (44%) were working in a solo practice. The majority (77%) had practices with more than 1000 patients quarterly. For details see Table 1 and Table S1.

Vaccination management

The indicators were met as follows: 69% met the patient-related quality indicator, 73% the vaccine-related quality indicator, 59% the personnel-related quality indicator and 41% the storage-related quality indicator. The overall quality indicator for good vaccination management was reached by 19%. Comparison of the quality indicators in the two samples showed no significant difference except that the percentage of practices fulfilling the patient consent quality indicator was 96% in the random sample and 87% among the teaching physicians (p = 0.02). For details see Table S2. Potential for improvement was documented for all storage-related aspects and single other items: only 79% used a separate refrigerator for vaccine storage, 51% kept a storage temperature log, 92% performed a regular storage control (wrapping, temperature and expiration date), 77% had a complete chart documentation (86% the charge number and dose, 84% the trade name), 66% recommended vaccinations according to current recommendations. Practice strategies to increase vaccination coverage were used often or routinely by 76% of practices. For details see Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Quality indicators for vaccination management.

https://doi.org/10.1371/journal.pone.0105119.t002

Various additional characteristics of practice vaccination management not included in the quality indicators were obtained. Of the 20 vaccines offered, the practices used on average 16.6 vaccines (8–20, SD: 2.28), 95% offered all of the five standard vaccinations, 6% offered all 15 indication vaccinations (on average 11.7 vaccines). The following pattern of responsibilities within the practice team was identified: in most practices physicians were responsible for the vaccine pre-selection (79%) and providing patient information (83%), whereas in the majority of practices medical assistants were designated as solely responsible for vaccine ordering (73%), stock control (82%) and storage control (60%). All practices (99%) reported that either the physician or the medical assistant is responsible for the storage control. In 20% of the practices vaccines were administered by the physician only, while the decision who inoculates was case-specific in 56% of the practices, depending either on the vaccine or the patient. The participation rate in vaccination-related CME within the last two years was 77% in physicians and 70% in medical assistants. A recall to all patients was offered by 41% of practices. Regular practice software for the development of patients’ lists was used by 35%, additional immunization specific software by 4%. Results on additional characteristics of practice vaccination management are presented in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Additional characteristics of vaccination management.

https://doi.org/10.1371/journal.pone.0105119.t003

Error and near-error reporting

At least one error or near-error was reported by 89% of the participants. On average, a total of 3.4 errors and near-errors had occurred: 2.3 errors and 1.2 near-errors. The five most frequently reported types of errors were: double vaccination due to missing documentation in 40%, intramuscular injection of a patient on anticoagulants therapy in 39%, vaccination despite missing indication in 36%, wrong vaccine in 19%, and the vaccination of a 14-year-old without parental approval in 16%. The three most frequently reported near-errors were: vaccination despite of acute disease in 13%, intramuscular injection of a patient on anticoagulants therapy in 13%, and expired vaccine in 12%. For details on the sum of all 15 errors and near-errors see Table 4 and Table S3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Frequencies of errors and near-errors in vaccination management^*.

https://doi.org/10.1371/journal.pone.0105119.t004

Physicians’ opinions about the survey

The survey was considered helpful by 30%, triggered a reconsideration of practice standards in 38% and caused the wish to participate in CME in 16%. Taking immediate action was reported by 10% of physicians, 69% voiced an interest in the results.

Predictors for good vaccination management quality and error reporting

One fifth (19%) of all practices showed “good vaccination management”. Chi-square tests for the four quality indicators and the overall quality indicator for good vaccination management did not identify any parameters influencing all quality indicators. Chi-square tests did not reveal any parameters influencing error reporting. The following predictors for good vaccination management quality were identified:

• Quality Indicator Storage: additional qualification in travel medicine (60% vs. 36%, p = 0.010, x² = 6.615)
• Quality Indicator Personnel: group practice (71% vs. 45%, p = 0.001, x² = 11.031), additional qualification in travel medicine (80% vs. 54%, p = 0.006, x² = 7.572), practices with more than 1500 patients quarterly (70% vs. 51%, p = 0.012, x² = 6.378)

Logistic regression did not identify any predictors for good vaccination management.

Patient-related process quality

In the medical literature, three groups of interventions were studied as strategies to increase immunization rates in children and adults: patient-oriented interventions (e.g. written reminders), provider interventions (e.g. preventive services flow sheets in patient charts) and system interventions (e.g. public immunization campaigns) [11]. A Cochrane review comparing RCTs with patient-oriented interventions showed that postcards, letters, autodialers, and person-to-patient phone calls increased immunization rates: phone reminders were most effective but also most costly, yet increased immunization rates up to thirty percent [11]. In 2000, a survey among 316 US primary care physicians showed that 23% of practices are using mail or phone reminders [12]. This is comparable to our results which showed that 31% of practices are using phone, and up to 14% written reminders, e.g. sms, e-mail, or letter. Provider interventions were more frequent than patient-oriented interventions both in the US survey and our study, yet the measures used differ. In decreasing order, US physicians used the following three strategies most frequently: preventive service flow sheets in patient charts (71%), walk-in immunization service (67%), and a policy to assess vaccination status at each visit [12]. In comparison, our survey showed that physicians apply the strategy to control risk groups (95%), provide appointment for next vaccination (90%) and use routine consultations to optimize immunization levels (90%). These differences may be due to health care system factors, physician education, and individual preferences. System differences also play a role with regard to the mode of patient consent and information: while verbal information and verbal patient consent is sufficient by German law, US law requires the provision of detailed written consent prior to each vaccination [4].

Vaccine-related process quality

Adequately trained personnel is of key importance to assure a high quality of vaccine management. This was documented in an intervention study in the Atlanta region: daily temperature monitoring of vaccine storage compartments was 2–3 times more likely if the designated coordinator had a higher level of medical education [13]. A survey of 221 US practices showed that 83% had designated a specific person as responsible for vaccine storage and handling, with a backup in 63% of practices [13]. Another US survey among 721 primary care offices differentiated between ordering, storing and application of vaccines: in the majority of practices only one person was responsible for ordering vaccines (75%), two or more staff members were responsible for storing (50%) and application of vaccines (77%) [14]. Our survey analyzed responsibilities within the practice team: in more than 70% of practices it is the sole responsibility of the physician in charge to pre-select vaccines, while ordering, storage and stock control are typically delegated to medical assistants. We consider it positive that physicians are involved in vaccine pre-selection, as these decisions may pre­vent programmatic vaccination errors described in the medical literature, such as mixed use of brands for the same vaccination or the use of different vaccines with similar names [15]–[17].

Personnel-related quality

The definition of our quality indicator addressed three typical aspects of personnel-related immunization quality: application of vaccination recommendations, personnel qualification and the physician or a designated person applies the vaccine. Considering the recommendation of catch-up vaccination regimes, studies among pediatricians showed a broad variance between physicians [18]–[20], which was confirmed in our study. Using one clinical vignette we showed discrepancies to current recommendations. These differences may be oblivious due to knowledge deficits (e.g. with regard to immunization recommendations and contraindications), or deliberate due to disagreement with immunization recommendations. Both aspects have been previously associated with the omission of vaccinations, especially in pediatricians [12], [19]. Discussed are also increasingly complex immunization schedules as barrier to immunization practices conform to recommendations [20], [19].

Storage-related quality

The definition of our quality indicator addressed three typical storage-related aspects: storage device, keeping a storage temperature log and regular storage control. In our study, 79% of practices used a separate refrigerator for vaccines, which was documented in 96% of 695 US primary care practices [14], yet only 59% of 172 Australian [21], and 9 to 22% of 135 Canadian practices [22]. All of the practices surveyed in our study had designated coordinators responsible for storage control, which is higher compared to 83% US private providers who designated a person for the joint tasks vaccine storage and handling [13]. Frequently, vaccine storage conditions are a weak point in the quality chain with the potential to cause disease outbreaks [23]–[25], to reduce the vaccine effectiveness and the tolerability of vaccines [26], [27]. A systematic literature review based on 14 studies in developed countries showed that 13.5% (6.4 to 20.7%) of refrigerators had temperatures below the freeze threshold [28]. The difficulty of maintaining refrigerators within the correct temperature range becomes clear considering that even after participating in an intervention study, 50% of the practices fail this criteria [29]. Additionally, the storage equipment used is crucial. An Australian study based on 28 general practitioners showed that the refrigerator type used is associated with maintaining correct temperatures, with purpose built vaccine refrigerators showing better results [30]. Our results are in agreement with these findings: only 92% regularly controlled their storage and only 51% of the practices kept a storage temperature log. The latter result is much lower compared to 73% US primary care practices [14], yet comparable to 53% US private physicians who do not have a log [13]. Thus, at least in our region, future interventions to improve vaccine-related quality should address the issue of a separate refrigerator, a storage temperature log and regular storage control as important content of CME and practice improvement strategies.

Errors and near-errors

In our understanding, the reported average of 3.4 errors and near-errors indicates good error awareness. In accordance with Bundy et al. who analyzed 607 error reports on an US error reporting database (“MEDMARX”) [17] we can confirm high rates for those types of errors reported most frequently in that database: double vaccination (40% vs. 25%) and wrong vaccine (19% vs. 25%). Thus, interventions should focus e.g. on improving vaccination documentation to reduce doubled vaccination (e.g. using electronic health records) and prevent look-alike/sound-alike confusion to reduce using the wrong vaccine.

Limitations, Conclusions and Perspectives

Our cross-sectional physician survey addressed a broad spectrum of issues. Several limitations need to be considered. First, it was performed in a country lacking detailed guidelines on vaccination management and the quality indicators used are based on common reasoning and standards from other countries. Second, we did not compare physicians’ answers with actual every-day vaccination management. Third, given the low response rate, a response bias and a bias towards the desired answers cannot be excluded, e.g. regarding time since last CME.

In summary, we demonstrated that about 19% of German primary care practices have good vaccination management, yet broad variance for single aspects. Interventions should focus on storage-related issues.