Figures
Abstract
Objectives
Previous clinical audits for chronic obstructive pulmonary disease (COPD) have provided valuable information on the clinical care delivered to patients admitted to medical wards because of COPD exacerbations. However, clinical audits of COPD in an outpatient setting are scarce and no methodological guidelines are currently available. Based on our previous experience, herein we describe a clinical audit for COPD patients in specialized outpatient clinics with the overall goal of establishing a potential methodological workflow.
Methods
A pilot clinical audit of COPD patients referred to respiratory outpatient clinics in the region of Andalusia, Spain (over 8 million inhabitants), was performed. The audit took place between October 2013 and September 2014, and 10 centers (20% of all public hospitals) were invited to participate. Cases with an established diagnosis of COPD based on risk factors, clinical symptoms, and a post-bronchodilator FEV1/FVC ratio of less than 0.70 were deemed eligible. The usefulness of formally scheduled regular follow-up visits was assessed. Two different databases (resources and clinical database) were constructed. Assessments were planned over a year divided by 4 three-month periods, with the goal of determining seasonal-related changes. Exacerbations and survival served as the main endpoints.
Citation: López-Campos JL, Abad Arranz M, Calero Acuña C, Romero Valero F, Ayerbe García R, Hidalgo Molina A, et al. (2015) Clinical Audits in Outpatient Clinics for Chronic Obstructive Pulmonary Disease: Methodological Considerations and Workflow. PLoS ONE 10(11): e0141856. https://doi.org/10.1371/journal.pone.0141856
Editor: Anna Nolan, New York University School of Medicine, UNITED STATES
Received: June 12, 2015; Accepted: October 14, 2015; Published: November 6, 2015
Copyright: © 2015 López-Campos et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: This study was financially supported by an unrestricted grant from Laboratorios Menarini, SA (Barcelona, Spain). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: This study was financially supported by an unrestricted grant from Laboratorios Menarini, SA (Barcelona, Spain). This does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.
Introduction
Despite the availability of guidelines, there is evidence that several diseases are frequently managed in a manner that is not in line with recommendations [1]. To a degree, this may reflect variations in clinical presentation, patient characteristics, and/or limited awareness of recommendations [2]. In this context, the critical analysis of medical records through an audit may prove useful in amending clinical practice [3, 4], with the ultimate goal of improving patient outcomes [5].
Clinical audits to assess the quality of care delivered and the attainment of treatment goals are paramount for the management of various chronic diseases, including chronic obstructive pulmonary disease (COPD). COPD is a major public health problem, greatly impacting morbidity and mortality [6], quality of life [7], and healthcare costs [8]. In this scenario, audits represent powerful tools for the creation of positive changes and improvements in COPD management.
Clinical audits in the field of COPD have been implemented in the United Kingdom [9], Spain [10, 11], and several other countries [2, 12, 13]. Recently, a European Clinical COPD Audit has been carried out through 13 European countries [14, 15]. The results of these in-hospital audits have provided valuable information on the clinical care provided to patients admitted to medical wards because of COPD exacerbations [15], as well as the available resources [16] and the interrelationships between health resources and clinical practice [17]. Unfortunately, clinical audits of COPD in an outpatient setting are scarce [18] and no general methodological guidelines are currently available. Insight into the success rate of implementation of guidelines and recommendations for COPD on an outpatient basis has the potential to improve long-term outcomes. Based on our previous experience in the field [11], herein we describe a clinical audit for COPD patients in specialized outpatient clinics with the overall goal of establishing a potential methodological workflow. The results for the current study can guide health information systems development and implementation in real-world settings.
Methodology
Audit implementation
We conducted a pilot clinical audit of COPD patients referred to respiratory outpatient clinics in the region of Andalusia, Spain (eight provinces with over 8 million inhabitants). The principal investigator and a project manager (both pulmonologists with expertise in auditing) were responsible of the design of the audit and periodically supervised the conduct of research (including research meetings and on-demand requests). In each center, there were two investigators: a local investigator (in charge of supervising the audit procedures, obtaining permissions, and coordinating the project) and a local data manager (in charge of recruiting cases and collecting data).
Selection of hospitals
Andalusian public hospitals are divided into four categories (Table 1). The goal of the current audit was to include 20% of all public hospitals located in the region (n = 47). Consequently, a total of 10 hospitals were invited to participate. Randomization was not performed because we did not aim to achieve a representative sampling. The selection of candidate hospitals was based on the following two criteria: 1) participation in previous in-hospital audits, and 2) expression of interest after an initial proposal of the audit. Both large university hospitals and small community-based centers with respiratory outpatient clinics were considered for inclusion. However, participation rates were higher for regional hospitals (Table 1).
Inclusion and exclusion criteria
Cases with an established diagnosis of COPD based on risk factors, clinical symptoms, and a post-bronchodilator FEV1/FVC ratio of less than 0.70 were deemed eligible [19]. Because our goal was to assess the usefulness of formally scheduled regular follow-up visits, only cases with at least 1 year of follow-up were included in the audit. Patients who underwent a first diagnostic visit or presented with an exacerbation were not eligible. Similarly, subjects with significant comorbidities that could have an impact on the COPD treatment approach were excluded at the local investigator’s discretion. Based on our previous experience, we estimated that 80 cases per center would be required for this pilot study. As of September 2013, assessments were planned over a year divided by 4 three-month periods (i.e., trimesters), with the goal of determining seasonal-related changes. Investigators were instructed to identify consecutive COPD cases at the beginning of each trimester until the desired sample size of 20 was reached. Enrollment was aimed at achieving a male to female ratio of 1:1 and including a proportionate distribution of COPD severity stages according to the degree of spirometric impairment (Table 2).
Data collection
The audit was based on two different datasets, i.e., the resources database and the clinical database. The resources database included information on resources and organization of outpatient care in each participating center. This dataset was compiled one time for each hospital before starting the collection of clinical data. All COPD patient data were included in the clinical database. The variables to be included in the clinical database were proposed by the study coordinators and then discussed via e-mail and subsequently in a face-to-face meeting (held on July 2013 in Antequera, Málaga, Spain) with all of the investigators. The methodological aspects of the audit were thoroughly discussed during the meeting. Finally, a standardized questionnaire including 55 variables for the resources database and 182 variables for the clinical database was developed. The resources variables comprised the following information: 1) type and size of the center, 2) organization and performance of the outpatient facilities, 3) complementary diagnostic tools available for routine use, and 4) availability of specific COPD treatments. The clinical variables included 1) personal data, 2) general medical history, 3) history of COPD, 4) current clinical status, 5) complementary studies, 6) complete diagnostic information, and 7) assigned treatment.
Web-based data entry system
A web-based data entry and management system was specifically developed for the audit by a quality freelance programmer. Access to the database was hierarchical, with local researchers having access to their own cases only and the study coordinators having the ability to retrieve and download all of the study data. Moreover, a computer program was made available to the coordinators for converting data directly into an SPSS system file format suitable for statistical analysis.
Study protocol
The 1-year audit took place between October 2013 and September 2014. Recruitment was performed along 4 three-month periods (October–December 2013, January–March 2014, April–June 2014, and July–September 2014). At the beginning of each period, investigators were instructed to review the list of patients attending their outpatient clinics and identify eligible COPD cases who met the inclusion criteria according to their medical history. After completion of the 1-year follow-up, the number of emergency room visits or acute hospital admissions for COPD exacerbations as well as data on the vital status were collected from the medical records. Results were not disclosed to the physicians being audited to avoid changes in their clinical practice. Moreover, they were unaware of the timeframe of the year being audited. When necessary, the principal investigator provided additional instructions on the general conduct of the audit as well as the methodology to collect the variables. Moreover, a dedicated telephone line was established to respond to questions arising from audit sites in a timely manner. The project manager was in charge of identifying missing values, outliers, and inconsistencies, requiring the participation of local investigators to correct these.
Guidelines benchmarking
The performance of the outpatient clinics was benchmarked against clinical guidelines available at the time of the audit. Throughout the study period, two guidelines–including GOLD 2013 [19] and the Spanish National Guideline for COPD (GesEPOC) [20]–were widely and uniformly used in Spain. We therefore carefully reviewed the two guidelines to extract the main statements for the purpose of benchmarking the audited performances. We also considered the 2009 Spain Health-Care Quality Standards in COPD that were active at the time of the audit [21]. The guideline statements used for benchmarking are summarized in Tables 3–5. The appropriateness of the outpatient performances in relation to such statements was categorized as follows: excellent (>80%), good (60−80%), adequate (40−59%), inadequate (20−39%), and highly inadequate (<20%).
Ethical statement
The audit was approved by the Ethics Committee of the Hospital Universitario Virgen del Rocío (code: 2013PI/201). Clinical records were anonymized in the database by assigning a numerical code through an algorithm. No personal information was registered that could be used directly or indirectly to identify an individual. The relationship between the audit code and the clinical history number was kept locally under the local investigator’s responsibility. Because of the retrospective nature of the study, the annonimization of data, and the lack of active research interventions, the need for informed consent was waived. The Ethics Committee was aware of this circumstance, clearly explained in the protocol, and approved this procedure.
Statistical analysis
All computations were performed using the Statistical Package for Social Sciences, version 20.0 (SPSS; IBM Corporation, Somers, NY, USA). Clinical variables are presented as means and standard deviations or absolute and relative frequencies, as appropriate. Quantitative variables were analysed using the unpaired Student’s t-test for independent data, preceded by the Levene test to evaluate homogeneity of variances. Categorical data were compared using the χ2 test. The variables associated with exacerbations and mortality during the 1-year follow-up were identified using hierarchical multilevel multivariate logistic regression analysis, as previously described [23]. The alpha error was set at 0.05.
Discussion
This paper describes a methodological framework for conducting a clinical audit of COPD patients in an outpatient setting. Because of the substantial burden of COPD on both patients and healthcare systems, the lack of implementation of evidence-based guidelines in clinical practice remains a major concern for professional respiratory societies [24]. In this context, clinical audits may have an important role in the drive to improve the quality of COPD care. Outpatient-based clinical audits have been previously performed in different disease conditions [25–27]. However, most audits in the field of respiratory diseases have been conducted in an in-hospital scenario [15]. Clinical audits of COPD in an outpatient setting are scarce [28, 29] and no methodological guidelines are currently available. In 2011, the ALT-BPCO (Appropriateness of Long Term treatment of COPD) project has been launched with the overall goal of auditing the applicability of COPD guidelines in an Italian region (Campania) and identifying potential obstacles to their implementation [18]. The audit was conducted in 29 respiratory units (65.8% of the total regional units), with patients being followed-up for 6 months. The authors identified significant gaps between current practice and guideline recommendations with regard to both medical practice (mean agreement: 25%) and health organization (mean agreement: 48%). Of note, significant improvements were observed after 6 months both in terms of clinical practice (60.7%) and organization (54.7%) [18].
Genuine auditing demands more comprehensive efforts than simply recording and analyzing clinical information. In this regard, the overall goal of clinical audits is to improve the quality of care through its systematic assessment against a defined standard [30], with all levels of staff being involved in reviewing activities [4, 11]. A number of methodological considerations need to be taken into account when planning an outpatient clinical audit. The main limitation of this audit lies in its lack of representativeness. Participating facilities should be representative (both in terms of type and number) of all centers located in the area under investigation. Because of the pilot nature of the current audit, we did not aim to obtain a representative sample of all Andalusian outpatient facilities for COPD. In general, large hospitals (overrepresented in the current study) may have more resources in place for delivering better clinical care [31]. However, the European COPD audit demonstrated that hospital size was positively related to clinical performances only for a minority of indicators and that differences in guideline adherence in relation to hospital size were lower than expected [16]. Nonetheless, auditing a random selection of outpatient centers for COPD will be paramount in the future because differences in guidelines adherence alone are not sufficient to explain the variance in COPD care [32]. Another limitation of the present auditing protocol is its focus on the cross-sectional evaluation of a single, specific clinical visit. Such an approach allowed assessing the performance of only one doctor during a routine clinical visit. It is also expected that former clinical acts could have had an impact on the visit under investigation (e.g., lack of spirometry if already recently performed). These limitations should be kept in mind when the final results will be available. Because of its pilot nature, the number of collected variables was extensive but not comprehensive. Notably, the audit was designed based on our previous experience. These caveats notwithstanding, the current report will be helpful to guide the selection of the study variables in future, larger audits being performed in an outpatient setting.
The potential clinical applicability of the current protocol was carefully considered when the audit was designed. Clinical audits are aimed at summarizing the clinical performance of healthcare over a specified period of time. In general, they allow health professionals to assess and adjust their performance. We are confident that the proposed protocol can be implemented in a prompt manner. To this aim, we believe it is important that investigators and/or their representative sampling are implicated in the following milestones that should be achieved in real practice: 1) the conception of the audit, 2) determination of the temporal coverage, 3) selection of variables to be collected, and 4) identification of the main clinical outcome measures.
The accuracy of an audit is generally dependent on the persons responsible for the collection of data. Although auditors external to the organization being audited could maintain a neutral, unbiased attitude, they are not normally experts in the field under investigation. Moreover, the use of external auditors leads to increased audit costs and imposes a heavier legal burden for data access rights. In turn, internal auditors spend most of their time working in the clinical structure being audited; as a result, they have a better understanding of the culture, workflows and the clinical relevance of data. However, their expertise in auditing may be limited and policies to maintain internal auditors’ objectivity should be implemented. In the present study, we relied on a selected number of internal auditors (pulmonologists with expertise in COPD) who participated in previous audits. In the initial stages of study implementation, the study protocol was thoroughly reviewed in a joint investigator meeting. In addition, adequate supervision was provided throughout the audit and results were not disclosed to the physicians being audited to avoid changes in their clinical practice. Because recruitment was performed along 4 three-month periods, auditors were unaware of the timeframe of the year under investigation. Finally, the personnel in charge of data analysis were external to the participating outpatient clinics.
The source of information to be used in clinical audits remains a matter of debate. The current audit was based on a retrospective review of prospectively included cases extracted from medical records. The two main consequences of this approach include 1) the occurrence of missing values and 2) the heterogeneity of clinical documents and sources (e.g., medical charts, nurses’ notes, administrative databases) used for data extraction [23]. One possible solution to this problem would lie in the use of electronic health records [33]. Because improvements in the quality of the underlying data lead to more reliable audit results, a careful data cleaning (aimed at keeping the number of errors and missing data as low as possible and gather maximum data for analysis) was performed in our study.
The number of candidate variables to be gathered and the choice of benchmark also pose significant methodological challenges for clinical audits [30]. Concerning the problem of variable selection, investigators should balance the inclusion of all of the useful variables with the efficiency of data analysis and the potential increases in statistical complexity and computation times. In turn, the selection of statements used for benchmarking should be informed by guidelines currently implemented in a specific geographic area (e.g., Spain for the current audit) [19, 21, 22].
Finally, an adequate data management is a critical phase for the generation of reliable conclusions from clinical audits. Quality control of data processing should be undertaken at regular intervals to minimize missing data, inconsistent data, and deviations from the protocol. Discrepancy management should include joint reviewing of discrepancies before declaring them as irresolvable. In addition, most of the data used for clinical audits are characterized by a hierarchical (multilevel) structure. Because standard multivariate regression methods tend to ignore this aspect (resulting in a loss of statistical efficiency and, in some cases, wrong conclusions), multilevel modeling is strongly recommended when hierarchical data structures are present [34]. Although the impact of a hierarchical structure is deemed to be minor when sample sizes are small (as in the current audit), multilevel modeling will be applied to identify variables significantly associated with outcomes in a follow-up study [23].
In summary, herein we have provided a methodological framework for conducting a clinical audit of COPD patients in an outpatient setting. Challenges to be considered when auditing current practice against a defined standard include a careful assessment of both patients’ characteristics and physicians’ attitude and knowledge. Hopefully, the methodology proposed in the current paper can guide health information systems development and implementation in real-world settings, with the ultimate goal of supporting a process of continuous quality improvement for COPD care.
Acknowledgments
The authors are grateful to all of the project researchers for their time and dedication to this project.
Author Contributions
Conceived and designed the experiments: JLLC. Performed the experiments: JLLC MAA CCA FRV RAG AHM RIAPG FGG FCM LCB MSP DPT AS JCB BHS AD MAV FGV JJCR. Wrote the paper: JLLC MAA CCA.
References
- 1. Asch SM, Kerr EA, Keesey J, Adams JL, Setodji CM, Malik S, et al. Who is at greatest risk for receiving poor-quality health care? The New England journal of medicine. 2006;354(11):1147–56. Epub 2006/03/17. pmid:16540615.
- 2. Agabiti N, Belleudi V, Davoli M, Forastiere F, Faustini A, Pistelli R, et al. Profiling hospital performance to monitor the quality of care: the case of COPD. The European respiratory journal: official journal of the European Society for Clinical Respiratory Physiology. 2010;35(5):1031–8. pmid:19840969.
- 3. McIntyre N. Evaluation in clinical practice: problems, precedents and principles. Journal of evaluation in clinical practice. 1995;1(1):5–13. Epub 1995/09/01. pmid:9238553.
- 4.
Flottorp SA. Using audit and feedback to health professionals to improve the quality and safety of health care. Copenhagen European Observatory on Health Systems and Policies,: WHO. regional office for Europe;; 2010. IX, 42 p. p.
- 5. Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Medical care. 2003;41(1 Suppl):I30–8. Epub 2003/01/25. pmid:12544814.
- 6. Lopez-Campos JL, Ruiz-Ramos M, Soriano JB. Mortality trends in chronic obstructive pulmonary disease in Europe, 1994–2010: a joinpoint regression analysis. The lancet Respiratory medicine. 2014;2(1):54–62. pmid:24461902.
- 7. Nonato NL, Diaz O, Nascimento OA, Dreyse J, Jardim JR, Lisboa C. Behavior of Quality of Life (SGRQ) in COPD Patients According to BODE Scores. Archivos de bronconeumologia. 2015. Epub 2015/01/28. pmid:25622995.
- 8. Kim C, Yoo KH, Rhee CK, Yoon HK, Kim YS, Lee SW, et al. Health care use and economic burden of patients with diagnosed chronic obstructive pulmonary disease in Korea. The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease. 2014;18(6):737–43. Epub 2014/06/07. pmid:24903947.
- 9. Roberts CM, Ryland I, Lowe D, Kelly Y, Bucknall CE, Pearson MG, et al. Audit of acute admissions of COPD: standards of care and management in the hospital setting. The European respiratory journal: official journal of the European Society for Clinical Respiratory Physiology. 2001;17(3):343–9. pmid:11405509.
- 10. Pozo-Rodriguez F, Alvarez CJ, Castro-Acosta A, Melero Moreno C, Capelastegui A, Esteban C, et al. [Clinical audit of patients admitted to hospital in Spain due to exacerbation of COPD (AUDIPOC study): method and organisation]. Archivos de bronconeumologia. 2010;46(7):349–57. pmid:20554103.
- 11. Lopez-Campos JL, Asensio-Cruz MI, Castro-Acosta A, Calero C, Pozo-Rodriguez F, Audipoc , et al. Results from an audit feedback strategy for chronic obstructive pulmonary disease in-hospital care: a joint analysis from the AUDIPOC and European COPD audit studies. PloS one. 2014;9(10):e110394. pmid:25333953; PubMed Central PMCID: PMC4198296.
- 12. Liaaen ED, Henriksen AH, Stenfors N. A Scandinavian audit of hospitalizations for chronic obstructive pulmonary disease. Respiratory medicine. 2010;104(9):1304–9. pmid:20363605.
- 13. Pretto JJ, McDonald VM, Wark PA, Hensley MJ. Multicentre audit of inpatient management of acute exacerbations of chronic obstructive pulmonary disease: comparison with clinical guidelines. Internal medicine journal. 2012;42(4):380–7. pmid:21395962.
- 14. Lopez-Campos JL, Hartl S, Pozo-Rodriguez F, Roberts CM, European CAt. European COPD Audit: design, organisation of work and methodology. The European respiratory journal: official journal of the European Society for Clinical Respiratory Physiology. 2013;41(2):270–6. Epub 2012/05/19. pmid:22599361.
- 15. Roberts CM, Lopez-Campos JL, Pozo-Rodriguez F, Hartl S, European CAt. European hospital adherence to GOLD recommendations for chronic obstructive pulmonary disease (COPD) exacerbation admissions. Thorax. 2013;68(12):1169–71. Epub 2013/06/05. pmid:23729193.
- 16. Lopez-Campos JL, Hartl S, Pozo-Rodriguez F, Roberts CM, European CAt. Variability of hospital resources for acute care of COPD patients: the European COPD Audit. The European respiratory journal: official journal of the European Society for Clinical Respiratory Physiology. 2014;43(3):754–62. pmid:23988775.
- 17. Pozo-Rodriguez F, Castro-Acosta A, Alvarez CJ, Lopez-Campos JL, Forte A, Lopez-Quilez A, et al. Determinants of between-hospital variations in outcomes for patients admitted with COPD exacerbations: findings from a nationwide clinical audit (AUDIPOC) in Spain. International journal of clinical practice. 2015. pmid:25651319.
- 18. Nardini S, Cicchitto G, De Benedetto F, Donner CF, Polverino M, Sanguinetti CM, et al. Audit on the appropriateness of integrated COPD management: the "ALT-BPCO" project. Multidisciplinary respiratory medicine. 2014;9(1):40. Epub 2014/08/07. pmid:25097757; PubMed Central PMCID: PMCPMC4122534.
- 19. Vestbo J, Hurd SS, Agusti AG, Jones PW, Vogelmeier C, Anzueto A, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. American journal of respiratory and critical care medicine. 2013;187(4):347–65. pmid:22878278.
- 20. Miravitlles M, Soler-Cataluna JJ, Calle M, Molina J, Almagro P, Quintano JA, et al. Spanish guideline for COPD (GesEPOC). Update 2014. Archivos de bronconeumologia. 2014;50 Suppl 1:1–16. Epub 2014/02/11. pmid:24507959.
- 21. Soler-Cataluna JJ, Calle M, Cosio BG, Marin JM, Monso E, Alfageme I. [Health-care quality standards in chronic obstructive pulmonary disease]. Archivos de bronconeumologia. 2009;45(4):196–203. Epub 2009/03/24. pmid:19304362.
- 22. Miravitlles M, Soler-Cataluna JJ, Calle M, Molina J, Almagro P, Quintano JA, et al. Spanish COPD Guidelines (GesEPOC): pharmacological treatment of stable COPD. Spanish Society of Pulmonology and Thoracic Surgery. Archivos de bronconeumologia. 2012;48(7):247–57. pmid:22561012.
- 23. Pozo-Rodriguez F, Castro-Acosta A, Alvarez CJ, Lopez-Campos JL, Forte A, Lopez-Quilez A, et al. Determinants of between-hospital variations in outcomes for patients admitted with COPD exacerbations: findings from a nationwide clinical audit (AUDIPOC) in Spain. International journal of clinical practice. 2015. Epub 2015/02/05. pmid:25651319.
- 24. Heffner JE, Mularski RA, Calverley PM. COPD performance measures: missing opportunities for improving care. Chest. 2010;137(5):1181–9. pmid:20348199.
- 25. Trevest K, Treadway H, Hawkins-van der Cingel G, Bailey C, Abdelhafiz AH. Prevalence and determinants of anemia in older people with diabetes attending an outpatient clinic: a cross-sectional audit. Clin Diabetes. 2014;32(4):158–62. Epub 2015/02/04. pmid:25646941; PubMed Central PMCID: PMCPMC4220597.
- 26. Gerber JS, Prasad PA, Fiks AG, Localio AR, Bell LM, Keren R, et al. Durability of benefits of an outpatient antimicrobial stewardship intervention after discontinuation of audit and feedback. JAMA: the journal of the American Medical Association. 2014;312(23):2569–70. Epub 2014/10/16. pmid:25317759.
- 27. Hunter MG, Post JJ. An audit of pneumococcal and hepatitis vaccination in an outpatient HIV clinic. Sex Health. 2012;9(5):495–6. Epub 2012/09/07. pmid:22950965.
- 28. Stewart IC, McHardy GJ. Audit in a chest clinic. Health Bull (Edinb). 1984;42(1):45–50. Epub 1984/01/01. pmid:6706561.
- 29. Simpson FG, Wilson J, Peake MD, Cooke NJ. Audit of work-load of physicians with a special interest in respiratory medicine. British journal of diseases of the chest. 1987;81(2):150–4. Epub 1987/04/01. pmid:3651308.
- 30. Esposito P, Dal Canton A. Clinical audit, a valuable tool to improve quality of care: General methodology and applications in nephrology. World J Nephrol. 2014;3(4):249–55. Epub 2014/11/07. pmid:25374819; PubMed Central PMCID: PMCPMC4220358.
- 31. Joynt KE, Harris Y, Orav EJ, Jha AK. Quality of care and patient outcomes in critical access rural hospitals. JAMA: the journal of the American Medical Association. 2011;306(1):45–52. pmid:21730240; PubMed Central PMCID: PMC3337777.
- 32. Hosker H, Anstey K, Lowe D, Pearson M, Roberts CM. Variability in the organisation and management of hospital care for COPD exacerbations in the UK. Respiratory medicine. 2007;101(4):754–61. pmid:17045788.
- 33. Blumenthal D, Tavenner M. The "meaningful use" regulation for electronic health records. The New England journal of medicine. 2010;363(6):501–4. pmid:20647183.
- 34. Merlo J. Multilevel analytical approaches in social epidemiology: measures of health variation compared with traditional measures of association. Journal of epidemiology and community health. 2003;57(8):550–2. pmid:12883048; PubMed Central PMCID: PMC1732554.