http://orcid.org/0000-0001-5190-3937
Figures
Abstract
Objective
Care for severely injured patients requires multidisciplinary teamwork. A decrease in the number of accident victims ultimately affects the routine and skills. PHTLS (“Pre-Hospital Trauma Life Support”) courses are established two-day courses for medical and non-medical rescue service personnel, aimed at improving the pre-hospital care of trauma patients worldwide. The study aims the examination of the quality of documentation before and after PHTLS courses as a surrogate endpoint of training effectiveness and awareness.
Methods
This was a prospective pre-post intervention trial and was part of the mixed-method longitudinal EPPTC (Effect of Paramedic Training on Pre-Hospital Trauma Care) study, evaluating subjective and objective changes among participants and real patient care, as a result of PHTLS courses. The courses provide an overview of the SAMPLE approach for interrogation of anamnestic information, which is believed to be responsible for patient safety as relevant, among others, “Allergies,” “Medication,” and “Patient History” (AMP). The focus of the course is not the documentation.
Results
In total, 320 protocols were analyzed before and after the training. The PHTLS course led to a significant increase (p < 0.001) in the “AMP” information in the documentation. The subgroups analysis of “allergies” (+47.2%), “drugs” (+38.1%), and “medical history” (+27.8%) before and after the PHTLS course showed a significant increase in the information content.
Conclusion
In summary, we showed that PHTLS training improves documentation quality, which we used as a surrogate endpoint for learning effectiveness and awareness. In this regard, we demonstrated that participants use certain parts of training in real life, thereby suggesting that the learning methods of PHTLS training are effective. These results, however, do not indicate whether patient care has changed.
Citation: Häske D, Beckers SK, Hofmann M, Lefering R, Gliwitzky B, Wölfl CC, et al. (2017) Quality of Documentation as a Surrogate Marker for Awareness and Training Effectiveness of PHTLS-Courses. Part of the Prospective Longitudinal Mixed-Methods EPPTC-Trial. PLoS ONE 12(1): e0170004. https://doi.org/10.1371/journal.pone.0170004
Editor: Takeru Abe, Yokohama City University, JAPAN
Received: July 20, 2016; Accepted: December 25, 2016; Published: January 20, 2017
Copyright: © 2017 Häske et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are included in the manuscript.
Funding: The German Association of Emergency Medical Technicians (Deutscher Berufsverband Rettungsdienst e.V. DBRD) funded the study in form of research materials (www.dbrd.de) (Funding-Account: 2013-Hae_01). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. There was no additional external funding received for this study. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Competing interests: BG, CW, DH, MM, MD are PHTLS instructors. Author BG received funding in the form of salary from Megamed Emergency Management GbR, a commercial company. This company offers different trainings for health care personnel. Megamed does not offer PHTLS-courses and has not funded or supported our trial in any way. Mr. Gliwitzky is not paid by Megamed to take part at this trial. The funders have no influence on study design, data collection and analysis, decision to publish, or preparation of the manuscript. This does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.
Introduction
Accidents have the most common cause of death since the middle ages [1]. In Germany, 4.7 deaths per 100,000 inhabitants have been reported, and in the United States (US), 11.4 per 100,000 inhabitants [2]. The peak age is between 18 to 25 years [3]. These young employed people particularly benefit from rapid rehabilitation after an accident. In Germany alone, the economic costs of traffic accidents amount to 31 billion euros [4].
Fortunately, there is a continuous decrease in the number of accidents [5,6]. The proportion of injured patients, based on the sum of all emergency patients in Germany, is approximately 10% [7]. The decreasing number of accident-related emergency calls destabilizes the routine [8]. This underlines the importance of effective training in emergency medicine.
In the 1970s, the treatment of trauma patients in the emergency room became increasingly standardized, following the introduction of Advanced Trauma Life Support (ATLS), which provided a new structure in the care for severely injured patients [9]. The associated pre-hospital care equivalent to ATLS is the Pre-Hospital Trauma Life Support (PHTLS) concept. There are also other training concepts, but PHTLS is an established concept in 66 countries around the world. Induction into PHTLS is delivered through two-day courses for medical providers, with the aim of improving the pre-hospital care of trauma patients.
In the late 1990s, Ali et al. showed that, in less developed emergency medical services (EMS) systems, PHTLS improves skills and procedures [10] and leads to a significant reduction of mortality [11,12]. However, these results cannot be transferred to current and developed EMS systems. A Scandinavian observational study showed that PHTLS training is associated with a small reduction in mortality (the mortality risk was 4.7% (36/763) without PHTLS training and 4.5% (94/2067) with PHTLS training) [13]. In a subgroup analysis of motor-vehicle traffic injuries, no reduction of mortality was observed [14]. It is questionable if the end-point “mortality” is sufficient to evaluate the effects of PHTLS in modern emergency care.
Importance
In EMS-district Wiesbaden (Germany), a previously used training concept has been revised due to the lack of learning success and employee satisfaction. At the instigation of the medical director, PHTLS courses were mandatorily established for all paramedics in EMS-district Wiesbaden [15].
Against the background of a large EMS district introducing PHTLS as standard training, with mortality as an endpoint having demonstrated little advantage of PHTLS in previous studies, the goal of this project was to analyze subjective and objective changes among participants and in real patient care, as a result of PHTLS courses.
Materials and Methods
Study design
This was a prospective pre-post intervention trial and was part of the mixed-methods longitudinal EPPTC (Effect of Paramedic Training on Pre-Hospital Trauma Care) study, evaluating the subjective and objective changes in participants and real patient care, as a result of PHTLS courses. The complete study is described in the previously published study protocol [16].
Research ethics approval
The Ethics Committee of the Medical Faculty of the Eberhard Karls University of Tuebingen and the University Hospital approved the study proposal, number 197/ 2013BO2, on May 24, 2013. The study is registered in the German Clinical Trials Register as ID, DRKS00004713. Data collection, coding, routing, and analysis were coordinated by a data protection officer at the University of Tuebingen and the University Hospital of Tuebingen. In another part of the project, participants were surveyed through a questionnaire, after providing written consent. For this part of the study, no personal information from participants or patients was collected.
Study setting and selection of participants
The study was performed in the EMS in Wiesbaden (Germany). The operational district in Wiesbaden has five commissioned EMS agencies (four charities, one private provider). The EMS in Wiesbaden had 375 paramedics and served 462,098 inhabitants during the study period of 2013/2014.
In the context of various difficulties and problems, the controlling authority committed all paramedics to attend PHTLS courses, so as to establish uniform structures and principles [15].
Intervention
PHTLS courses are well-established worldwide, comprising two-day courses for paramedics and emergency physicians, with the aim being to improve pre-hospital care for trauma patients. PHTLS courses are characterized by a large variety of teaching methods (e.g., lectures, practical case studies, and skills training), with a low instructor—participant ratio (1:4), many practical activities, and continuous interaction. In addition to various skills, the priority-based structure, ABCDE (Airway, Breathing, Circulation, Disability and Exposure), is taught and practiced in scenario-based training sessions. The teaching of PHTLS conforms with a high conformity to the key recommendations of the German “Guideline on Treatment of Patients with Severe and Multiple Injuries” [17].
“ABCDE” represents the core strategy and has the highest attention during the course. With regard to the awareness and transfer of teaching content to the real working world, a surrogate parameter, which was not so focused in the courses, seemed more appropriate. As part of the so-called “secondary assessment,” PHTLS courses, as well as other courses like ATLS and AMLS, use “SAMPLE” as a mnemonic tool [18]. The letters “AMP” are an abbreviation for allergies, medication and patient history (pre-existing illness). The analysis reviewed these items in the standardized EMS protocols from actual field operations performed by paramedics. The focus of the lesson was not set on the operation documentation. The decision to focus on “AMP” has different reasons. The first is its importance for patient safety [19]. The other reason came up during the initial data analysis, which showed that data processing of “L” and “E” was bugged. To retain data quality, just “AMP” was included.
The state of Hessen used so-called report categories (“RMZ”), in which medical indications (e.g. combustion, hyperventilation) were reported. The main groups are shown in Table 1. To indicate the patient’s condition, an emergency severity score (“RMC”) and timestamps were used (Table 2). The minimum number of points assigned as the RMC is six; the maximum number of points is 42. Deviations from the physiological condition of the patient are indicated by RMC > 6. There was no validation of correct assessment or use by the paramedics or emergency physicians.
Each digit reflects the patient's condition (at first contact) in relation to the characteristics of consciousness, respiration, circulation, injury, neurological condition, and pain again. The minimum value of "1" means "inconspicuous," in reference to each characteristic; the maximum value of "5" refers to an extremely severe degree of the relevant impairment.
Data collection and processing
The EMS protocols were required in the context of each EMS field operation and archived at the EMS agencies, in concordance with the local EMS regulations and supervisory authority. We collected the first protocols the year before and after the courses; the inclusion criteria were all operations without emergency physicians and RMC > 6. Because the highest impact is given when participants use SAMPLE for all emergency patients, we included all kind of emergencies, not only trauma patients.
Outcome measures
The primary outcome measure was a change in the quality of documentation, as a surrogate endpoint for training effectiveness and awareness. Secondary outcome measures were subgroup analyses for differences between the documentation items in the main diagnosis groups (based on the report digits) and the group of patients in severe conditions (based on the emergency severity score).
Primary data analysis
The sample size of 320 cases per group (pre; post) was based on a detectable difference of 15, with a power of 80%. Type I error was probably 0.05.
The protocols were selected for participating EMS agencies, proportional to their level of participation in the total operations protocols.
The documentation items were offset by one point for each completed item. With a total of three available documentation items (AMP), 0–3 points per protocol were possible, the total of which was obtained for the pre- and post- groups. In each group, a minimum of 0 points and a maximum of 960 points (3 × 320 protocols) were possible.
Statistical analysis
Formal pre- and post-intervention statistical evaluation was performed with Mann-Whitney’s U test for continuous variables and Fisher's Exact Test for categorical variables. A two-tailed test, with p < 0.05 was considered statistically significant. All data were analyzed using the statistical software, SPSS (Version 23.0, IBM Inc., Armonk, NY, USA). For continuous variables, data are shown as mean ± standard deviation. For categorical variables, percentages are presented.
Results
In total, 640 protocols (320 for each year) were included in the study. The two most common indications were surgical (n = 308, 48.1%) and internal (n = 197, 30.8%) emergencies (Fig 1).
The mean (with standard deviation) of the RMC (emergency severity score) was 8±2 in both years; the median prior to training was 9, and 8, after training. Fig 2 shows an overall right-skewed distribution, with 1.42.
The totals of the documentation items (AMP) were n = 364 points in the pre-course group and n = 726 points in the post-course group (p < 0.001), which made up an overall increase of 37.7% (Fig 3).
The shifts in the subgroups’ allergy, medication, and patient history were also significant (p < 0.001), as shown in Table 3.
According to the indications of operations, a subgroup analysis of surgical and internal indications, and all the other indications in one group, as well as a different point of view on operations with RMC > 9, also showed a significant increase in documentation values (Fig 4).
Discussion
In this trial, we used the quality of documentation as a surrogate endpoint for the training effectiveness of PHTLS courses in 640 operations protocols. Our score, related to the three items, allergy, medication, and patient history, showed a significant increase of 37.7% after the courses. Based on the emergency severity score or “RMC,” Fig 2 shows that most of the patients were not severely injured or sick. The subgroup analysis of the operation protocols of most severely injured or sick patients, respectively, with RMC > 9, confirmed the increase by a significant +44.8%. The individual documentation items also increased by 27.8% for patient history, 38.1% for medication, and a notable 47.2% for allergy.
Considering the subgroups’ surgical and internal indications, it seems that the largest increase among both groups was the 42.1% observed in the surgical group. Improvement was also observed in the internal group (34.7%). The baseline value of the fulfilled "AMP"-information was 41.1% in the internal group and 32.3% in the trauma group. Based on the relative proportions of the indications, it is arguable as to whether trauma training is responsible for the larger increase in the trauma group or if for example discussion about new anticoagulation [20–22] increased documentation.
RMC (emergency severity score) was 8±2 in both years, this shows that patients in our investigation were not seriously ill or injured. This maybe represents a typical patient collective for a large city. The extent to which our results can be transferred to the stressful emergency care of seriously injured patients is unclear.
Overall, the quality of documentation increased significantly, leading to the notion that PHTLS courses influence participants, with remarkable effects on real patient care and documentation.
Despite the increase in the quality of documentation, documentation still needs to be improved in 24.4% of the cases, even after the courses.
The loss of relevant medical information (e.g., during handovers) is a known problem that concerns not only documentation, but also verbal handovers [23]. This particularly affects the collection of medical history and physical examination results [24]. A video-based error analysis, as conducted by Bergrath et al., of documentation by physicians, following simulation, showed that 20% of the information was missing and 22% of the documented information was incorrect [25].
It has been shown that training or tutorials on documentation improve the quality of documentation [26–28] by 12.5% to 51%. In contrast to cited studies, in which improvement in documentation was considered an endpoint, in our trial, secondary assessment with queries on allergies, medication, and patient history, based on the learned SAMPLE schema, was only a secondary aspect, preceded by use of documentation. Nonetheless, our investigation showed an overall increase in documentation quality by 37.7%, which we think is remarkable.
It could already be demonstrated that the use of protocols, which have integrated memory aids, significantly reduce documentation errors [29]. In the present study, however, the operation protocols had memory aids, such that analyzed items had to be documented in a free-text field.
The results also indicate that, as already known [30], obvious educational interventions may generally have an impact on the system or students. This is due to not only improvement in the documentation for trauma and surgical patients, respectively, but for all other operations, too. Despite the internal medicine group’s initial documentation, which was already better, at 41%, compared to the surgical group, at 32%, an increase on 75.8% was also noted.
This result may also be explained by increased awareness regarding the three items “AMP”. Such increased awareness can be seen partly through simple educational interventions aimed at the public, in relation to stroke detection [31].
For healthcare professionals, awareness is one of the main elements of non-technical skills and a characteristic of high-performing teams [32]. The results of the present study show that awareness can be instilled via training, without much effort. Often, the amount of information in a given (emergency) situation causes an information overload, because the information is not adequately prioritized and categorized [33]. The taught SAMPLE scheme seems to bring some order, in this regard.
This trial, like many others in medical and complementary disciplines, uses surrogate endpoints, which have no value of their own [34–39]. They simply constitute indirect evidence for training effectiveness in this study. Therefore we decided not to use the ABCDE, because this was highlighted in the PHTLS trainings. Also, documentation of ABCDE is quite more difficult. In the care for less injured patients with no ABCDE problem, the need of documentation of the absence of a problem is forgotten easily. For example a patient with extremity injury without analgesia has not a worse treatment, without documentation of the uncompromised airway and breathing. The need of documentation of the SAMPLE information is more obvious—even if negative, for instance “no allergies” or “no medication”. On the other hand “SAMPLE” was just slightly taught in one lesson, but is important for every patient. So the transfer of this into real patient documentation shows the big effects of one training on awareness and the learning process.
An alternative validation could have been the use of a written exam to evaluate training effectiveness, similar to Ali et al., in 1998, who showed good learning success among PHTLS-course students [40], using a pre-/post-test. However, the aim of our study was to investigate changes in real patient care, as a result of the courses, which would not have been possible with a written exam. This led to the decision to use the documentation quality of real emergency operations as surrogate endpoints for training effectiveness.
The study participants were informed about the study and consented to scientific evaluation. Therefore, a Hawthorne effect [41] could be assumed and was critically discussed. As it had not been announced that the documentation would be evaluated, in our opinion, the Hawthorne effect is not applicable, in this instance.
Due to the high volume of EMS operations, the predetermined sample size of operation protocols was reached within a short period of time, even though the same month was chosen each year to collect protocols.
A follow-up for this part of the trial could not be initiated because subsequent training could falsify the result. Therefore, the continuous quality of documentation, variance thereof, and long-term training effectiveness cannot be verified. These effects could perhaps be improved through refresher training, which was not evaluated in the present study.
Conclusion
In summary, we showed that PHTLS training improves the quality of documentation, which we used as a surrogate endpoint for learning effectiveness and awareness. In this regard, we demonstrated that participants use certain parts of training in real life, thereby suggesting that the learning methods of PHTLS training are effective. However, these results do not indicate as to whether patient care has changed.
Acknowledgments
We would like to thank the following for their contributions: Participating paramedics and the respective EMS agencies, and HSK Hospital Wiesbaden. Special thanks to Götz Brodermann, Marc Dieroff, and Nobert Hagner, who initiated the project and coordinated the study project at the site.
Author Contributions
- Conceptualization: DH MM BG CW.
- Data curation: DH.
- Formal analysis: DH RL MM SB MH.
- Funding acquisition: DH BG.
- Investigation: DH MM BG SB MH CW.
- Methodology: DH RL SB MH.
- Project administration: DH BG MD.
- Supervision: MM US PG.
- Writing – original draft: DH MM SB MH.
- Writing – review & editing: DH SB MH RL BG CW PG US MD MM.
References
- 1.
Deutsche Gesellschaft für Unfallchirurgie (2011) S3 –Leitlinie Polytrauma/Schwerverletzten-Behandlung. Accessed 14 February 2014.
- 2.
WHO (2015) Global Health Observatory (GHO) data. The road traffic death rate. Available: http://www.who.int/gho/road_safety/en/. Accessed 27 October 2015.
- 3.
Statistisches Bundesamt (2015) Verletzte bei Verkehrsunfällen. Available: https://www.destatis.de/DE/ZahlenFakten/Wirtschaftsbereiche/TransportVerkehr/Verkehrsunfaelle/Tabellen/VerletzteAlter.html. Accessed 27 October 2015.
- 4.
Bundesanstalt für Strassenwesen Ermittlung der Volkswirtschaftliche Kosten durch Straßenverkehrsunfälle in Deutschland. Berichte der Bundesanstalt für Straßenwesen, Unterreihe „Mensch und Sicherheit“. Bergisch Gladbach. 2 p.
- 5. Bernhard M, Hilger T, Sikinger M, Hainer C, Haag S, Streitberger K. et al. (2006) Patientenspektrum im Notarztdienst. Was hat sich in den letzten 20 Jahren geändert. Der Anaesthesist 55 (11): 1157–1165. pmid:17063342
- 6.
Statistisches Bundesamt (Destatis) (2015) Verkehrsunfälle. 50 p.
- 7.
(2015) Leistungen des Rettungsdienstes 2012/13. Berichte der Bundesanstalt für Straßenwesen, Unterreihe „Mensch und Sicherheit“, Heft M 260. Bergisch Gladbach. 2 p.
- 8. Gries A, Zink W, Bernhard M, Messelken M, Schlechtriemen T (2005) Einsatzrealität im Notarztdienst. Notfall + Rettungsmedizin 8 (6): 391–398.
- 9. Waydhas C (2003) Advanced Trauma Life Support. Notfall & Rettungsmedizin 6 (1): 33–36.
- 10. Ali J, Adam RU, Gana TJ, George B, Taylor A, Patino T. et al. (1998) Impact of the prehospital trauma life support programme in Trinidad and Tobago. The West Indian medical journal 47 (3): 102–104. pmid:9861861
- 11. Ali J, Adam RU, Gana TJ, Williams JI (1997) Trauma patient outcome after the Prehospital Trauma Life Support program. The Journal of trauma 42 (6): 1018–21; discussion 1021–2. pmid:9210534
- 12. Arreola-Risa C, Mock CN, Lojero-Wheatly L, La Cruz O de, Garcia C, Canavati-Ayub F. et al. (2000) Low-cost improvements in prehospital trauma care in a Latin American city. The Journal of trauma 48 (1): 119–124. pmid:10647576
- 13. Johansson J, Blomberg H, Svennblad B, Wernroth L, Melhus H, Byberg L et al. (2012) Prehospital Trauma Life Support (PHTLS) training of ambulance caregivers and impact on survival of trauma victims. Resuscitation 83 (10): 1259–1264. pmid:22366502
- 14. Blomberg H, Svennblad B, Michaelsson K, Byberg L, Johansson J, Gedeborg R et al. (2013) Prehospital Trauma Life Support Training of Ambulance Caregivers and the Outcomes of Traffic-Injury Victims in Sweden. Journal of the American College of Surgeons 217 (6): 1010. pmid:23978531
- 15. Häske D, Kreinest M, Wölfl CG, Frank C, Brodermann G, Horter J et al. (2013) Bericht aus der Praxis: Strukturierte Fortbildung zur Verbesserung der Versorgungsqualität im Rettungsdienst. Einsatz-Supervision als neuer Ansatz im Bereich der Rettungsdienst-Fortbildung in Wiesbaden und im Rheingau-Taunus-Kreis. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 107 (7): 484–489.
- 16. Häske D, Beckers SK, Hofmann M, Wölfl CG, Gliwitzky B, Grützner P et al. (14) The effect of paramedic training on pre-hospital trauma care (EPPTC-study): a study protocol for a prospective semi-qualitative observational trial. BMC Medical Education 2014 (32).
- 17. Häske D, Stuke L, Bernhard M, Heller AR, Schweigkofler U, Gliwitzky B et al. (2016) Comparison of the Pre-hospital Trauma Life Support (PHTLS) Recommendations and the German National Guideline on Treatment of Patients with Severe and Multiple Injuries. The journal of trauma and acute care surgery.
- 18.
NAEMT (2011) Präklinisches Traumamanagement. Prehospital Trauma Life Support (PHTLS); auf Grundlage der 7. englischen Auflage. München: Elsevier Urban & Fischer. XXIV, 619 p.
- 19. Güldner S, Mang H, Popp S, Heuser D, Krause M Christ M (2011) Gedanken zur Fehler- und Sicherheitskultur in deutschen Notaufnahmen. Notfall Rettungsmed 14 (5): 351–360. Accessed 26 August 2015.
- 20. Menditto VG, Lucci M, Polonara S, Pomponio G, Gabrielli A (2012) Management of minor head injury in patients receiving oral anticoagulant therapy: a prospective study of a 24-hour observation protocol. Annals of emergency medicine 59 (6): 451–455. pmid:22244878
- 21. Nishijima DK, Offerman SR, Ballard DW, Vinson DR, Chettipally UK, Rauchwerger A et al. (2012) Immediate and delayed traumatic intracranial hemorrhage in patients with head trauma and preinjury warfarin or clopidogrel use. Annals of emergency medicine 59 (6): 460–8.e1-7. pmid:22626015
- 22. Ulitsky A, Agresti D, Jeanmonod D, Agresti K, Jeanmonod R (2012) 269 When Elders Fall. Indications for Antiplatelet Agents and Anticoagulants in Elders Who Fall and the Incidence of Fall Recidivism. Annals of emergency medicine 60 (4): S96.
- 23.
Enke N (2009) Schnittstellen in der Notfallversorgung. Eine problemorientierte Systemanalyse. Hamburg: Diplomica Verlag. 1 online resource (vii, 116 p.
- 24. Schiff GD, Hasan O, Kim S, Abrams R, Cosby K, Lambert B et al. (2009) Diagnostic error in medicine: analysis of 583 physician-reported errors. Archives of internal medicine 169 (20): 1881–1887. pmid:19901140
- 25. Bergrath S, Rörtgen D, Skorning M, Fischermann H, Beckers SK, Mutscher C et al. (2011) Notärztliche Einsatzdokumentation in der Simulation. Videobasierte Fehleranalyse. Der Anaesthesist 60 (3): 221–229. Accessed 26 October 2015.
- 26. French SC, Salama NP, Baqai S, Raslavicus S, Ramaker J, Chan SB et al. (2006) Effects of an educational intervention on prehospital pain management. Prehospital emergency care: official journal of the National Association of EMS Physicians and the National Association of State EMS Directors 10 (1): 71–76.
- 27. Francis RCE, Schmidbauer W, Spies CD, Sörensen M, Bubser F, Kerner T et al. (2010) Standard operating procedures as a tool to improve medical documentation in preclinical emergency medicine. Emergency medicine journal: EMJ 27 (5): 350–354. pmid:20442162
- 28. Loftus T, Najafian H, Pandey SR, Ramanujam P (2015) The Impact of Documentation Training on Performance Reporting. Curēus 7 (7): e283. pmid:26261751
- 29. Häske D, Schempf B, Gaier G, Niederberger C (2014) Prähospitale Analgosedierung durch Rettungsassistenten. Anaesthesist 63 (3): 209–216.
- 30. Stewart M, Purdy J, Kennedy N, Burns A (2010) An interprofessional approach to improving paediatric medication safety. BMC Medical Education 10 (1): 1. Available: http://bmcmededuc.biomedcentral.com/track/pdf/10.1186/1472-6920-10-19?site=http://bmcmededuc.biomedcentral.com.
- 31. Kim Y, Park S-S, Bae H-J, Cho A-H, Cho Y-J, Han M-K et al. (2011) Stroke awareness decreases prehospital delay after acute ischemic stroke in Korea. BMC Neurol 11 (1): 2.
- 32. Westli H, Johnsen B, Eid J, Rasten I, Brattebø G (2010) Teamwork skills, shared mental models, and performance in simulated trauma teams. An independent group design. Scand J Trauma Resusc Emerg Med 18 (1): 47.
- 33. Jarodzka H, Jaarsma T, Boshuizen HPA (2015) In my mind. How situation awareness can facilitate expert performance and foster learning. Med Educ 49 (9): 854–856. pmid:26296399
- 34. Buyse M, Sargent DJ, Grothey A, Matheson A, Gramont A de (2010) Biomarkers and surrogate end points—the challenge of statistical validation. Nature reviews. Clinical oncology 7 (6): 309–317. pmid:20368727
- 35. Haes H de, Bensing J (2009) Endpoints in medical communication research, proposing a framework of functions and outcomes. Patient education and counseling 74 (3): 287–294. pmid:19150197
- 36. Hohn EA, Brooks AG, Leasure J, Camisa W, van Warmerdam J, Kondrashov D et al. (2015) Development of a surgical skills curriculum for the training and assessment of manual skills in orthopedic surgical residents. Journal of surgical education 72 (1): 47–52. pmid:25108508
- 37. Li Z, Ahn S, Chung K, Ragland DR, Wang W, Yu JW et al. (2014) Surrogate safety measure for evaluating rear-end collision risk related to kinematic waves near freeway recurrent bottlenecks. Accident; analysis and prevention 64: 52–61. pmid:24316507
- 38. Renard D, Geys H, Molenberghs G, Burzykowski T, Buyse M (2002) Validation of Surrogate Endpoints in Multiple Randomized Clinical Trials with Discrete Outcomes. Biom. J. 44 (8): 921–935. Accessed 2 December 2015.
- 39. Peschanski N, Garcia L, Delasalle E, Mzabi L, Rouff E, Dautheville S et al. (2015) Can transcutaneous carbon dioxide pressure be a surrogate of blood gas samples for spontaneously breathing emergency patients? The ERNESTO experience. Emergency medicine journal: EMJ. Accessed 9 February 2016.
- 40. Ali J, Adam R, Josa D, Pierre I, Bedsaysie H, West U et al. (1998) Effect of basic prehospital trauma life support program on cognitive and trauma management skills. World journal of surgery 22 (12): 1192–1196. pmid:9841742
- 41. Campbell JP, Maxey VA, Watson WA (1995) Hawthorne Effect. Implications for Prehospital Research. Annals of emergency medicine 26 (5): 590–594. pmid:7486367