https://orcid.org/0000-0003-3553-5796
Figures
Abstract
Background
The Health Resources and Services Administration (HRSA), Federal Office of Rural Health Policy (FORHP) funded the Evidence-Based Tele-Emergency Network Grant Program (EB TNGP) to serve the dual purpose of providing telehealth services in rural emergency departments (teleED) and systematically collecting data to inform the telehealth evidence base. This provided a unique opportunity to examine trends across multiple teleED networks and examine heterogeneity in processes and outcomes.
Method and findings
Six health systems received funding from HRSA under the EB TNGP to implement teleED services and they did so to 65 hospitals (91% rural) in 11 states. Three of the grantees provided teleED services to a general patient population while the remaining three grantees provided teleED services to specialized patient populations (i.e., stroke, behavioral health, critically ill children). Over a 26-month period (November 1, 2015 –December 31, 2017), each grantee submitted patient-level data for all their teleED encounters on a uniform set of measures to the data coordinating center. The six grantees reported a total of 4,324 teleED visits and 99.86% were technically successful. The teleED patients were predominantly adult, White, not Latinx, and covered by Medicare or private insurance. Across grantees, 7% of teleED patients needed resuscitation services, 58% were rated as emergent, and 30% were rated as urgent. Across grantees, 44.2% of teleED patients were transferred to another inpatient facility, 26.0% had a routine discharge, and 24.5% were admitted to the local inpatient facility. For the three grantees who served a general patient population, the most frequent presenting complaints for which teleED was activated were chest pain (25.7%), injury or trauma (17.1%), stroke symptoms (9.9%), mental/behavioral health (9.8%), and cardiac arrest (9.5%). The teleED consultation began before the local clinician exam in 37.8% of patients for the grantees who served a general patient population, but in only 1.9% of patients for the grantees who provided specialized services.
Conclusions
Grantees used teleED services for a representative rural population with urgent or emergent symptoms largely resulting in transfer to a distant hospital or inpatient admission locally. TeleED was often available as the first point of contact before a local provider examination. This finding points to the important role of teleED in improving access for rural ED patients.
Citation: Heppner S, Mohr NM, Carter KD, Ullrich F, Merchant KAS, Ward MM (2021) HRSA’s evidence-based tele-emergency network grant program: Multi-site prospective cohort analysis across six rural emergency department telemedicine networks. PLoS ONE 16(1): e0243211. https://doi.org/10.1371/journal.pone.0243211
Editor: Abhishek Makkar, University of Oklahoma Health Sciences Center, UNITED STATES
Received: March 31, 2020; Accepted: November 17, 2020; Published: January 12, 2021
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: Data for this study are maintained by the Rural Telehealth Research Center (RTRC). All data requests should be referred by email to rtrc-inquiry@uiowa.edu and the RTRC Executive Committee will review and respond to all requests. Data for this project were obtained under restrictive Data Use Agreements (DUAs) between the University of Iowa researchers and each service network which, in turn, received their data under DUAs with their participating hospitals. Those DUAs stipulate that the data, other than statistical summary information such as in the paper, cannot be shared publicly as they contain potentially identifying and sensitive patient information.
Funding: The efforts described in this manuscript were funded by the U.S. Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA). The project to identify a standardized set of measures was completed under contract number HHSH250201300018I/HHSH25034002T. The project to refine the measures, collect standardized data, and conduct analyses was completed under grant number UICRH29074. Funding for delivering teleED services and data collection was provided under grant numbers GO1RH27868, G01RH27870, G01RH27871, G01RH27872, G01RH27873, and G01RH27869.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Telehealth is increasingly being used to improve access to specialized emergency care for the approximately 60 million rural residents in the United States [1, 2]. Emergency departments (EDs) are a key source of care for rural communities and often face challenges with providing specialty care, such as trauma and behavioral health [3, 4]. The low volume seen in most rural EDs and the unpredictable nature of emergency care can make it difficult for rural hospitals to provide specialty services. For example, in many rural communities the majority of physicians covering the ED are family physicians rather than emergency specialists [5, 6]. Previous studies have found that telehealth in rural EDs can lead to improved patient care and support for rural providers [7–12].
In September 2014 the Federal Office of Rural Health Policy (FORHP), located in the U.S. Department of Health and Human Services’ Health Resources and Services Administration (HRSA), launched the Evidence-Based Tele-Emergency Network Grant Program (EB TNGP). This innovative new program allowed HRSA and its stakeholders to use grant funding to serve the dual purpose of delivering ED consultation services via telehealth to rural hospitals without emergency medicine specialists and systematically collecting data to inform the evidence base assessing the effectiveness of tele-emergency (teleED) care for rural populations. Compiling a cohort of otherwise unconnected provider-to-provider ED-based telehealth networks into a unified analysis offered a unique opportunity to generate a significant sample of rural telehealth encounters.
This program, administered by the Office for the Advancement of Telehealth, which is located in HRSA/FORHP, was developed in response to recommendations from the National Advisory Committee on Rural Health and Human Services. The committee advised HRSA to inform the evidence base by targeting its grant funds at specific areas of clinical telehealth services and analyzing the impact of those services. These recommendations were informed in part by a 2012 Institute of Medicine workshop entitled “The Role of Telehealth in an Evolving Health Care Environment” which was convened with financial support from a HRSA contract [13]. A published summary of this meeting noted the challenges in generating statistically significant results in small rural areas that demonstrate the impact of telehealth and inform health care policy [13]. These organizations stressed the need to increase the evidence base for telehealth services with a specific focus on rural populations. HRSA is in a unique position to utilize its grant-making authority to simultaneously increase access to care in rural areas through the use of telehealth and support rigorous analysis to inform the evidence base.
The purpose of this study was to describe the hospital and patient characteristics of those providing and receiving teleED services, including the indications, success, and outcomes of teleED encounters, and to compare the use of teleED between general and specialized teleED services. Our hypotheses were that teleED would be used for broad indications in general teleED networks and very narrow indications in specialized targeted networks, that it would be most commonly used with the sickest patients, and that it would be technically successful in these established networks.
Methods
Study design
This study was a prospective cohort study of patients treated with teleED in any of the six grantee rural teleED networks. For the purposes of this study, teleED was defined as an immediate, synchronous, interactive audio/video connection between an ED originating site (spoke) and a distant site where a specialist is located (hub). Participating networks collected a defined uniform set of measures and submitted those data to the data coordinating center for pooling and analysis.
Participating networks
Six health systems received grant funding from HRSA’s FORHP under the EB TNGP to implement teleED services. The six grantees were: Avera Health, Saint Vincent Healthcare, Union Hospital, University of California–Davis, University of Kentucky, and University of Virginia. The six grantees provided teleED services to 65 hospitals in 11 states–California, Indiana, Iowa, Kansas, Kentucky, Minnesota, Montana, Nebraska, North Dakota, South Dakota, and Virginia. All six organizations had at least ten years of experience in offering telehealth services to their regions and were continually improving and expanding telehealth services.
Grantees provided teleED services customized to the needs and resources in their areas for specific project goals. In particular, three of the grantees (Avera Health, St. Vincent Healthcare, University of Kentucky) provided teleED services to a general patient population (i.e., any patient seen in the ED was eligible for teleED) while the remaining three grantees provided teleED services to specialized patient populations (behavioral health and neurology at Union Hospital; pediatric emergency/critical care at University of California–Davis; stroke care at University of Virginia). At the three general teleED networks, the hub sites were staffed by board-certified emergency physicians. At the three specialized teleED networks, the hub sites were staffed by specialty physicians (e.g., psychiatrists, pediatric critical care physicians, neurologists and vascular physicians). Five of the six hub sites used on-call physicians, while the sixth hub had dedicated staff on site.
Measure development
HRSA awarded a contract in Fall 2014 to Mathematica Policy Research to identify a set of evidence-based measures that would be appropriate for use in teleED studies. University of Iowa investigators were subcontracted for the project and completed a systematic review of the teleED literature and compiled a library of existing measures for consideration [12]. A set of measures that could be applied to all teleED encounters and a set of measures that could be applied to teleED encounters for patients with specific conditions were identified. Mathematica Policy Research worked with several EB TNGP grantees to pilot test the measures, then developed and refined a data collection tool (Tele-Emergency Performance Assessment Reporting Tool, T-PART) [14]. University of Iowa investigators in the Rural Telehealth Research Center (RTRC), funded under a HRSA cooperative agreement, continued refinement of the standardized data elements and T-PART before serving as the data coordinating and analysis center for the EB TNGP [15].
Data collection
From November 1, 2015 to December 31, 2017, each grantee submitted patient-level data for all their teleED encounters to RTRC [15]. Inclusion criteria encompassed all patients seen by each network through their teleED program. Data on 49 data elements were submitted on a predetermined schedule by each grantee using the Microsoft Excel (Microsoft Corporation, Redmond, WA)-based T-PART tool [15]. RTRC provided the grantees with an extensive data dictionary which included definitions, allowable data values, and guidelines for data abstraction [15]. RTRC used data checking algorithms and worked iteratively with grantees to address missing and out-of-range data submissions. Data elements reported in these analyses include patient demographics (age, sex, race, and ethnicity reported using U.S. Census definitions), insurance coverage, technical success, weekday and time of ED visit and teleED consult, severity of illness, and ED disposition. Severity of illness was reported by grantees using the Emergency Severity Index, a validated measure of patient acuity designed to rate the urgency of needed care and predict ED resource consumption [16–18].
Additional data sources
Spoke hospital characteristics were abstracted from the 2016 American Hospital Association annual survey [19]. Rural-Urban Commuting Area codes were used to define spoke hospitals as located in metro or rural locations [20].
Analysis
The primary analysis is descriptive, reporting counts and proportions. Categorical variables were compared with chi-squared tests or Fisher’s exact test, as appropriate. Continuous variables were compared with t-tests and Wilcoxon rank sum tests, as appropriate. All statistical tests were conducted as two-tailed tests with significance defined as p<0.05. The sample size of this observational study was selected to be the entire population of interest based on the 26-month enrollment period. Because all study procedures occurred in the initial visit, no cases were lost to follow-up. Missing data were not imputed, and summary data are reported of complete data on each data element. The study protocol was approved by the Institutional Review Board at the RTRC (University of Iowa Institutional Review Board) and each of the grantee organizations (Avera Institutional Review Board, Indiana State University Institutional Review Board, Rector and Visitors of the University of Virginia Intuitional Review Board, St. Vincent Institutional Review Board, University of California, Davis Institutional Review Board Administration, and University of Kentucky Institutional Review Board) and data use agreements defined data sharing arrangements between all entities. No personal health information was transmitted and all data were anonymized prior to transfer. The findings are reported in accordance with the Strengthening The Reporting of Observational Studies in Epidemiology (STROBE) guidelines [21].
Results
Over the 26-month data collection period, the six grantees reported a total of 4,324 ED visits for which teleED was used.
Hospital characteristics
The 65 spoke hospitals participating in the teleED networks were predominantly in rural locations (90.8%). Spoke hospitals included 44 (67.7%) Critical Access Hospitals and 21 (32.3%) Prospective Payment System) hospitals. General teleED networks were more likely to include Critical Access Hospitals (81.1%) than Prospective Payment System hospitals (18.9%) while the specialized teleED networks had an even split (50% Critical Access Hospitals, 50% Prospective Payment System hospitals). The median ED annual volume was 8,427 visits across spoke hospitals. General teleED network hospitals had an annual median of 2,437 ED visits compared to 13,760 for specialized teleED network hospitals, consistent with their respective Critical Access Hospital/Prospective Payment System hospital status.
Patient characteristics
The demographics of the EB TNGP telehealth patient population are shown in Table 1 for the total group and for general teleED networks vs. specialized teleED networks. Because of the condition-specific criteria for the specialized teleED services, the demographics of the teleED patients differed significantly between the two groups. Overall, the teleED patients were predominantly adult, White, not Latinx, and covered by Medicare or private insurance.
TeleED presenting complaint
Presenting complaints for the patients where teleED was activated differed for the three grantees who provided general teleED services compared to the three grantees who provided specialized teleED services (p<.0001). For the grantees who provided teleED services to a general patient population, the most frequent presenting complaints for which teleED was activated were chest pain (25.7%), injury or trauma (17.1%), stroke symptoms (9.9%), mental/behavioral health (9.8%), and cardiac arrest (9.5%). In contrast, grantees who provided specialized teleED services had chief complaints distributed to match the services on which the network focused. In particular, two of the grantees provided teleED specialized stroke services and one of the grantees provided teleED specialized behavioral health services, so the most frequent presenting complaints for these grantees aligned with these services. The distribution of chief complaints (by network) are summarized in Fig 1.
Networks are divided into General teleED Networks or Specialized teleED Networks. Chief complaints were assigned at the beginning of a teleED encounter. The shade of each box indicates the proportion of cases treated within the teleED network that fall within a chief complaint category. The legend to the right shows the shade associated with each proportion.
TeleED processes
Successful administration was defined as encounters where voice and video quality were sufficient to complete the consultation. Unsuccessful administration was defined as encounters where the voice and/or video quality (e.g. unreachable network, poor image quality, poor ECG signal quality) were not sufficient to complete the consultation. Only 6 of the 4,324 teleED encounters were judged to be unsuccessful, resulting in 99.86% of teleED encounters technically successful.
The distribution of teleED encounters was uniform across day of week, with the highest utilization on Thursdays (15.0%) and the lowest on Wednesdays (13.3%). Among teleED encounters, a higher proportion occurred for patients who arrived at the ED during business hours (7:30 am to 5:30 pm) compared to outside business hours (53.8% and 46.2%, respectively), and the difference was particularly evident for the grantees with general teleED services, as shown in Table 2.
Timing of TeleED consultation
The sequence of consultation differed significantly for the three grantees who provided teleED services to a general patient population compared to the three grantees who provided teleED services to specialized patient populations (Table 3). Grantees providing general teleED services began consultations prior to local provider evaluation much more frequently than those providing specialized teleED care (37.8% vs 1.9%, p<0.0001).
Severity of illness
Analysis of the Emergency Severity Index (ESI) indicated that across grantees, 7% needed resuscitation services (ESI level 1), 58% of teleED patients were rated as emergent (ESI level 2), and 30% were rated as urgent (ESI level 3). ESI did not differ significantly for the patients seen by grantees delivering services to a general patient population compared to grantees delivering specialized services (p = 0.211). However, the specialized teleED services took longer to activate the teleED consultation across all ESI levels, even those that were most serious (Table 4, p<0.0001). Unfortunately, because ESI was not routinely collected from all rural hospitals, this was the variable with the greatest missingness (e.g., 44% of participating cases from one grantee).
The proportion of cases within each ESI category are different between telehealth cases in General and Specialized teleED networks (p<0.001).
TeleED outcomes
The discharge status of all patients receiving teleED services was examined. Over all six grantees, the most common disposition was transfer to another inpatient facility (44.2%), followed by routine discharge (26.0%), and admission to the local inpatient facility (24.5%). As shown in Table 5, discharge status differed for grantees delivering services to a general patient population compared to grantees delivering specialized services, with the specialized teleED services transferring and admitting more patients than the general teleED services (p<.0001).
The patient disposition is different in General teleED networks vs Specialized teleED networks (p<0.001).
Discussion
TeleED services are one tool available to address challenges with access to specialty care in rural EDs. This study describes hospital and patient characteristics of those served by specialized and generalized tele-ED programs, across a cohort of established networks. Our findings show that teleED is used for a variety of presenting urgent and emergent conditions largely resulting in transfer to a distant hospital or inpatient admission locally and it provided timely access to care that would otherwise be unavailable. These findings also show that in these established networks, teleED was nearly always technically successful, which aligns with prior research showing that technical success is not a barrier to teleED use [12, 22]. This study illustrates how the structure and goals of networks affect the application and function for their participating spoke hospitals and patients.
Access to care is one of the primary drivers for telehealth in rural areas observed in this study. Interestingly, 37.8% of generalized ED provider encounters began prior to evaluation by a local health care provider. This telehealth-first strategy occurs primarily in hospitals without a dedicated ED provider. In many of these facilities, the provider responsible for covering the ED may be in clinic, seeing patients elsewhere in the hospital, or on call from home. A 2013 memorandum from the Centers for Medicare & Medicaid Services (CMS) clarified that Critical Access Hospitals could use a physician at a telehealth hub to fulfill the requirements under their conditions of participation and the Emergency Medical Treatment and Active Labor Act (EMTALA) related to physician coverage for the ED [23]. In these low-volume EDs, telehealth can provide immediate access to a provider who can perform an evaluation, order tests, and arrange transportation for transferring patients, if appropriate. Tele-ED, paired with the policy clarification from CMS, increases rural hospitals’ options to address physician shortages and challenges with ED coverage, ultimately improving access to care for rural communities [24]. This focus on access to care reflects one of the priorities of a multi-stakeholder panel convened by the National Quality Forum, with funding from a contract with CMS to propose a telehealth measurement framework [25].
One interesting finding is the breadth of conditions for which teleED was consulted. Prior reports have described single network experiences [26, 27] or condition-specific telehealth interventions [28–30]. While many networks have developed telehealth infrastructure for cardiac emergencies, trauma, and stroke, that infrastructure is often used for other clinical scenarios where expert consultation may be valued. This was particularly apparent in the specialty-specific networks, where 5% of the encounters reported conditions outside the focus area of the network. Our study was unable to measure the impact of these off-specialty consultations, but local providers may have viewed the telehealth infrastructure as an option for patients who otherwise were difficult to manage. Even in the generalized networks, only 72.1% of encounters were for the top five chief complaints, suggesting that infrequent and unusual presentations may be over-represented in patients for whom telehealth is consulted.
The largest prior report of teleED service implementation came from the University of Mississippi TelEmergency network (n = 26,697 from 1 network) [27]. In that service, very strict consultation criteria were applied because all participating spoke hospital EDs were staffed by nurse practitioners overseen by faculty at the University of Mississippi ED. Over a 3-year period, over 40% of all ED patients in 10 rural centers used teleED, and most of those patients (65%) were discharged to home. The intervention was paired with a quality improvement initiative, and the patients in their cohort (compared with our study) were much younger with a much lower proportion of chest pain, trauma, and stroke. These markedly different distributions of users reflect the heterogeneity in telehealth adoption, and highlight the importance of cross-network data set analysis.
One challenge in telehealth research is predicting the long-term impact of telehealth programs at scale. Pilot projects, single-center analyses, and demonstration programs may over-estimate the impact of novel care delivery mechanisms. With tele-stroke programs, for example, the earliest studies showed the greatest benefits, although consistent benefit was shown even in later studies [31]. Very few analyses of telehealth interventions have pooled data from multiple networks, and a systematic review of teleED implementation papers did not identify any multi-network studies [12]. The goal of the EB TNGP program was to pool heterogeneous networks that have expanded to scale to better estimate how teleED might reshape rural health care if it were expanded beyond the networks that pioneered its use [32–35]. Observing the variation between networks in their application and use increases confidence in how teleED might perform at scale, and it allows for better prediction of the impact of systematic telehealth expansion into new health systems.
Limitations
This study has several limitations. In order to maximize feasibility and participation, the number of data elements collected for the T-PART instrument was limited. Second, this data set does not collect long-term clinical outcomes. By restricting outcomes to the ED, we have no significant loss to follow-up based on our data set. Third, comparing the racial distribution in this sample to the US Census rural population in the 11 states represented, the American Indian/Native Alaska percentage was about twice that expected and the Black/African American was about a third that expected.
Conclusion
TeleED services are used across a variety of clinical conditions in rural EDs, and the indication for use, the timeliness of telehealth activation, and the ultimate ED disposition of patients varies based on the structure, design, and purpose of the provider-to-provider network being studied. HRSA’s Evidence-Based Tele-Emergency Network Grant Program has assembled a large dataset using uniform measures to enable cross-network collaborative research, and unifying projects like this will continue to enable objective evaluation of novel care delivery systems. Future work should continue to evaluate the impact of teleED on costs of care, clinical outcomes, and long-term recovery from acute illness, and strong collaborative projects such as this can enable these comparative effectiveness studies.
Supporting information
S1 Checklist. STROBE statement—Checklist of items that should be included in reports of observational studies.
https://doi.org/10.1371/journal.pone.0243211.s001
(DOCX)
References
- 1.
Urban and Rural Classification and Urban Areas Criteria. Suitland, Maryland: US Census Bureau; 2010.
- 2. Mueller KJ, Potter AJ, MacKinney AC, Ward MM. Lessons from tele-emergency: improving care quality and health outcomes by expanding support for rural care systems. Health Affairs. 2014;33(2):228–34. pmid:24493765
- 3. Carr BG, Bowman AJ, Wolff CS, Mullen MT, Holena DN, Branas CC, et al. Disparities in access to trauma care in the United States: A population-based analysis. Injury. 2017;48(2):332–8. pmid:28069138
- 4. Andrilla CHA, Patterson DG, Garberson LA, Coulthard C, Larson EH. Geographic variation in the supply of selected behavioral health providers. American Journal of Preventive Medicine. 2018;54(6):S199–S207. pmid:29779543
- 5. Groth H, House H, Overton R, Deroo E. Board-certified emergency physicians comprise a minority of the emergency department workforce in Iowa. The Western Journal of Emergency Medicine. 2013;14(2):186–90. pmid:23599868
- 6. Hall MK, Burns K, Carius M, Erickson M, Hall J, Venkatesh A. State of the national emergency department workforce: Who provides care where? Annals of Emergency Medicine. 2018;72(3):302–7. pmid:29753519
- 7. Dayal P, Hojman NM, Kissee JL, Evans J, Natale JE, Huang Y, et al. Impact of telemedicine on severity of illness and outcomes among children transferred from referring emergency departments to a children’s hospital PICU. Pediatric Critical Care Medicine. 2016;17(6):516–21. pmid:27099972
- 8. du Toit M, Malau-Aduli B, Vangaveti V, Sabesan S, Ray RA. Use of telehealth in the management of non-critical emergencies in rural or remote emergency departments: A systematic review. Journal of Telemedicine and Telecare. 2019;25(1):3–16. pmid:28980853
- 9. Kim JW, Tiyyagura G, Langhan M. A qualitative analysis of general emergency medicine providers’ perceptions on pediatric emergency telemedicine. Pediatric Emergency Care. 2019 Dec;35(12):856–861. pmid:28225376
- 10. Mohr NM, Vakkalanka JP, Harland KK, Bell A, Skow B, Shane DM, et al. Telemedicine use decreases rural emergency department length of stay for transferred North Dakota trauma patients. Telemedicine Journal and e-Health. 2018;24(3):194–202. pmid:28731843
- 11. Mohr NM, Young T, Harland KK, Skow B, Wittrock A, Bell A, et al. Emergency department telemedicine shortens rural time-to-provider and emergency department transfer times. Telemedicine Journal and e-Health. 2018;24(8):582–93. pmid:29293413
- 12. Ward MM, Jaana M, Natafgi N. Systematic review of telemedicine applications in emergency rooms. International Journal of Medical Informatics. 2015;84(9):601–16. pmid:26072326
- 13.
IOM (Institute of Medicine). The role of telehealth in an evolving health care environment: Workshop summary. Washington, DC: The National Academies Press. 2012
- 14. Harris Y, Gilman B, Ward MM, Ladinsky J, Crowley J, Warren C, et al. Building the evidence base for tele-emergency care: Efforts to identify a standardized set of outcome measures. Telemedicine Journal and e-Health. 2017;23(7):561–6. pmid:28079461
- 15.
Ward MM, Ullrich F, Merchant KAS, Mohr NM, Weigel P, MacKinney AC, et al. Identifying measures and data elements for the HRSA Evidence-Based Tele-Emergency Network Grant Program. RTRC Research & Policy Brief. March 2020
- 16. Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. Journal of Emergency Nursing. 2004;30(1):22–9. pmid:14765078
- 17. Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Academic Emergency Medicine. 2000;7(3):236–42. pmid:10730830
- 18.
Gilboy N, Tanabe T, Travers D, Rosenau A. Emergency Severity Index (ESI): A triage tool for emergency department care, Version 4. Implementation handbook 2012 edition. Rockville, MD: Agency for Healthcare Research and Quality; 2011.
- 19.
AHA Annual Survey Database, 2015 ed. Chicago, IL: American Hospital Association, 2016.
- 20. Hart LG, Larson EH, Lishner DM. Rural definitions for health policy and research. American Journal of Public Health. 2005;95(7):1149–55. pmid:15983270
- 21. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ (Clinical research ed). 2007;335(7624):806–8.
- 22. Cutting S, Conners JJ, Lee VH, Song S, Prabhakaran S. Telestroke in an urban setting. Telemedicine Journal and e-Health. 2014;20(9):855–7. pmid:24968197
- 23.
Critical Access Hospital (CAH) Emergency Services and Telemedicine: Implications for Emergency Services Condition of Participation (CoPs) and EMTALA On-Call Compliance Bethesda, Maryland: Center for Clinical Standards and Quality/Survey & Certification Group; 2013 [https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/downloads/Survey-and-Cert-Letter-13-38.pdf.]
- 24. Ward MM, Merchant KAS, Carter KD, Zhu X, Ullrich F, Wittrock A, et al. Use of telemedicine for ED physician coverage in critical access hospitals increased after CMS policy clarification. Health Affairs (Project Hope). 2018;37(12):2037–44. pmid:30633684
- 25.
Hollander J, Ward M, Alverson D, Bashshur R, Darkins A, DePhillips H, et al. Creating a Framework to Support Measure Development for Telehealth. Washington, DC: National Quality Forum, 2017.
- 26. Ward MM, Ullrich F, MacKinney AC, Bell AL, Shipp S, Mueller KJ. Tele-emergency utilization: In what clinical situations is tele-emergency activated? Journal of Telemedicine and Telecare. 2016;22(1):25–31. pmid:26026189
- 27. Sterling SA, Seals SR, Jones AE, King MH, Galli RL, Isom KC, et al. The impact of the TelEmergency program on rural emergency care: An implementation study. Journal of Telemedicine and Telecare. 2017;23(6):588–94. pmid:27470505
- 28. Mohr NM, Harland KK, Chrischilles EA, Bell A, Shane DM, Ward MM. Emergency department telemedicine is used for more severely injured rural trauma patients, but does not decrease transfer: A cohort study. Academic Emergency Medicine. 2017;24(2):177–85. pmid:28187248
- 29. Medeiros de Bustos E, Berthier E, Chavot D, Bouamra B, Moulin T. Evaluation of a French regional telemedicine network dedicated to neurological emergencies: A 14-Year Study. Telemedicine Journal and e-Health. 2018;24(2):155–60. pmid:29346039
- 30. Mohr NM, Young T, Harland KK, Skow B, Wittrock A, Bell A, et al. Telemedicine is associated with faster diagnostic imaging in stroke patients: A cohort study. Telemedicine Journal and e-Health. 2019;25(2):93–100. pmid:29958087
- 31. Baratloo A, Rahimpour L, Abushouk AI, Safari S, Lee CW, Abdalvand A. Effects of telestroke on thrombolysis times and outcomes: A meta-analysis. Prehospital Emergency Care. 2018;22(4):472–84. pmid:29345529
- 32. Swanson MB, Miller AC, Ward MM, Ullrich F, Merchant KA, Mohr NM. Emergency department telemedicine consults decrease time to interpret computed tomography of the head in a multi-network cohort. Journal of Telemedicine and Telecare. 2019 Nov 4. pmid:31684801
- 33. Mohr NM, Campbell KD, Swanson MB, Ullrich F, Merchant KAS, Ward MM. Provider-to-provider telemedicine improves adherence to sepsis bundle care in community emergency departments. Journal of Telemedicine and Telecare. 2020 Jan 5. pmid:31903840
- 34. Miller AC, Ward MM, Ullrich F, Merchant KAS, Swanson MB, Mohr NM. Emergency department telemedicine consults are associated with faster time-to-ECG and time-to fibrinolysis for myocardial infarction patients. Telemedicine and e-Health. 2020 Feb 28. pmid:32109200
- 35. Ward MM, Carter KD, Ullrich F, Merchant KAS, Natafgi N, Zhu X, et al. Averted transfers in rural emergency departments using telemedicine: Rates and costs across six teleED networks. Telemedicine and e-Health, 2020 Aug 24. pmid:32835620