Introduction

Diabetes is a complex chronic disease that afflicts millions of individuals worldwide, resulting in significant morbidity, mortality, and health care resources utilization [1–6]. With projected increases in the prevalence of diabetes and costs arising from the long-term complications incurred by this condition (e.g. cardiovascular disease, strokes, retinopathy, nephropathy, and amputations), policy makers and health care providers continue to focus on identifying self-management interventions that improve diabetes care and patient related outcomes. Prior research has shown that diabetes self-management education and support improve hemoglobin A1c (HbA1c) levels by as much as 1%, reduce the risk of developing debilitating and life‐threatening complications, and have a positive effect on other psychosocial and behavioral aspects of diabetes [7]. However, providing optimal care for patients with diabetes remains a challenge for healthcare systems and providers. Patients often encounter various barriers in adhering to self-management programs due to lack of knowledge and understanding of self-care activities, lack of individualized and coordinated care, inconvenient and costly education sessions, and poor patient-provider communication [8].

Mobile technologies such as cell phones/smartphones, handheld tablets, and other wireless devices known as mHealth, offer new and exciting opportunities for addressing some of these challenges by enabling remote patient monitoring and delivery of clinical advice through a wide-range of functions (e.g. text messaging, web browsing, email, and videos). Mobile phones are now omnipresent with worldwide usage rates nearing 100% (96% globally; 128% in developed countries; and 89% in developing countries) [9]. The ubiquitous nature of mobile phones coupled with their constantly evolving processing and connectivity power create opportunities for new and innovative approaches to support blood glucose and diet monitoring, measurement of daily physical activity, education, and other activities that can facilitate diabetes self-management and enhance patient-provider communication [10, 11].

Current interest in mHealth interventions to improve management of diabetes has led to a plethora of empirical studies [12]. Healthcare providers, researchers, and policy makers who have an interest in the effects of mHealth for diabetes management are inundated with vast amounts of information from numerous clinical trials and observational studies. Evidence syntheses, and in particular systematic reviews (SRs) partly address the problem of information overload by pooling together and collating all the available evidence from multiple trials. However, practitioners often find it difficult to reliably retrieve and keep up to date with the growing volume of SRs that is published in a variety of formats and sources. Furthermore, as with all types of research, available SRs may vary in quality and their findings can potentially be flawed due to methodological issues and risks of bias. These flaws can cause a SR to be biased and potentially misleading for decision-making. Prior research in the area of home telemonitoring for chronic disease management has shown that not all SRs are truly systematic and their methodological quality varies significantly [13, 14]. Hence, critical appraisal and synthesis of this evidence is needed to provide reliable and accessible information to clinicians and decision makers. Overviews of SRs (also known as umbrella reviews) [15] are an efficient way to critically appraise prior reviews and gather the best available evidence in a single source to provide broad, cumulative statements that summarize the current evidence and knowledge on the effectiveness of interventions [16,17]. Such overviews are helpful and can serve as starting points for decision makers to unpack the evidence towards finding solutions to improving practice and identify areas where new research is needed [16].

This overview of SRs aims to collect, appraise, and synthesize evidence from multiple SRs and meta-analyses on the effectiveness of mHealth interventions for patients with diabetes, in order to provide researchers, policy makers and practitioners with the evidence they need to make informed decisions. It also aims to identify research gaps that need to be addressed and develop a series of recommendations for improving the quality of future reviews and studies in this area.

Methods

Results

Our initial search yielded 989 citations after removal of 217 duplicates (Fig 1). Based on titles and abstracts, we excluded 901 references that clearly did not meet our inclusion criteria. We then retrieved the full-text of the remaining 88 articles for further examination and manually screened their references to identify any relevant systematic reviews that were not captured by the original search strategy. This process yielded another 6 articles. After close examination, we excluded 78 articles that did not meet our eligibility criteria. The references of the excluded articles along with the primary reasons of exclusion are available in S3 Appendix. Overall, we identified 15 SRs eligible for inclusion [20–34], over 16 references. One Cochrane SR [30] was also published as a journal article [35].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Screen and selection process.

https://doi.org/10.1371/journal.pone.0173160.g001

Description of the included systematic reviews

Reviews included in our sample were published between 2008 and 2014. Most (n = 9) were published from 2012 onward with one third of the reviews being published in 2013 alone. Two of the SRs were Cochrane reviews. Six reviews originated in the United States, five in the United Kingdom, two in China, one in Canada, and one in Iran. Despite some overlap, the individual SRs varied in their scope of inquiry and used different inclusion and exclusion criteria for the selection of eligible studies.

As shown in Table 1, seven SRs included only randomized controlled trials (RCT) [22, 23, 30–34]. The remaining reviews included, in addition to RCTs, non-randomized controlled trials and cohort studies with a pre-post design. Two SRs included only studies that involved patients with T1D [23,25]; two reviews focused explicitly on adult patients with T2D [30,32]; and the remaining 11 included studies that involved patients with type 1, type 2, or both types of diabetes. Of these 11 reviews, only 2 analyzed and reported the effects of mHealth interventions separately for each type of diabetes [20, 29].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the included systematic reviews.

https://doi.org/10.1371/journal.pone.0173160.t001

The number of included studies across the reviews varied significantly (range: 2–21; median: 10 [95% CI: 8 to 15]) due to differences in the inclusion criteria and search strategies. Altogether, the reviews identified 22 unique studies involving patients with T1D (7 RCTs, 5 RCOTs, and 10 with other designs); 26 studies across 31 publications involving patients with T2D (18 RCTs over 23 reports and 8 with other designs); 4 studies involving both T1D and T2D patients (2 RCTs and 2 with other designs); and 1 study that did not specify the type of diabetes of the participants. Figs 2 and 3 present the citation matrices that cross-link with black boxes the individual SRs with all the published reports of the studies included in them. The colored boxes indicate the type of diabetes of the participants included in each of these studies. The full references of the trials are available in S4 Appendix.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Citation matrix of previously published reports of randomized controlled and cross-over trials included in the systematic reviews (all references are available in S4 Appendix).

https://doi.org/10.1371/journal.pone.0173160.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Citation matrix of previously published observational, non-randomized controlled, and uncontrolled trials included in the systematic reviews (all references are available in S4 Appendix).

https://doi.org/10.1371/journal.pone.0173160.g003

Six SRs focused explicitly on the effects of mHealth interventions for patients with diabetes. The remaining reviews had a broader scope of inquiry: 4 SR examined various remote patient monitoring interventions in addition to mHealth (e.g. video and tele-conferencing), and 5 SRs investigated the effects of mHealth on a range of chronic conditions that included diabetes as an identifiable subset. In line with our inclusion criteria, these reviews were included in our sample because they reported the effects of mHealth interventions for diabetes separately from other interventions and/or conditions.

The mHealth interventions included in the SRs were diverse in nature and utilized a wide range of technological innovations, including text messaging (SMS), mobile applications, Bluetooth-enabled glucose meters, as well as secure websites/web-portals that could be accessed through the participants’ mobile device for data entry and patient support. The most prevalent method for sending blood glucose measurements and/or receiving self-management support (e.g. encouragement, education, reminders, and recommendations) involved the use of text messages (SMS) via mobile phones (34 out of 53 studies). Twelve studies involved the use of web-portals that patients accessed through their cell phones to enter diabetes self-care data and receive feedback. In 19 studies, data entry and transmission of glucose data were performed automatically via Bluetooth-enabled devices or other connectivity methods (e.g. infrared transmission or attachment of the glucose device to the mobile phone). In the remaining studies data entry was performed manually. The majority of studies included in the SRs (38 out of 53) encouraged self-monitoring of blood glucose combined with other intervention components, such as support of medication adjustment and reinforcement of lifestyle changes. Half of the studies (26 out of 53) provided educational support to patients via SMS (15 studies) and/or the Internet (11 studies). In 28 of 53 trials patient data were transmitted daily or more often. The duration of follow-up ranged from 3 to 12 months.

Glycemic (HbA1c) control was the primary outcome of interest in all the included reviews. Out of the 15 reviews, only 6 reported other outcomes besides HbA1c. These included changes in body weight [23, 26], participant satisfaction [25, 26], quality of life [27, 33], self-efficacy [26, 28, 33], and costs [26, 33]. Five SRs [23, 24, 29, 30, 32] performed a meta-analysis to calculate the effects of mHealth interventions on HbA1c. The remaining reviews used narrative/qualitative synthesis approaches.

Methodological quality of the included systematic reviews

As shown in Table 2, the methodological quality of the SRs varied considerably and the majority of them had important limitations. Only 3 reviews were found to be of high quality, achieving a score of 8 or more on the 11-point AMSTAR scale [23, 24, 30]. These reviews adhered to all or most of the methodological criteria, indicating minimal bias in their design and execution. Of the three high quality reviews, two were Cochrane SRs. Cochrane reviews are internationally recognized as the highest standard in evidence-based health care, with rigorous procedures and reporting requirements. Six reviews were of moderate quality, scoring between 4 and 7 AMSTAR points, and six reviews were of low quality scoring less than 4 points. As shown in Table 2, the majority of SRs performed poorly in several domains. All but one SR (n = 14, 93%) failed to assess the sources of support or conflict of interest in the included studies (Q11) and 80% (n = 12) did not provide evidence of a published a priori protocol (Q1). Also, most reviews (n = 12, 80%) did not search the grey literature for eligible studies (e.g. conference proceedings) and restricted their search to English articles only (Q4), which in turn increases the risk for publication and language bias. Only 3 SRs provided a list of excluded studies along with reasons for exclusion of each study (Q5). In more than half of the reviews (n = 8, 53%) there were no details on the number of reviewers involved in, or the process used for, the selection of studies and data extraction, which also raises concerns about the potential subjectivity of decisions at these stages. Finally, one third of the reviews did not appraise the quality of the included trials to identify potential sources of bias (Q7), and approximately half of the included SRs formulated conclusions without taking into consideration the scientific quality of the included trials (Q8). Hence, the possibility that biased studies have inflated the results of these SRs cannot be ruled out. In addition to the methodological issues identified through the AMSTAR instrument, during data extraction and citation analysis we found that three reviews [21, 28, 31], double-counted the results of single trials with duplicate publications and companion papers describing the same study.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Methodological quality of systematic reviews based on AMSTAR criteria^1,2.

https://doi.org/10.1371/journal.pone.0173160.t002

Discussion

This umbrella review identified, critically appraised, and synthesized evidence from 15 SRs that evaluated the effectiveness of mHealth interventions, involving the use of mobile devices and applications for remote patient monitoring and delivery of clinical feedback for self-management of diabetes. The reviews included 52 unique studies, most of which were RCTs, published between 1996 and 2014, and evenly distributed between T1D and T2D. To our knowledge, this is the first overview of SRs to pool together in a single source 18 years of mHealth research evidence in the area of diabetes. A major strength of this overview is the use of rigorous methods suggested by the Cochrane Collaboration [17] with the intent to minimize the impact of bias arising from the different sources within and across the included SRs, as well as the overview process itself. More specifically, we carried out a comprehensive search to identify all relevant SRs, performed study selection, data extraction, and quality appraisal in duplicate manner and used a structured, standardized approach that has been developed to help synthesize and rank the evidence across SRs with complex and diverse interventions [14,17]. Nonetheless, the results of this overview are inevitably constrained by the methodological quality and reporting characteristics of the included SRs [36], as well as the quality of evidence within these reviews. Using the AMSTAR instrument, we found that the methodological quality of SRs in this area is suboptimal. Nearly half of the SRs (n = 6; 40%) are characterized by important methodological limitations and risks of bias that impair their internal validity and limit their usefulness for clinical and policy decision-making purposes. In addition to the methodological quality, another key issue that was identified relates to the grouping of T1D and T2D studies in the analysis and synthesis of results. Despite important differences in the demographic characteristics and management approach of T1D and T2D patients, most SRs in our sample (60%, n = 9) mixed the results of the trials and did not differentiate the effects of mHealth interventions between types of diabetes. This is an important limitation that influenced the results of these reviews and made the interpretation of their findings difficult due to the clinical heterogeneity of the included studies. Taken collectively, the findings of our critical appraisal highlight an urgent need to further improve the rigor of future SRs of mHealth interventions in the area of diabetes.

Focusing on the high-quality SRs that offered the most direct evidence, this overview demonstrates that on average, mobile phone-based interventions with clinical feedback improve glycemic control (HbA1c) compared to standard care or other non-mHealth approaches by as much as 0.8% for patients with T2D and 0.3% for patients with T1D, at least in the short-term (≤12 months). The available evidence effectively rules out relatively larger treatment effects (HbA1c >1%). Nonetheless, these findings are consistent with clinically relevant improvements, particularly with respect to patients with T2D [37]. Based on the exploratory subgroup analyses of two reviews [29, 32], mean reductions in HbA1c appear to be more pronounced in interventions that combine text-messaging with Internet functionalities for self-monitoring of blood glucose and delivery of clinical feedback, including health education. Stand-alone text-messaging interventions (single media approach), although effective, have a smaller effect on HbA1c by as much as 0.4%. Also, studies with daily intervention frequency and interactive communication between patients and providers reported greater reductions in HbA1c than those with weekly intervention frequency or unidirectional data collection approaches [29, 32]. However, these findings are exploratory in nature and therefore, should be interpreted with caution. At best, they should be tested as new hypotheses in future trials. Limitations in the quality of evidence such as risk of bias in the clinical trials included in the SRs, statistical heterogeneity between study results in terms of magnitude of effects, and evidence of publication bias, suggest that future research is very likely to have an important impact on our confidence and change these estimates of effect. The presence of publication bias in the available evidence should be further explored in future SRs. The reviews that found evidence of publication bias on the basis of statistical tests and visual inspection of funnel plots did not search for eligible studies in the grey literature [29, 32, 34]. Hence, the possibility that publication bias may have been an artefact of the search limitations of the reviews rather than non-publication of small studies with small or negative effects cannot be excluded.

Patients with T1D represent a unique population with challenging circumstances. Studies including adolescents and young adults with T1D frequently have higher HbA1c levels overall due to hormonal fluctuations, difficulty with increasing demands of diabetes self-care, resistance to parental and health care provider involvement, competing priorities with school or work, and personal life/social stressors [38]. Under these circumstances, mHealth interventions may have limited impact on glycemic control, similar to other traditional behavioral interventions studied in the past over a longer period of time [37]. However, as younger age groups demonstrate higher utilization of mobile devices for most communications, it is compelling for researchers to further investigate the impact of real-time, multi-media approaches to diabetes self-management. Strategies such as gamification, decision aids, and social media among others should systematically be explored in future trials, beyond efforts to simply increasing self-monitoring and transmitting monitoring results [11]. Future studies should also explore the use and impact of diabetes mobile apps that are publicly available through the Apple and Google Play stores. Most trials included in the SRs involved proprietary applications that are not available to the general public. However, as the rates of smartphone ownership and availability of diabetes self-management apps increase [39, 40], it is important to investigate the safety and clinical effectiveness of these apps.

Prior research has shown that when mHealth interventions are designed with a theoretical basis and the content of the intervention is dynamic and adaptive to the needs of each individual, the outcomes are more likely to be successful [22,41]. However, the findings of this review indicate that there is insufficient evidence to determine whether mHealth interventions that are adaptive in nature or based on behavioral change theories are more effective than others. Only few primary studies included in the SRs specified a theoretical basis for the intervention and none of the SRs attempted to explore the impact of theory-based methods and interventions on clinical outcomes. It is not clear whether this is an issue of underreporting in the original studies or lack of theoretical basis due to well-established practice guidelines for managing T1D and T2D [41]. Future empirical studies should improve clarity of reporting with respect to the use of behavioral change theories or models, and other relevant intervention details (e.g. use of adaptive, interactive, or dynamic interventions). Subsequently, SRs should attempt to examine and synthesize evidence on factors related to behavioral change, attitude towards technology and perceived value in order to inform the design of future interventions and optimize their success. In order to leverage mHealth technologies effectively, it is important to understand the mechanisms of behavioral change and factors that affect the adoption of these technologies by patients. The use or lack of theoretical models may be used to better understand and explain variations in the effectiveness of mHealth interventions for diabetes across different age groups (heterogeneity of treatment effects). It is also important to further investigate and build evidence on the optimal type of mHealth interventions (i.e. “modality” of SMS, reminders, education etc.) that would be most effective across different types of patients, with various characteristics. Although mHealth interventions may increase the frequency of contact between patients and health care providers, a more thorough assessment of the behavioral change among providers and the impacts of these interventions on the process of care should also be closely examined.

There is also necessity for further development of research that investigates the impact of mHealth interventions on the utilization of health care services (e.g., emergency room visits and hospitalizations). With the continuous increase in health care costs and focus on quality, several health care systems face the challenge of caring for a constantly growing number of diabetes patients at minimal cost. As a result, a shift of patient care away from health care organizations is necessary to reduce congestion in these settings and to diminish costs. Alike home telemonitoring programs, mHealth represents a promising approach for achieving these objectives. Yet, systematic evaluation of its effects on the consumption of conventional health care services has not been conducted so far to support its wide diffusion. Studies included in the SRs provided little information about the cost of the interventions and did not include cost-effectiveness evaluations, preventing managers and policy makers from confirming their economic viability [20, 22, 29–31]. Importantly, no observations were made in relation to the effects of mHealth on health care providers, their acceptance of this approach, and their concerns about it, which are important issues to consider in future studies [26]. Further, a comparison between the time spent by health care providers delivering individual feedback and tailored advice via email or SMS and the time spent by them otherwise caring for exacerbated cases and complications that could have been minimized by the use of mHealth interventions is worth examining to have a clearer idea of the actual long-term and overall effects of this approach on clinicians’ workloads. The treatment effects found in SRs with mHealth interventions would be important if they could be achieved and sustained across the general population via text messaging and Internet technologies at very low cost, but far from cost-effective if they required significant support from nurses and other HCP and/or additional drugs [30]. When considering the abovementioned issues, future randomized controlled trials should consider adequately-powered samples of patients and be conducted over longer periods of time to be able to draw firm conclusions regarding both the perceived and objective impacts of mHealth interventions on patients with diabetes [26].

Supporting information

Acknowledgments

The authors thank Carla DeSisto and Dan Zhao for their assistance with data extraction and organization as well as Kevin Brennan, Molly Huttner, and Ashley Ulm for their assistance with the production and refinement of the figures.