Introduction

The benefits to general health of physical activity are universally recognized [1–3]. Clear guidelines have been published by the US Surgeon General (1996), the Centers for Disease Control (CDC, 2008) and the World Health Organization (WHO, 2010). To maximize the health benefits of physical activity for the general population, these guidelines recommend that activities, such as brisk walking, be carried out daily [4–6]. An essential part of achieving a behavioral change in promoting physical activity is the capacity to easily monitor goal setting. The use of physical activity monitors has exploded in the last 5 years, both in the consumer market and increasingly as tools for clinical settings [7–9]. The most commonly utilised output parameter from these monitors is the cumulative step count over a day, week and beyond. In line with the WHO recommendations, many individuals aim to improve on their physical activity levels by reaching a personal goal or the recommended goal of 10,000 steps per day [4–6]. Utilizing an easily quantifiable output such as step count supports a person in achieving physical activity goals and maintaining a healthy lifestyle [10, 11]. Cumulative logging of step count can become a key factor in the promotion and maintenance of physical activity levels and mental wellbeing. As persons become increasingly dependent on the activity monitor to unobtrusively record their activity level, the question of the device’s sensitivity and specificity to the reported output becomes an increasingly important one.

We have previously examined the step count sensitivity of a number of physical activity-monitoring devices during walking over a variety of surface types while achieving the recommended daily step goal of 10,000 steps. We found all devices tested to be sensitive in step detection, with a mean absolute percentage error ranging from 1.36% to 4.61% for the four activity monitors tested [12]. However, while this is a very important factor to consider in assessing the performance of an activity monitor, it is equally critical that the activity-monitors are not just sensitive but also specific in their capacity to monitor actual steps performed. The ideal activity monitor should only record a step when a step is actually taken, while disregarding non-stepping body movements.

The literature is sparse when reporting on the specificity of physical activity monitors examined. Chen et al [13] investigated three consumer wristband activity monitors; Fitbit Flex, Garmin Vivofit and Jawbone UP during a range of normal daily activities, including playing a tablet computer game, folding laundry, pushing a stroller and carrying a bag. They reported substantial false step detection events during the task of folding laundry and reported the accuracy under each activity to range widely both between monitors and activities performed [13]. In a similar study, Sellers et al [14] investigated the ActivPAL3 monitor during a range of activities of daily living. These activities included hanging laundry out to dry, putting on a duvet cover and pillowcase, cleaning a framed picture, writing a letter/list and vacuuming. The authors reported step detection sensitivity of 76.1% for young people but only 40.4% for older adults during normal activities of daily living. While they reported the ActivPAL3 to accurately detect ‘purposeful stepping’, the detection of smaller stepping movements during the activities of daily living was reported as being poor [14]. These studies reveal a significant degree of variability in the step detection specificity during activities that are not solely purposeful stepping activities like walking. Normal activities of daily living include a range of stepping and non-stepping activities and also range from low to high levels of intensity. As step count is now a pervasive and easily understandable measure of physical activity it is essential that monitors be both sensitive and specific in their measures.

The aim of this paper is to assess five activity monitors in healthy adults: the Jawbone UP^™, Withings Smart Activity Monitor Tracker (Pulse O₂)^™, NL-2000 pedometer^™, ActivPAL micro^™ and Fitbit One^™ for their step detection specificity over a range of different physical activities that mimic typical activities of daily living of various intensity levels.

Methods

Participants

A pool of 37 participants (12 male) took part in the study involving the performance of 9 prescribed activities which may trigger false detection of stepping: (i) deskwork, (ii) taking an elevator, (iii) taking a bus journey, (iv) driving an automobile, (v) washing and drying dishes, (vi) functional reaching task, (vii) indoor cycling, (viii) outdoor cycling and (iv) indoor rowing. Participants had a mean age of 39 ± 13.87 years and a mean body mass index (BMI) of 25 ± 3.79 kg/m². A random sample of ten participants carried out each activity. The mean ages and BMIs of the ten participants that carried out each activity are given in Table 1. None of the participants had any history of cardiovascular disease or neurological disorders. Ethics committee approval (reference number CA-1069) was obtained from the Galway University Hospitals Research Ethics Committee for this study and all participants provided written, informed consent.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Demographic details for the ten participants that carried out each specificity activity.

https://doi.org/10.1371/journal.pone.0169616.t001

Study protocol

All activity monitors were positioned as per manufacturer’s recommended body locations (Fig 1). Tegaderm transparent dressing (3M Health Care, Minnesota, USA) was used to affix the ActivPAL^™ to the mid-anterior aspect of the thigh, with the Jawbone UP^™ worn on the wrist, the Withings Smart Activity Monitor Tracker (Pulse O₂)^™ and NL-2000 pedometer^™ worn on opposite hips and the Fitbit One^™ clipped on at the level of the chest.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Positioning of each activity monitor.

The ActivPAL^™ is worn on the right thigh, the NL-2000^™ on the left hip, the Withings^™ on the right hip, the Jawbone^™ on the right wrist and the Fitbit^™ at the level of the chest.

https://doi.org/10.1371/journal.pone.0169616.g001

Participants wore the activity monitors simultaneously for the duration of each test condition. For the NL-2000^™, Withings^™ and Fitbit^™ monitors the recorded step count was obtained directly from the device screen interface. To extract the step count data from the Jawbone^™, it was synced to a smartphone via Bluetooth and step count was obtained from the Jawbone^™ smartphone application. To extract the step count data from the ActivPAL^™, it was manually connected to a PC and the data downloaded using the ActivPAL^™ software program, which outputs the step count data.

To ensure the step count was accurate for the activity only, and not contaminated by activity just before or after the prescribed activity, specific procedures were put in place for each monitor. For the NL-2000^™, Withings^™ and Fitbit^™ monitors, the step counts displayed on the device interfaces were noted immediately before and immediately after each activity. For the Jawbone^™, it was synced both immediately before and immediately after each individual activity. The ActivPAL^™ device was programmed before each individual activity, a procedure that cleared all previous data. Additionally, to prevent capturing data pre- or post-activity, all participants were instructed to be/stand still before commencing the activity and to be/stand still once the activity was completed. Participants were also video recorded with a hand-held camcorder during each activity excepting automobile driving and the bus journey when participants were seated for the duration of testing. The timing for each activity, excepting the automobile driving and bus journey, was started at the same time as the video began recording. The step count obtained from each activity monitor was compared to the step count obtained from the recorded video.

Results

All monitors tested recorded steps when no steps actually took place (false positives) to a greater or lesser extent depending on the activity being performed. With the exception of the Jawbone^™ device, all other monitors generated most false positives during active non-stepping activities (Fig 2). The Jawbone^™ registered its greatest number of false positives during moderate non-stepping activities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overall specificity (fp/min) of activity monitors during the moderate and active non-stepping activities.

Fp/min = false positives per minute. N = 10/activity. Data represents mean ± SEM.

https://doi.org/10.1371/journal.pone.0169616.g002

ActivPAL^™ activity monitor

The ActivPAL^™ activity monitor was found to be specific for step detection during all but one of the passive and moderate non-stepping activities (Figs 3 and 4). For the functional reaching task the ActivPAL^™ registered a non-significant number of false positives. The ActivPAL^™ correctly (no false positives registered), did not record any steps during the other activities: deskwork, taking an elevator ride, taking a bus journey, automobile driving or washing and drying dishes (Table 2). This was also true during the active non-stepping activity of indoor rowing, whereby the ActivPAL^™ correctly did not register any false positives. However, the ActivPAL^™ was found to be less specific for step detection during the cycling activities. The ActivPAL^™ registered 97±12 fp/min during the indoor cycling activity and 90±26 fp/min during the outdoor cycling activity (Fig 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Specificity of each activity monitor during the passive non-stepping activities.

Data for deskwork, taking an elevator and automobile driving on a motorway have been excluded due to the high number of zero false positives correctly recorded by the devices during these activities. The mean false positives detected during the bus journey are expressed as false positives detected per minute as represented by the left y-axis. The mean false positives detected during the driving activity are expressed as false positives detected per kilometer driven as represented by the right y-axis. * P ≤ 0.001 vs. zero false positives, # P < 0.05 vs. zero false positives. N = 10/activity. Data represents mean ± SEM.

https://doi.org/10.1371/journal.pone.0169616.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Specificity (fp/min) of each activity monitor during the moderate non-stepping activities.

While the ActivPAL^™ registered a number of fp/min during the functional reaching task, this was found to be non-significant. *** P < 0.001 vs. video recording (zero fp/min), ** P < 0.01 vs. video recording (zero fp/min). N = 10/activity. Data represents mean ± SEM.

https://doi.org/10.1371/journal.pone.0169616.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Specificity (fp/min) of each activity monitor during the active non-stepping activities.

*** P < 0.001 vs. video recording (zero fp/min), * P < 0.05 vs. video recording (zero fp/min). N = 10/activity. Data represents mean ± SEM.

https://doi.org/10.1371/journal.pone.0169616.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Mean false positives per minute for all activity monitors during the non-stepping prescribed activities.

https://doi.org/10.1371/journal.pone.0169616.t002

Discussion

This study assessed five activity monitors: the Jawbone UP^™, Withings Smart Activity Monitor Tracker (Pulse O₂)^™, NL-2000 pedometer^™, ActivPAL micro^™ and Fitbit One^™ for their specificity for step detection during a series of non-stepping activities, representing examples of typical activities of normal daily living.

While there are many possible outputs from most activity monitors, such as time spent active, distance moved and calories burned, step count remains one of the most popular and translatable outputs in use today. Many research studies and consumer electronic articles have focused on step count as an output for measuring physical activity [11, 15, 16]. Many national and international health organizations have provided recommendations on step count goals in line with improving population health. The current recognized recommendation is 10,000 steps per day, which is a convenient method of physical activity quantification for the general public to comprehend. As many activities of daily living involve both stepping and non-stepping movements, it is important that activity monitors are both sensitive and specific in their detection of their primary output, i.e. actual steps taken. Previously, we have shown a number of physical activity monitors to be sensitive in step detection when actively walking over a variety of surfaces and in different types of footwear, while reaching the recommended daily 10,000 step goal [12]. However, our previous work focused on the performance of the activity monitors during purposeful stepping only. A critical question for activity monitors is, are steps recorded when no steps are actually taken? Considering this issue, in our current study we tested the specificity of activity monitors in distinguishing stepping from non-stepping movements during a number of prescribed activities typical of activities of daily living. It is important to gain an understanding of the true specificity of activity-monitoring devices in situations reflective of normal everyday activities.

In this paper we report on specificity as the number of false positives registered per minute for each activity. Our results demonstrate that, when grouped by activity type, the majority of the activity monitors tested were least specific during active non-stepping activities, which involved indoor cycling, outdoor cycling and indoor rowing. As these activities involve a significant amount of non-stepping body movements, and considering the typical movements involved in these activities, it is not entirely surprising that a number of false positive were registered specifically during these activities. It is interesting to note however, that during the indoor activities on a static rowing or cycling ergometer, there was less variation in the false positive count compared to outdoor cycling. During outdoor cycling, all activity monitors registered a significant number of false positives.

While the ActivPAL^™ monitor was found to be specific during passive and moderate non-stepping activities, a significant number of false positives were registered during the active cycling activities. This is likely due to the continuous changing of the leg position from the horizontal to the vertical plane and thus the location of the ActivPAL^™ on the anterior aspect of the thigh has a role to play in these results. In a similar manner, the Jawbone^™ activity monitor registered the greatest number of false positives during the moderate non-stepping activities of washing and drying dishes and the functional reaching task. This may reflect the fact that the Jawbone^™ is worn on the wrist. Both moderate non-stepping activities performed focused on upper body movement involving both the arms and hands, which mostly likely played a role in the significant number of false positives registered by the Jawbone^™. Additionally, the Jawbone^™ registered a significant number of false positives during automobile driving on a local road. The local road, through a built-up area, consisted of a number of speed bumps and required regular gear changes. As the Jawbone^™ recommended wearing location is the non-dominant wrist, the majority of participants wore the Jawbone^™ on the left wrist throughout this study, which is the hand involved in gear changing in the manual car used in this work. This, at least in part, may explain why the Jawbone^™ registered a significant number of false positives during this section of automobile driving. It could be interesting to assess different placements of the Jawbone^™ while driving. The Jawbone^™ was not the only activity monitor found to register a significant number of false positives during automobile driving however. The NL-2000^™ registered a significant number of false positives on a local road, an urban road and a category C road, while the Fitbit^™ also registered a significant number of false positives on a category C road in addition to on a category B road. These monitors also registered a significant number of false positives during the bus journey. As the NL-2000^™ monitor was worn at the waist and on the left had side of the body in this study, which is the clutching side in a manual car, this may have affected the false positives recorded. The Fitbit^™ however, was worn at the level of the chest, and as with the bus journey, it is likely that surface type, i.e. the bumpiness of the road, played a role in the false positives observed in that case.

These findings are relevant to all users of physical activity monitors from the typical sedentary user to the most active of users. The sensitivity and specificity of these devices are important considerations for both the general public interested in tracking step count for personal achievement, but also special populations tracking activity as part of a lifestyle prescription. Increasingly, wearable activity monitors are part of a suite of tools employed in tackling conditions such as obesity, COPD and type-2 diabetes. In these instances, the activity monitors play a critical role in monitoring progress towards improving physical activity levels and goal setting as part of behavioral change lifestyle interventions.

By way of example we can take four individuals representing a spread of activity levels from sedentary (taxi driver), low active (office worker), somewhat active (school teacher) to highly active (student football player). Based on these classifications individuals would be expected to achieve less than 5000 steps, 6250 steps, 8750 steps or greater than 12500 steps per day on average respectively [11]. The possible effect of false positives generated by each monitor on daily step count is presented in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Estimated false positives generated daily based on conservative estimates of time spent on sample activities for different categories of potential activity monitor users over a typical 12-hour day.

https://doi.org/10.1371/journal.pone.0169616.t003

As is evident from Table 3, across a spectrum of users from sedentary to highly active, all monitors have the capacity to contribute false positives to daily step counts. A recent survey published by the National Transport Authority of Ireland reported that taxi drivers drive a minimum of 4 and a maximum of 5 hours per day. Within this timeframe taxi drivers drive up to 14.9 kilometers per hiring cycle, with hiring cycles lasting anywhere up to 59 minutes [17]. Based on our conservative estimates in Table 3, the Jawbone^™ and NL-2000^™ can generate false positives accounting for up to 25.1% and 10.7% of the daily average of 5000 steps for a sedentary Taxi driver [11]. Had the Taxi driver been using a Withings^™ or Fitbit^™ device in our example then the false positives could be significantly less.

In a typical day in the life of an office worker, washing and drying dishes, carrying out several hours of deskwork, stacking books, taking the elevator several times throughout the day, taking a bus and short car journey to and from work is not uncommon. Again, based on our estimates, the Jawbone^™ monitor can generate false positives accounting for up to 22.5% of the daily average of 6250 steps for a low active office worker. Using a Withings^™, NL2000^™ or Fitbit^™ could have resulted in a much lower contribution of false positives at 2.4, 3.8 and 9.6% of the daily average respectively. A similar pattern is apparent for a typical active user such as a schoolteacher, while a highly active user such as a student football player could equally be exposed to the accumulation of false positive step counts.

While a typical day in the life of the average activity monitor user will be highly variable and could consist of many different stepping and non-stepping activities, in this paper we have highlighted the effect of some normal activities of daily living on step count. Our data, and when considered for typical users in Table 3, serve to highlight the potential for false positive steps to be registered during a normal day consisting of some passive, moderate and active non-stepping activities. In a similar study carried out by Stackpool et al [18] participants wore the Nike Fuelband, Fitbit Ultra, Jawbone UP and NL-2000i pedometer when utilising an elliptical cross-trainer and carrying out agility exercises in a gymnasium, including agility ladder drills (seven different moves), 10 basketball free throws and a basketball half-court lay-up drill for one minute. The authors reported inaccuracies in the activity monitors during the agility tests, with all monitors excepting the Jawbone UP significantly underestimating steps [18]. The Nike Fuelband was found to underestimate steps by 34%, the Fitbit Ultra by 20% and the NL-2000i by 17%. During the elliptical exercise the only significant difference lay with the NL-2000i, which underestimated steps by 6% [18]. Although these monitors underestimated step counts during these activities and our results highlight the potential for the registering of false positive steps, both studies highlight the variability that can occur during physical activities that are not specifically stepping activities.

Overall, this data is important to consider when utilising activity monitors in both the research and clinical domains whether using research grade or off-the-shelf consumer-based tracking tools. While the ActivPAL^™ is a research-grade activity monitor and the other activity monitors tested here are consumer-grade monitors, the specificity was observed to vary for all monitors during one or more non-stepping activity. However, all activities in this study are classed as physical activity and go towards maintaining a healthy and active lifestyle. Accurate translation of the activity carried out during these non-stepping activities into step count could be of interest. Clearly it is important to consider non-stepping activities when wearing physical activity monitors to accurately quantify overall physical activity levels. It is evident from our data that typical non-stepping activities, likely to be carried out in everyday life, result in the recording of steps when none are taken. As mentioned above, these factors are especially important to consider in situations whereby physical activity is prescribed as an intervention in the prevention or treatment of chronic disease, where accurate physical activity quantification is of the utmost importance.

Conclusions

This study provides the first comprehensive specificity evaluation of five activity monitors during a variety of prescribed physical activities involving non-stepping body movements. From our results, we can conclude that false positive steps are detected and registered during a range of non-stepping activities, affecting the quantification of physical activity with regard to step count as the primary output. The range and impact of the false positives detected will differ depending on the activities carried out and the activity monitors utilised, with the results highlighting the specificity variability that can exist during a range of non-stepping physical activities. The Withings^™ monitor performed best across all non-stepping activities, registering a significant number of fp/min only during the outdoor cycling activity. The specificity of an activity monitor needs to be considered carefully when choosing an activity monitor to accurately quantify the variety of physical activities that can be carried out on any typical day.