Background

Physical activity (PA) is a major protective factor for most noncommunicable diseases [1, 2] and it is generally accepted that most populations in the developed world are insufficiently active.[3] However, even within a given population both estimates of PA levels [4] and changes over time [5, 6, 7] vary and thus interventions are difficult to design.

Because accelerometry is not vulnerable to reporting bias as are self-reports, it is an increasingly popular technique for assessing PA under field conditions. Accelerometers measure accelerations, which are then converted to unitless counts which are summed per unit time (epoch). Activity intensity (sedentary, light, moderate, or vigorous) in each epoch is defined by applying cutpoints that have been observed, in calibration studies, to best indicate metabolic demands [8, 9] in subjects of the appropriate age. However, these cutpoints vary substantially [4, 9, 10, 11, 12, 13] and thus accelerometrically-estimated PA in the same or similar population varies [4] based on the calibration study chosen. Furthermore, since different cutpoints were calibrated for different activities, the magnitude of differences between them may also vary depending on the pattern of body movement most common in the population. [14] This is especially true if one cutpoint measures only the vertical axis (uniaxial, as is typical of older studies) and the other measures all three axes (triaxial.) Thus accelerometric studies are difficult to compare to each other, and establishment of longitudinal, population-level differences in accelerometrically-estimated PA may be impossible.

Comparisons of different accelerometric protocols (cutpoints, weartime, epoch length) and device types have become increasingly common as accelerometry moves into the mainstream of research. [12, 14, 15, 16, 17, 18] These comparisons find large differences in estimated activity, but have not established the stability of these differences: for example, while Banda et al [15] found that the cutpoints of Evenson estimated 92% more MVPA than those of Treuth, they did not establish whether the difference is similar in other populations. If it is (i.e. the difference is stable) interconversion of activity estimates may be as simple as multiplication or division by a constant: while if it is not, interconversion may be impossible.

This study specifically addresses these concerns by applying three sets of cutpoints to the same dataset with the same data-cleaning protocol and comparing their estimates of activity in each intensity, quantified both as minute-by-minute agreement and as estimated total minutes. One set of cutpoints, created by Freedson et al, [9] is among the most common for youth; thus we apply it as a standard against which others should be compared. The other two (one uniaxial, one triaxial) were calibrated to a single test dataset by Romanzini et al[8] and thus agreed perfectly on activity intensity within it: thus any differences we observe appear to be specific to our population.

Methods

Results

Daily activity for 1402 Germans (mean age 15.6 years, 46% male) was accelerometrically monitored over 4–7 days per subject for an average of 14.7 hours per day. (Table 1) Detailed population characteristics and activity levels according to Freedson’s cutpoints have been previously published.[25] [31] All three sets of cutpoints agreed that subjects spent about 2/3 of time sedentary (range: 67–74%); and about 5% of time in MVPA (range: 4.06–5.67) with the remainder in light activity (LPA.) (Table 1, Table 2, S1 Table and S2 Table). However, estimated MPA and VPA differed by over 50% between cutpoints and all pairwise comparisons between cutpoints’ estimates of activity minutes of each intensity were significant at p = 0.01. All but one (MPA, Romanzini triaxial vs Freedson) were also significant at p<0.0001. (Table 1.)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Population characteristics.

Mean (standard deviation) unless otherwise stated.

https://doi.org/10.1371/journal.pone.0187706.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Agreement of cutpoints on activity intensity, minute by minute.

Percent of time (total 14.7 hours / day, 8780 days).

https://doi.org/10.1371/journal.pone.0187706.t002

The three cutpoints agreed well on total estimated sedentary behaviour (largest difference 9%, Table 1), but disagreed more, in relative terms, for activity of higher intensity. The largest pairwise differences were 40% for LPA, 102% for MPA, 88% for VPA, and 40% for MVPA. Romanzini’s triaxial cutpoints estimated the most sedentary behaviour, the least LPA, and the highest MVPA and VPA; their estimated MPA was close to that of Freedson. Romanzini’s triaxial cutpoints on average estimated 27.7 (SD = 13) minutes MPA and 23.7 (19) minutes VPA per day, or 51.4 minutes MVPA: 41% more than the uniaxial cutpoints calibrated to the same reference population which estimated 14.2 (SD 7.6) minutes MPA per day and 22.5 (16) minutes VPA. For Freedson the corresponding numbers were 28.7 (15) and 12.6 (12) minutes MPA and VPA: twice as much MPA and just over half as much VPA, with total MVPA 36.7 and 41.3 minutes respectively.

Table 2 compares estimated activity intensity minute-by-minute for Romanzini’s two sets of cutpoints. Freedson’s cutpoints are compared to both of Romanzini’s in S1 Table and S2 Table. Although for most minutes the two sets of cutpoints agree on activity intensity, some scatter is visible: in particular the triaxial cutpoints estimate much higher activity than the uniaxial ones for a few minutes. The uniaxial cutpoints estimated that 73% of time was sedentary, 23% in LPA, 1.6% in MPA, and 2.5% in VPA: corresponding numbers for triaxial were 74%, 21%, 3.1%, and 2.6%. Of all the minutes classified by triaxial accelerometry as MPA, a few were classified by the uniaxial cutpoints as sedentary; almost half as LPA; and 13% as VPA. Of all the minutes classified by uniaxial accelerometry to be MPA, the triaxial signal indicated that none were sedentary, 4% were LPA, 6.6% were MPA, and 23% were vigorous.

In addition to the above-described fixed bias, there was also proportional bias for most activity intensities. In Bland-Altman plots (Fig 1, “Activity minutes by two sets of cutpoints“) the difference between estimated minutes in each activity intensity, according to Romanzini’s uni- and triaxial cutpoints, is plotted against the mean of the two for each recording day. The thin solid line (“Mean difference”) shows the average difference for all recording days; the dotted black line (“Linear trend”) shows how this difference varies as a function of activity level (amount of time in that intensity.) Proportional bias is obvious for the higher activity intensities: as more time is spent in each intensity, the mean difference between the two algorithms gets further from zero. For LPA and MVPA, the difference becomes more and more negative; while for MPA and VPA, the difference becomes more and more positive. Only sedentary time had little proportional bias, and still estimated sedentary time varied widely between cutpoints: 5^th and 95^th percentiles of the difference were -23 and 314 minutes per day.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Large differences in estimated activity intensity between uni- and triaxial accelerometric cutpoints originally calibrated to the same dataset.

Romanzini’s uni- and triaxial cutpoints from Romanzini M; Petroski, EL; Ohara, D; Dourado, AC; Reichert, FF. Calibration of ActiGraph GT3X, Actical and RT3 accelerometers in adolescents. European Journal of Sport Science. 2014;14(1):91–9. doi: 10.1080/17461391.2012.732614.

https://doi.org/10.1371/journal.pone.0187706.g001

Discussion

Although accelerometry is an objective measure of acceleration, it is not an objective measure of either physical activity or energy expenditure.[32] Furthermore, differences in data handling have potential to affect accelerometric estimates of activity above and beyond limitations of the method itself. In this large study of adolescents, we empirically confirm [4, 17, 26] large differences in accelerometrically-estimated physical activity depending on the cutpoints chosen. We add that accelerometric calibration algorithms which agree on one dataset may still disagree on another, and thus population-specific features may contribute to differences in estimated PA between algorithms. Converting estimated activity according to one set of cutpoints into another appears to be impossible without access to the raw data: since two sets of cutpoints can agree on one dataset and disagree on another, simple multiplication or division of estimates is not sufficient.

Ours is not the first study to find large differences in estimated accelerometric physical activity depending on data handling. [4, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18] However, we are among the first to prove the effect of cutpoints alone and to prove that differences in estimated activity level by cutpoint are themselves not fixed. While a recent review [4] found even larger differences than we did, these are from different studies and thus could conceivably be attributed to weartime protocols, device type, or even population differences. The same is true for most studies which examine the effects of data handling, in which weartime protocols, epoch length, and cutpoints may all simultaneously vary.

While it is often suggested [10] that triaxial accelerometry differs significantly from uniaxial for capturing activities of daily living, comparatively few studies (with the exception of Vanhelst) [14] have explicitly compared the two as done in our study. Vanhelst found close agreement between uni- and triaxial accelerometric estimates of moderate and vigorous physical activity. In contrast, we found that relative differences were larger for high activity intensities and smaller for sedentary behavior. Vanhelst’s population and data-handling protocol were similar to ours, including epoch length and device weartime: however, the biggest differences are likely to be in the cutpoints chosen. Theirs appeared to be from two different studies, which in turn were almost certainly calibrated to different activities: the uniaxial cutpoints between activity levels were often higher than the triaxial ones. [14] In contrast, Romanzini’s uni- and triaxial cutoffs are calibrated to the same test dataset and thus triaxial counts cannot be less than uniaxial. However, we concur with Vanhelst that there was no large difference between uni- and triaxial estimates of activity in daily living: indeed, in our study two different uniaxial sets of cutpoints were about as similar as each was to a set of triaxial cutpoints. It is plausible that differences are larger during activities with primarily non-vertical patterns of acceleration, such as certain sports: however sport was a relatively small contributor to total physical activity in this sample. [23] Thus we concur with the literature that triaxial accelerometry was generally comparable to uniaxial in monitoring activities of daily living: the effect of the additional two movement axes was small compared to that of cutpoints.

However, our study of the effects of data handling on activity estimates is not immune to the effects we describe. While we describe the decisions we made for each of many data-handling protocols, we recognize that other decisions would have been almost equally valid and consistent with the literature. For example, while a 60-second epoch length is the commonest in this age group [28] it is also the longest and longer than that used to validate two of the cutpoints[8]. This creates a discrepancy which may [28] contribute to systematic over- or underestimates of MPA and/or VPA in our study; however Banda et al found ([33]) that Romanzini’s cutpoints estimated similar levels (within 25%) of MVPA whether 15-second or 60-second epochs were used.

Similar caveats apply to decisions to accept data from a given subject or day of recording, and to criteria for identifying NWT. Since NWT is most likely to be confused with sedentary time, falsely identifying NWT as sedentary time may artificially inflate weartime, lead to the acceptance of time and days where in fact the device was not worn, and thus underestimate total daily activity. For this reason, our estimates of total physical activity should only be compared to other studies using similar devices and data-handling protocols.

Our findings of large and variable differences between accelerometric cutpoints add to the growing scientific consensus [4, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18] that accelerometry, as a method, cannot be viewed as objective if data handling is not standardized. The need for standardization is particularly acute as accelerometric research expands from the academic to the private sector. Data are collected with wearable devices such as smartphones, processed with apps, and combined in real time with data from other apps which already track sleep, detect sleep apnea, [34]or detect eating.[35] Activity trackers are easier to use than dedicated accelerometers, are much less expensive, and can even provide real-time feedback for desired or undesired behaviors, such as eating or physical activity. [36, 37] Traditional methods of accelerometry cannot compete with this rapid growth and diversification: scientific consensus cannot keep up with commercial innovation.

However, the flexibility of activity-tracking apps is also their greatest weakness: since their data collection and processing constantly changes they cannot be used to establish records of past physical activity [36] and thus they are unlikely to be useful for any attempt to establish population-level changes in activity pattern. In contrast, cutpoint-based accelerometry is replicable and consistent: it has existed for over thirty years [38] compared to ten for the iPhone, and data-handling protocols used in scientific studies are publicly available. Both academic and non-academic methods of accelerometric activity assessment have distinct strengths, which could be profitably combined to benefit both researchers and the general public.

Validation studies show close agreement between activity counts registered by Actigraph accelerometers and both smartphones[39, 40, 41] and other devices (e.g. Fitbit, Jawbone)[39] in activities of daily living, which suggests that the hardware of these devices is adequate to capture physical activity if the phone is carried in a pocket or the device is worn as directed. Thus apps could be designed to apply a validated data-handling protocol (standards for epoch length, detection of NWT, cutpoints between intensities, and total weartime) using the sensors in these devices. [39]Since data processing may not be of primary interest to physical-activity researchers, we believe that many would prefer to collect data with a peer-reviewed and validated app rather than dealing with an expensive dedicated device, proprietary software, and complex data-handling protocols which often offer no clear guidance. [28] Activity estimates based on a standard protocol could grow to play the same role in population-level activity assessment as does body mass index in estimating adiposity: as a replicable standard which, in spite of known limitations, provides useful estimates of population-level trends.

Use of a single standard protocol does not preclude improvements in methodology. Data processing and transmission are now inexpensive enough to allow multiple algorithms to be applied to the same dataset (as is done here and elsewhere[15]) and published either in the main text or online in a supplement: soon even raw data will be archivable and thus available for reprocessing with the newest algorithms. We encourage researchers (academic and private) to use up-to-date methods of handling and processing data: however we suggest that they do this in addition to, not instead of, creating replicable standards which ensure compatibility with past and future research.

Conclusions

In a single dataset, we found that adolescents’ estimated moderate and vigorous physical activity differed by up to 50% based on the cutpoints used for data handling. This difference was highly statistically significant and would be clinically relevant if it were real and not artefactual. Furthermore agreement was no better with a pair of algorithms calibrated to the same test dataset [8] than with two algorithms calibrated to different test datasets [8, 9]: thus differences appear to themselves vary with population. It may be impossible to convert activity estimates from one algorithm to another if raw data are not available, and this limits the utility of accelerometry as a research technique. Thus we suggest that academic accelerometric research be standardized, with all studies of each age group publishing estimates processed in a standard way in addition to more modern methods of data processing.

Supporting information