Figures
Abstract
Introduction
To determine time spent at different physical activity intensities, accelerometers need calibration. The aim of this study was to develop and cross-validate intensity thresholds for the Actigraph GT3X+ accelerometer for wrist and hip placement in four-year-old children.
Methods
In total 30 children (49 months, SD 3.7) were recruited from five preschools in Stockholm. Equipped with an accelerometer on the wrist and another on the hip, children performed three indoor activities and one free-play session while being video recorded. Subsequently, physical activity intensity levels were coded every 5th second according to the Children’s Activity Rating Scale. Receiver Operating Characteristic (ROC) curves was used to develop wrist and hip intensity thresholds, the upper threshold for sedentary, and lower threshold for moderate-to-vigorous physical activity (MVPA), for the vertical axis (VA) and for the vector magnitude (VM). A leave-one-out method was used to cross-validate the thresholds.
Results
Intensity thresholds for wrist placement were ≤ 178 (VA) and ≤ 328 (VM) for sedentary and ≥ 871 (VA) and ≥ 1393 (VM) counts/5 seconds for MVPA. The corresponding thresholds for hip placement were ≤ 43 (VA) and ≤ 105 (VM) for sedentary and ≥ 290 (VA) and ≥ 512 (VM) for MVPA. The quadratic weighted Kappa was 0.92 (95% CI 0.91–0.93) (VA) and 0.95 (95% CI 0.94–0.96) (VM) for the wrist-worn accelerometer and 0.76 (98% CI 0.74–0.77) and 0.86 (95% CI 0.85–0.87) for the hip-worn.
Citation: Johansson E, Larisch L-M, Marcus C, Hagströmer M (2016) Calibration and Validation of a Wrist- and Hip-Worn Actigraph Accelerometer in 4-Year-Old Children. PLoS ONE 11(9): e0162436. https://doi.org/10.1371/journal.pone.0162436
Editor: Karen Hind, Leeds Beckett University, UNITED KINGDOM
Received: April 19, 2016; Accepted: August 23, 2016; Published: September 12, 2016
Copyright: © 2016 Johansson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Due to ethical restrictions, the data will not be shared publicly, but an ethically compliant data set will be made available to interested researchers on request. Data underlying the results described in our manuscript are available from: Elin Johansson, Division of Physiotherapy, Department of Neurobiology, Care Science and Society, Karolinska Institutet, Alfred Nobels Allé 23, 141 83 Huddinge, Sweden. E-mail: elin.johansson@ki.se.
Funding: This study was funded by The Swedish Council for Working Life and Social Research; The Swedish Research Council and Karolinska Institutet. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Physical activity is associated with a broad range of health indicators and is probably already of importance during the preschool years [1]. An accurate method to measure physical activity and sedentary time is essential to study correlates of these behaviors and to evaluate interventions, both in research and in clinical settings. Accelerometers have revolutionized the research field by enabling objective measures of physical activity intensity, frequency and duration, which are possible to use in large cohorts [2]. Objective methods are of particular importance for preschool children who can neither recall nor estimate past physical activities, and whose intermittent activity patterns make proxy reports very difficult [3]. Accelerometers have been proven feasible, reliable and valid for assessing physical activity in children [4, 5]. Accelerometers can be worn on different body sites. The hip has been the traditional location, but during recent years the wrist has been increasingly used as it has been shown to yield superior compliance in children, compared with hip placement [6]. Accelerometer output, expressed as counts, is affected by the body part it is attached to. As an example, an accelerometer placed on the hip does not capture arm movements. However, the estimated time in different intensities is similar when site-specific cut-points are applied [7]. To determine the time spent at different intensity levels, accelerometer counts need to be calibrated against a reference method. In children, observational methods such as The Children’s Activity Rating Scale (CARS) [8] have been suggested as the best reference method and CARS has been used in many previous calibration studies [5, 9, 10]. Taken the large difference in children’s motor skills between the age of one and six year there is a need for age-specific cut-points. Calibration studies for wrist-placement have been performed in two-year-olds [5] and in 8–12 year olds [11] but cut-points for children aged 3–5 are lacking.
The aim was to calibrate the accelerometer Actigraph GT3X+ for non-dominant wrist and hip placement in four-year-old children by developing upper vertical axis (VA), and vector magnitude (VM), intensity thresholds for sedentary behavior and lower VA and VM intensity thresholds for moderate-to-vigorous intensity physical activity (MVPA). Further aims were to cross-validate the developed intensity thresholds and to test the sensitivity and specificity of two existing cut points for hip-worn accelerometers.
Method
Subjects
A feasibility sample of preschools (n = 5) located in the inner-city and suburbs of the Stockholm area were invited to participate. The Head of each preschool received oral and written information about the study and was contacted again one week later. All invited preschools agreed to participate. In total 30 children, aged four years +/- 6 months, were included after their parents had given written informed consent. No child had any impairment that affected their participation in physical activity. The study was approved by the Regional Ethical Review Board in Stockholm County, Dnr 2009/217-31/2.
Procedures
The calibration took place at each preschool in the morning hours between 8–11 am. While being barefoot, children’s weight was measured with an electronic scale (Tanita HD-316, Tanita Corp.; Tokyo, Japan) and height with a portable stadiometer. Body mass index (BMI) was calculated and children were classified as either normal weight or overweight [12]. Each child was equipped with two Actigraph GT3X+ accelerometers, one with a nylon wrist band on the non-dominant wrist [5, 11], determined by the preschool staff, and the other attached with an elastic band on the left hip. A maximum of six children at a time performed three indoor activities (sitting watching a cartoon, sitting drawing, dancing to music) and one free-play outdoor session. The activities chosen were considered representative of 4-year old children and were selected after discussions with staff at one preschool. Each indoor session was performed for 4–6 minutes and the outdoor session lasted for about 15 minutes. Each session was video recorded (Canon Inc., Tokyo, Japan). Subsequently, physical activity intensity levels for each child were coded according to CARS [8], every 5th second by watching the videos. The scoring was performed by one researcher who had received several hours of training prior to the scoring session. In case of uncertainty the scoring researcher had the possibility to discuss together with another researcher with experience of using CARS. By video recording the computer clock on that same computer that was used for initializing the accelerometers in the beginning of each session, accelerometer data and CARS score could subsequently be synchronized.
Accelerometry
The Actigraph GT3X+ accelerometer (Actigraph LLC, Pensacola, FL, USA) is a small (45x33x15 mm), lightweight (19 g) monitor. It is water resistant and can be worn on various body sites including the wrist, hip and ankle. Data is sampled in a pre-determined interval and summarized over a so-called epoch. In this study, accelerometer data was sampled at 30 Hz and summarized over 5-second epochs. A 5-second epoch has been recommended to be used in order to capture the short bursts of MVPA that characterize preschool children’s physical activity [3]. The Actigraph GT3X+ accelerometer is three-axial, meaning that accelerations are captured in three directions. The VM is a combined measure of the three axes, defined as . Whether the VA or the VM gives the most accurate measure of physical activity remains the subject of ongoing debate [13].
CARS
CARS is an observational method that has been used in its original form, as well as in modified versions, in previous calibration studies and deemed appropriate for this study [5, 9, 14]. The method allows an observer to score the physical activity intensity for a child on a scale from 1–5. Level 1 represents only minor movements; Level 2 standing, moving arms; Level 3 walking slowly; Level 4 walking briskly; and Level 5 represents running. A detailed manual is available [8].
Data analysis
A CARS score of 1 was considered sedentary, CARS scores 2 and 3 were considered light intensity and 4 and 5 a proxy for MVPA [5, 8, 15]. Receivers Operating Characteristic (ROC) curves were used to develop wrist and hip upper intensity threshold for sedentary and lower thresholds for MVPA, for the VA and for the VM. Using ROC curve analysis, accelerometer count thresholds are chosen based on sensitivity and specificity and it is a method commonly used in calibration studies [5, 16]. When using ROC curve analysis, a measure of the area under the curve (AUC), representing the classification accuracy is provided. An AUC of under 0.70 is considered as poor, 0.70–0.79 fair, 0.80–0.89 good, and at least 0.90 excellent [17]. A leave-one-out method was used to cross-validate the derived intensity thresholds. Data from all of the participants except one, which was “held out”, was used as the test dataset. The process was repeated until all participants had served as the test data set and the accuracy results were averaged. ROC analysis was conducted on n-1 participants, followed by determination of agreement between the numbers of 5-second epochs at each intensity by developed cut points. This was compared with the number of 5-second epochs in each CARS score of the left-out child. This was repeated throughout the total sample. Absolute agreement and quadratic weighting Kappa was calculated [18]. The previously developed MVPA intensity thresholds for children aged 3–5 by Pate et al. [19] and Sirard et al. [10] were compared against the output from our VA ROC curves for the hip in terms of sensitivity and specificity. Since a 15-second epoch length was used in these studies, these count thresholds were divided by three before being applied to the output from the ROC curves. The MVPA intensity threshold was then ≥140 counts/5 seconds and ≥271 counts/5 seconds for the Pate et al. and the Sirard et al. intensity thresholds, respectively.
Results
Table 1 shows descriptive characteristics of the included children. Of the 30 children, 14 were boys. One fifth was considered overweight, but no child was obese.
n = 30.
Average observed time was 29 (SD 6) minutes per child. Mean accelerometer counts were lowest when children were watching a cartoon and highest when dancing to music (Table 2).
Pursuant to the protocol, mean accelerometer counts increased by CARS score (Table 3).
n = 30.
Table 4 shows the developed intensity thresholds together with sensitivity, specificity and AUC.
Sensitivity was 100% and 70%, and specificity was 60% and 93–100% for the sedentary and MVPA intensity thresholds, respectively. AUC ranged from 0.91–0.99.
The absolute agreement was 82% (VA and VM) for the wrist-worn accelerometer and 68% (VA) and 75% (VM) for the accelerometer worn on the hip. For the wrist-worn accelerometer, the quadratic weighted Kappa was almost perfect; 0.92 (95% CI 0.91–0.93) and 0.95 (95% CI 0.94–0.96) for the VA and the VM, respectively. The Kappa values for the hip-worn accelerometer were lower compared to the wrist-worn device; 0.76 (98% CI 0.74–0.77) and 0.86 (95% CI 0.85–0.87) for the VA and VM, respectively.
When applying the Pate et al. intensity thresholds to our VA ROC curve for the hip, sensitivity and specificity for the MVPA intensity threshold were 97% and 67%, respectively. For the Sirard et al. threshold, sensitivity and specificity were 73% and 97%.
Discussion
This is the first study to develop intensity thresholds for both the VA and the VM for a wrist-worn Actigraph accelerometer in preschool children. A previous study has been performed in two-year-olds, using an almost identical protocol as in this study [20], as well as in 8–12 years olds [11]. Compared to those, the intensity thresholds developed in this study were higher. This indicates that age-specific thresholds are needed. During the infant, toddler and preschool years, motor skills are developing rapidly. The gait becomes increasingly stable which enables the child to move around more effectively than at earlier ages. Arm movements decrease gradually as the child ages, explaining the higher count thresholds in comparison with the study on 8–12 year olds [11]. In comparison with these studies, we found higher measures of AUC, sensitivity and specificity for the developed intensity thresholds.
Two previous studies have developed intensity thresholds for the hip-worn uniaxial Actigraph accelerometer for four-year-old children [10, 19]. In the study by Pate et al. oxygen consumption was used as the criterion measure and children performed structured activities during the calibration session [19]. In the study by Sirard et al. CARS was used as the reference method, as in the present study. Using the same definition of MVPA (CARS score of at least 4), the developed count thresholds were similar to those derived in this study [10], confirming the validity of our hip MVPA intensity threshold. When applying the intensity thresholds by Pate et al. and Sirard et al. to the output from the ROC curves from this study, the sensitivity and specificity were almost the same in our study as for the MVPA threshold developed by Sirard et al. In order to avoid overestimation of the time spent in MVPA, we chose an intensity threshold with high specificity, at the expense of a lower sensitivity for that threshold. For the threshold developed by Pate et al., however, the sensitivity was high but the specificity relatively low, which will lead to an overestimation of time spent in MVPA. This indicates that the choice of criterion measure and definition of moderate intensity have great impact on the intensity thresholds derived.
The definition of moderate intensity is under debate and has been defined differently across studies [11, 21]. Like in previous studies [9, 10, 14] CARS scores 4 and 5 were categorized as a proxy for MVPA. This is based on a validation study, in which CARS was validated against oxygen consumption in 5-6-year-olds. In that study, CARS score 3 and 4 was found to correspond to 2.69 and 3.45 metabolic equivalents, respectively [8]. CARS has also been found to accurately classify the level of energy expenditure and to correlate with Caltrac motion sensors in children aged 2–6 [15]. Nevertheless, estimating what represents moderate intensity in preschoolers is problematic. It is known that the energy cost for physical activity per kilo body weight tends to decrease with advancing age [22]. Since the ability to absorb oxygen, muscle strength and motor skills develop as the child mature, it is likely that the metabolic equivalent for a certain activity differs between a four- and a six-year-old. This area of research needs further exploration.
Since a three axial accelerometer was used in the present study, intensity thresholds for the VM were also developed. These were found to have higher validity than those for the VA (Kappa value of 0.95 vs 0.92 for the wrist and 0.86 vs 0.76 for the hip). This is in accordance with a previous study, which found that activity counts derived from the VM yielded a more accurate estimation of energy expenditure than those derived from the VA, in youths, adults and older adults [23]. In the present study, the VM also had higher validity for the wrist intensity thresholds, as was found in a previous study on two-year-olds [5]. Thus, placing the Actigraph on the wrist and using the VM may be recommended for assessing time in different physical activity intensities in this population.
The accelerometer output differs according to the body site to which it is attached. A wrist-worn accelerometer captures movements performed by the arms, unlike a hip-mounted monitor. In accordance with results from previous studies, we found that accelerometer data collected from the wrist yielded higher accelerometer output compared with the hip-worn monitor, while measuring the same individual concurrently [6, 7]. Previous studies on adults have demonstrated that hip-mounted monitors have better accuracy in predicting physical activity energy expenditure than wrist-mounted monitors [24–26]. Wearing the accelerometer on the wrist can be problematic if the subject is walking with the hands constrained, for example when carrying a load. The accelerometer output will probably then be much different from walking with the hands swinging freely. However, when observing children during free play children tend to engage in activities like carrying heavy objects infrequently, making this likely to represent a minor problem among the preschool population. A wrist placement has been shown to accurately estimate energy expenditure [27] and, as in the present study, to estimate time spent in different intensities [5, 11] in children. Since a wrist placement can increase compliance [6], these results are promising.
Strengths and limitations
Strengths of the present study are the use of a commonly used reference method to categorize physical activity into intensity levels, including both structured activities and free play. Data were summarized over a short epoch length of 5 seconds, which has been recommended for young children [3]. The study design enables comparison of the VA and the VM, as well as wrist and hip placement of the accelerometer, helping researchers and practitioners chose the optimal monitor placement site and data handling for assessing time in physical activity intensities for this population.
Some limitations need to be discussed. First, one fifth of the included children were considered overweight, but none were obese. The output does not seem to differ between adults with different BMI [28] but it is possible that our results would have been different with a more heterogeneous sample. Secondly, accelerometer and CARS data were matched on a 5-second level because children in this age group change their activity level frequently. This is a very narrow time span making it possible that data were mismatched. Hence, by video recording the computer clock on that same computer that was used for initializing the accelerometers in the beginning of each session, we ensure that the correct 5 seconds were matched. Third, when using ROC curve analysis, the intensity thresholds were chosen based on sensitivity and specificity. A high sensitivity was chosen at the expense of low specificity, and vice versa. Since we did not want to underestimate sedentary time, we chose to prioritize a high sensitivity for that threshold. In the same way, we did not want to overestimate time in MVPA, which is why high specificity was prioritized for the MVPA thresholds. A high specificity for the MVPA threshold is important in order to correctly evaluate if a child is reaching recommendations for physical activity. This approach can, however, be questionable. Regarding external validity, only healthy children without impairments were included, which is why these results should not be generalized to populations of children with disabilities.
Conclusion
The derived thresholds were found to be valid for categorizing physical activity intensity in four-year-old children. For both wrist and hip placement the VM had higher validity than the VA. In comparison with placing the accelerometer on the hip, the wrist-mounted monitor performed better in accurately assessing time spent being sedentary, in light activity and in MVPA. Based on the findings of this study, the VM based on wrist-worn accelerometers for measuring physical activity in 4-year-old children is recommended to use.
Acknowledgments
We would like to acknowledge the involved preschool staff and all participating children.
Author Contributions
- Conceptualization: EJ MH.
- Data curation: EJ LL MH.
- Formal analysis: EJ LL MH.
- Funding acquisition: CM.
- Investigation: EJ LL CM MH.
- Methodology: EJ MH.
- Project administration: CM.
- Resources: EJ LL CM MH.
- Software: CM.
- Supervision: CM MH.
- Validation: EJ LL CM MH.
- Visualization: EJ.
- Writing – original draft: EJ.
- Writing – review & editing: EJ LL CM MH.
References
- 1. Timmons BW, Leblanc AG, Carson V, Connor Gorber S, Dillman C, Janssen I, et al. Systematic review of physical activity and health in the early years (aged 0–4 years). Appl Physiol Nutr Metab. 2012; 37: 773–92. pmid:22765840
- 2. Bassett DR Jr., Rowlands A, Trost SG. Calibration and validation of wearable monitors. Med Sci Sports Exerc. 2012; 44: S32–8. pmid:22157772
- 3. McClain JJ, Abraham TL, Brusseau TA Jr., Tudor-Locke C. Epoch length and accelerometer outputs in children: comparison to direct observation. Med Sci Sports Exerc. 2008; 40: 2080–7. pmid:18981941
- 4. Van Cauwenberghe E, Gubbels J, De Bourdeaudhuij I, Cardon G. Feasibility and validity of accelerometer measurements to assess physical activity in toddlers. Int J Behav Nutr Phys Act. 2011; 8: 67. pmid:21703004
- 5. Johansson E, Ekelund U, Nero H, Marcus C, Hagstromer M. Calibration and cross-validation of a wrist-worn Actigraph in young preschoolers. Pediatr Obes. 2015; 10: 1–6. pmid:24408275
- 6. Fairclough SJ, Noonan R, Rowlands AV, van Hees V, Knowles Z, Boddy LM. Wear Compliance and Activity in Children Wearing Wrist and Hip-Mounted Accelerometers. Med Sci Sports Exerc. 2015.
- 7. Hildebrand M, VANH VT, Hansen BH, Ekelund U. Age group comparability of raw accelerometer output from wrist- and hip-worn monitors. Med Sci Sports Exerc. 2014; 46: 1816–24. pmid:24887173
- 8. Puhl J G K, Hoyt M, Baranowski T. Children's Activity Rating Scale (CARS): description and calibration. Res Q Exerc Sport. 1990; 61: 26–36. pmid:2091164
- 9. van Cauwenberghe ELV, Trost SG, de Bourdeaudhuij I, Cardon G. Calibration and comparison of accelerometer cut points in preschool children. Int J Pediatr Obes. 2011; 6: 582–9.
- 10. Sirard JR T SG, Pfeiffer KA, Dowda M, Pate R. R. Calibration and evaluation of an objective measure of physical activity in preschool children. J Phys Act Health. 2005; 3: 345–57.
- 11. Chandler JL, Brazendale K, Beets MW, Mealing BA. Classification of physical activity intensities using a wrist-worn accelerometer in 8-12-year-old children. Pediatr Obes. 2015.
- 12. Cole TJ, Bellizzi MC, Flegal KM, Dietz WH. Establishing a standard definition for child overweight and obesity worldwide: international survey. BMJ. 2000; 320: 1240–3. pmid:10797032
- 13. Keadle SK, Shiroma EJ, Freedson PS, Lee IM. Impact of accelerometer data processing decisions on the sample size, wear time and physical activity level of a large cohort study. BMC Public Health. 2014; 14: 1210. pmid:25421941
- 14. Costa S, Barber SE, Cameron N, Clemes SA. Calibration and validation of the ActiGraph GT3X+ in 2–3 year olds. J Sci Med Sport. 2014; 17: 617–22. pmid:24365695
- 15. DuRant RH, Baranowski T, Puhl J, Rhodes T, Davis H, Greaves KA, et al. Evaluation of the Children's Activity Rating Scale (CARS) in young children. Med Sci Sports Exerc. 1993; 25: 1415–21. pmid:8107551
- 16. Trost SG, Fees BS, Haar SJ, Murray AD, Crowe LK. Identification and validity of accelerometer cut-points for toddlers. Obesity (Silver Spring). 2012; 20: 2317–9.
- 17. Hanley JA M B. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143: 29–36. pmid:7063747
- 18. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005; 85: 257–68. pmid:15733050
- 19. Pate RR, Almeida MJ, McIver KL, Pfeiffer KA, Dowda M. Validation and calibration of an accelerometer in preschool children. Obesity (Silver Spring). 2006; 14: 2000–6.
- 20. Johansson E, Hagstromer M, Svensson V, Ek A, Forssen M, Nero H, et al. Objectively measured physical activity in two-year-old children inverted question mark levels, patterns and correlates. Int J Behav Nutr Phys Act. 2015; 12: 3.
- 21. Crouter SE, Flynn JI, Bassett DR Jr. Estimating physical activity in youth using a wrist accelerometer. Med Sci Sports Exerc. 2015; 47: 944–51. pmid:25207928
- 22. Ridley K, Ainsworth BE, Olds TS. Development of a compendium of energy expenditures for youth. Int J Behav Nutr Phys Act. 2008; 5: 45. pmid:18782458
- 23. Santos-Lozano A, Santin-Medeiros F, Cardon G, Torres-Luque G, Bailon R, Bergmeir C, et al. Actigraph GT3X: validation and determination of physical activity intensity cut points. Int J Sports Med. 2013; 34: 975–82. pmid:23700330
- 24. Rosenberger ME, Haskell WL, Albinali F, Mota S, Nawyn J, Intille S. Estimating activity and sedentary behavior from an accelerometer on the hip or wrist. Med Sci Sports Exerc. 2013; 45: 964–75. pmid:23247702
- 25. Freedson PS, Melanson E, Sirard J. Calibration of the Computer Science and Applications, Inc. accelerometer. Med Sci Sports Exerc. 1998; 30: 777–81. pmid:9588623
- 26. Swartz AM, Strath SJ, Bassett DR Jr., O'Brien WL, King GA, Ainsworth BE. Estimation of energy expenditure using CSA accelerometers at hip and wrist sites. Med Sci Sports Exerc. 2000; 32: S450–6. pmid:10993414
- 27. Ekblom ONG, Bak EE, Ekelund U, Marcus C. Validity and comparability of a wrist-worn accelerometer in children. J Phys Act Health. 2012; 9: 389–93. pmid:22454440
- 28. Feito Y, Bassett DR, Tyo B, Thompson DL. Effects of body mass index and tilt angle on output of two wearable activity monitors. Med Sci Sports Exerc. 2011; 43: 861–6. pmid:20962689