Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A natural user interface to integrate citizen science and physical exercise

  • Eduardo Palermo,

    Affiliations Department of Mechanical and Aerospace Engineering, New York University Tandon School of Engineering, Brooklyn, NY, United States of America, Department of Mechanical and Aerospace Engineering, “Sapienza” University of Rome, Rome, Italy

  • Jeffrey Laut,

    Affiliation Department of Mechanical and Aerospace Engineering, New York University Tandon School of Engineering, Brooklyn, NY, United States of America

  • Oded Nov,

    Affiliation Department of Technology Management and Innovation, New York University Tandon School of Engineering, Brooklyn, NY, United States of America

  • Paolo Cappa †,

    † Deceased.

    Affiliations Department of Mechanical and Aerospace Engineering, New York University Tandon School of Engineering, Brooklyn, NY, United States of America, Department of Mechanical and Aerospace Engineering, “Sapienza” University of Rome, Rome, Italy

  • Maurizio Porfiri

    mporfiri@nyu.edu

    Affiliation Department of Mechanical and Aerospace Engineering, New York University Tandon School of Engineering, Brooklyn, NY, United States of America

Abstract

Citizen science enables volunteers to contribute to scientific projects, where massive data collection and analysis are often required. Volunteers participate in citizen science activities online from their homes or in the field and are motivated by both intrinsic and extrinsic factors. Here, we investigated the possibility of integrating citizen science tasks within physical exercises envisaged as part of a potential rehabilitation therapy session. The citizen science activity entailed environmental mapping of a polluted body of water using a miniature instrumented boat, which was remotely controlled by the participants through their physical gesture tracked by a low-cost markerless motion capture system. Our findings demonstrate that the natural user interface offers an engaging and effective means for performing environmental monitoring tasks. At the same time, the citizen science activity increases the commitment of the participants, leading to a better motion performance, quantified through an array of objective indices. The study constitutes a first and necessary step toward rehabilitative treatments of the upper limb through citizen science and low-cost markerless optical systems.

Introduction

Citizen science empowers people with little to no scientific training to participate in research led by professional scientists in different ways [1, 2]. The benefit of such an activity is often bidirectional [3], whereby professional scientists leverage the effort of a large number of volunteers in data collection or analysis, while the volunteers increase their knowledge and get the satisfaction of taking part in a scientific endeavor. A well-known example is Galaxy Zoo [4], a citizen science astronomy project that involves people in the classification of a large number of images obtained through several astronomic surveys. To date, volunteers have provided millions of classifications, enabling the development of improved classification and anomaly-detection algorithms for future surveys [5]. Citizen science is not limited to online projects, but also may involve the physical participation of volunteers [611].

In some cases, citizen science provides an additional benefit, such as increasing awareness of social and environmental issues [12]. In Gowanus Voyage [13] (part of the citizen science project Brooklyn Atlantis [14]), volunteers were part of the environmental monitoring of one of the most polluted bodies of water in the United States, the Gowanus Canal in Brooklyn, NY [15]. Driving miniature sensor-equipped boats via remote control in the canal, volunteers contributed to mapping the water quality, while learning from scientists about the pollution of a body of water flowing through the highly populated urban area where they lived.

The wide spectrum of motivational factors tapped by citizen science [1618] may be applied to other settings such as neuro-rehabilitation. Certain rehabilitative regimens may consist of regularly performing repetitive exercises [19], and adherence to these regimens is crucial for the effectiveness of the therapy [20]. Although a number of “serious games” has been recently explored for rehabilitation purposes [21], their use in long term rehabilitation treatments has been often questioned, especially with respect to elderly patients [22, 23]. Such a population was found to prefer the inclusion of content that has the potential to stimulate reflection and problem solving activity [24].

The possibility of engaging participants in science learning while performing rehabilitative exercises has been investigated in [25]. In that study, healthy adults who navigated a virtual environment using a low-cost haptic joystick attained higher levels of satisfaction when exposed to science content. A more recent effort [26] demonstrated that contributing to a citizen science project through the same low-cost device could provide an additional stimulus for children with hemiplegia. The study indicated that conducting a scientific task represented a more engaging means to perform the protocol, compensating for the tediousness of repetitive exercise schemes.

Low-cost haptic devices, such as the Novint Falcon used in these studies [25, 26], afford the administration of force feedback—the application of a contact force to the user’s hand based on a pre-programmed strategy [2730]. The employment of such a technology, typical of robot-mediated therapy (RMT) devices [3137] in a domestic environment, represents a promising opportunity for long-lasting treatments [3842]. However, this portability is accompanied by a limited workspace, which hampers the possibility to perform large movements needed for some treatments [43]. To address these drawbacks, consumer motion systems have been recently proposed for human analysis and rehabilitation [44].

The Microsoft Kinect sensor, for example, is a markerless human motion tracker capable of estimating three-dimensional (3D) coordinates of human joints in a skeleton model [45]. Initially designed for gaming, it has been repurposed as an input device for natural user interfaces [46], where the movement of the human body, or a part of it in the form of gestures [47], is integrated in human-computer applications [48]. Such a functionality stimulated the use of Kinect both in computer-based rehabilitation [49, 50] and human motion analysis [51]. Importantly, Fernandez-Baena et al. [52] have demonstrated that the Kinect yields joint angles highly comparable to those identified by professional-grade optoelectronic systems, validating its use for objective evaluation of physical exercise.

The present study is driven by the overarching research goal of integrating the motivational factors of citizen science in physical rehabilitation activities. In particular, this study builds on our previous studies, by exploring the possibility of improving engagement in upper limb rehabilitation through citizen science participation and haptic devices in [25] and [26]. In contrast with these efforts, we shift from a laboratory or clinical setting to directly involve participants in environmental data collection. This study represents the first step towards this goal, where we test its usability on healthy participants.

Specifically, we seek to demonstrate the possibility of integrating Kinect-based physical activities and citizen science. The citizen science task was based on Gowanus Voyage, and consisted of water quality mapping of a polluted body of water by means of a sensor-equipped miniature boat. To evaluate the benefit of the physical exercise toward citizen science, we assessed whether the motion commonly involved in rehabilitative exercises represented an engaging and effective means to control the instrumented boat with respect to a traditional system. To evaluate the benefit of citizen science on physical performance, we tested if a better motion performance of participants was associated with a better performance on the environmental task, and whether participation in this scientific project enhanced engagement and physical performance.

Materials and methods

The entire study, along with the consent procedure, was approved by the NYU Institutional Review Board (IRB). Participants gave their written informed consent.

Experimental setup

In Fig 1, we display a schematic of the experimental setup, consisting of a laptop computer, a Microsoft Kinect sensor (release 1.0, Microsoft, Redmond, WA, USA), and a miniature boat instrumented with water quality sensors, a GPS unit, and a radio module. The boat was controllable in thruster speed and rudder angle up to a distance of about 100m, and equipped with an Arduino microcontroller. The microcontroller was programmed to collect GPS and water quality data (conductivity, pH, temperature, and dissolved oxygen) and transmit the information to a base station via the radio module. The Arduino received driving data (thruster speed and rudder angle) via the radio module to control the DC motor connected to the thruster and the rudder servomotor.

thumbnail
Fig 1. Schematic of the experimental setup.

The participant performed the trial on the canal bank. A tent reduced the sunlight exposition of the Kinect to reduce artifacts. The gestures captured by the Kinect were processed in real time by the software running on a laptop and the corresponding commands were sent to the boat via a radio connection. At the same time, water quality data were transmitted from the boat to the laptop.

https://doi.org/10.1371/journal.pone.0172587.g001

The base station, where the subjects performed the experimental session, was located on the canal bank. It included a laptop computer and the Kinect sensor. The Kinect fused data-streams from two cameras: a VGA resolution RGB camera and an infrared camera. An additional linearly polarized filter film was placed on the infrared camera for using the Kinect outdoors. The experimental sessions were conducted underneath a 2.44 × 3.04 m tent with one side open to the canal, to block direct sunlight glare on objects within the sensor range, and permit a clear recognition of the subject by the Kinect.

A National Instruments LabVIEW software (release 12.0, NI, Austin, TX, USA) was developed and run on the laptop to: acquire 3D coordinates of the body joints from the Kinect at a frequency of 30 samples per second; log data to a binary file; compute and send driving commands to the boat based on predefined gesture; receive water quality data from the boat; and provide a driving interface feedback on the thruster speed and rudder angle levels to the participant, through virtual analog gauges. In our experiments, we found that the acquisition rate of the Kinect was 30.05±1.74 frames per second, different from [53], in which a larger variation in the sampling rate was documented. The software was developed using the “Kinesthesia” NI LabVIEW Programming APIs for Microsoft Kinect SDK. The Kinect SDK allows for setting the smoothing parameters for the 3D joint position estimation [54]. Based on pilot experiments, we chose the following parameter set to adequately estimate joint positions in outdoor environments: {Smoothing = 0.5, Correction = 0.1, Prediction = 0.5, JitterRadius = 0.1, MaxDeviationRadius = 0.1}.

The experimental protocol and the data processing were designed around the capabilities of the Kinect [52, 55]. For example, Obdrzalek et al. [55] evaluated the accuracy of the Kinect in localizing human joint centers and demonstrated the primary role of both the subject view angle from the Kinect sensor and the subject posture (sitting or standing). The highest accuracy was attained for the subject standing in frontal view, with a maximum localization error of approximately 70 mm for the wrist joint. The error in the estimation of the joint angles was assessed by Fernandez-Baena et al. [52], during dynamic trials, and a maximum error of 16% of the range of motion (ROM) was found for the shoulder motion in the frontal plane.

Resting on this evidence and a few pilot experiments, the participant was positioned in front of the Kinect sensor (placed on a tripod at a height of 155 cm) at a distance of approximately 2 m. The participant was instructed to move only their dominant arm, without constraining the other arm, and not to move their legs and feet, unless a fall was imminent. An arm gesture in front of the Kinect was used to drive the boat tasked with completing a path in the minimum possible time. This type of gesture was designed to push the subject to extend their arm, keeping it far from the trunk. A widely adopted typology of rehabilitative exercises in hemiplegia is based on the repetitively performing simple movements with the affected arm, in some cases constraining the less affected one [56, 57]. Here, by lifting up their dominant arm, the participant engaged the control, and the software interface began indicating the thruster speed and the steering angle as the driving feedback.

Moving the wrist forward and backward in the horizontal plane allowed the participant to regulate the thruster speed, while the left-right motion of the wrist in the horizontal plane enabled them to set the rudder angle. In particular, referring to the Kinect reference axes shown in Fig 2, the input value of the thruster speed was proportional to the z-component of the vector joining the wrist and the hip center, divided by the arm length (from the shoulder joint to the wrist). Similarly, the input value for the rudder angle was proportional to the x-component of the above mentioned vector, divided by the arm length. In this way, the control parameters were independent of the participant’s position with respect to the sensor or their arm length.

thumbnail
Fig 2. Schematic of the driving gesture implemented (top view).

Moving the wrist forward and backward controls the thrust of the boat, while the left-right movement controls the steering angle. On the right, the Kinect reference frame is depicted.

https://doi.org/10.1371/journal.pone.0172587.g002

Experimental protocol

To examine the viability of the platform, all subjects were instructed to drive the boat along the same predefined path marked by a set of five buoys in the shortest time possible, and to avoid deviations from the path. Since each participant controlled the same boat along the same path, the driving performance of subjects was assessed by measuring the lap time with a stopwatch. Fig 3 shows the path obtained by researchers, by placing numbered buoys in the canal using a canoe. The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details. The path was traversed starting from the orange buoy, turning around the yellow buoys following the numbered order, and coming back to the starting place. Each participant performed only one trial, and only trials in which the complete path was followed were considered in the study. None of the participants experienced fatigue by the end of the trial.

thumbnail
Fig 3. The challenge path.

The orange buoy represents the starting and the finish of the path, performed driving the boat around four numbered yellow buoys.

https://doi.org/10.1371/journal.pone.0172587.g003

A total of 60 healthy adults (46 male and 14 female) were recruited to participate in the experiments, among pedestrians passing by the canal bank. Only participants with no prior experience with Brooklyn Atlantis project were considered eligible for the study. The age range of participants was 18–50 years, as within such a range, the kinematic behavior of the upper limb is comparable across subjects [58]. The participants involved were divided into three groups of equal size, described below and classified in Table 1. The inclusion of subjects in the groups was randomized.

thumbnail
Table 1. Classification of the three experimental groups.

https://doi.org/10.1371/journal.pone.0172587.t001

To evaluate the effectiveness of the entire Kinect-based platform developed as means for helping to collect environmental data, one of the three groups drove the boat along the path using a traditional joystick (group J), with a trigger to control the thruster and a knob for the steering. The other two groups used a boat controlled via the Kinect (group KC and group KNC), differentiated as described in the following. To test the effect of the scientific research context on motion performance, the KC group participants were informed at the beginning of the trial that their effort will be used to automatically collect data from water quality sensor, similar to data collected by scientists and included in the Brooklyn Atlantis project. At the end of the trial, the KC group participants saw the result of this collection as a screen pop-up with the map of the canal with water quality values superimposed, the mean values of the collected data, and a brief explanation of their meaning. The participants of the KNC group were told that their trial was only targeted at testing their driving skills and assessing the platform and no environmental data collection report was shown at the end. During the trials, kinematic data of subjects in KC and KNC were recorded to evaluate the quality and quantity of the body motion through the motion performance indices detailed below.

At the end of the session, participants were asked to fill out a survey, described in the next subsection. The survey consisted of ten items (Q1–Q10). The first two items were about the controlling device they used (joystick or Kinect) and their gender. The third asked participants if they were a registered user on the Brooklyn Atlantis website [59] to test their eligibility in the study. The remaining items are reported below. The first four (Q4–Q7) were intended to rank the engagement of the participants and the perceived maneuverability of the control strategy they used. The last three items were dedicated specifically to the group involved in the data collection (KC), to evaluate their interest in the scientific activity. For each item the participants chose a level of agreement using a four point Likert scale [60] (strongly agree, somewhat agree, somewhat disagree, and strongly disagree). We used a four point scale with no neutral option to push participants to think about the question before answering, as the neutral option is often chosen from indecision [61].

  1. Q.4 Today’s activity was enjoyable.
  2. Q.5 It was easy to control the boat.
  3. Q.6 It was fun to control the boat.
  4. Q.7 The boat moved as I intended it to.
  5. Q.8 I am helping scientists.
  6. Q.9 I am contributing to environmental monitoring.
  7. Q.10 I have learned much about pollution and recovery of the Gowanus Canal today.

We assigned an ordinal values scale from 1 to 4 to the answers of survey items Q4–Q10, increasing with the level of agreement. The scores obtained for the answers to items Q4 and Q6 should be related to the level of engagement perceived during the experience, and they were summed to yield the “engagement factor”, ranging from 2 to 8. A higher score should correspond to a higher level of engagement of the participant. Similarly, answers Q5 and Q7 were combined to yield the overall “maneuverability factor”, ranging from 2 to 8, where a higher score should indicate that the subject felt the boat easier to control. Survey items Q8, Q9, and Q10 concerned the interest of the subject in the scientific task and their feeling to be an active part of it. The scores obtained for these answers were summed to gauge the interest of the participants toward the citizen science project, obtaining a total score ranging from 3 to 12.

Data processing

Using the 3D coordinates of the 20 joint centers provided by Kinect, a kinematic model of the human body can be built [54]. Fig 4 shows a schematic of the model reconstructed in MATLAB (release 7.12, The MathWorks, Natick, MA, USA) from data of one acquisition frame. The model does not identify three reference points in each body segment, thereby not allowing to solve the complete three-degree of freedom joint angular kinematics. As a result, the model cannot be used to acquire information on some joint angles, such as the arm rotation around its longitudinal axis. For this purpose, each body segment should be assumed to be rigid so that its pose could be computed from the coordinates of at least three points belonging to it. In the Kinect model, only the trunk and the pelvis have more than two measured points. Thus, focusing on the upper limb, it is possible to define two joint rotations for the shoulder and one for the elbow.

thumbnail
Fig 4. Schematic of the kinetic model of the human body obtainable with Kinect data.

The circles represent the 20 articular joint positions provided by the Kinect system. The local coordinate system of the trunk is depicted in black and vse and vew are the vectors connecting the shoulder joint with the elbow joint and the elbow joint with the wrist joint, respectively.

https://doi.org/10.1371/journal.pone.0172587.g004

We defined the local coordinate system for the trunk with the zt-axis directed parallel to the vector connecting the spine joint with the shoulder center [54], the xt-axis perpendicular to the zt-axis and lying in the plane containing the two shoulder joints and the shoulder center, and the yt-axis following the right-hand rule. The origin of the local coordinate system was located at the shoulder center joint. In such a model, the shoulder motion is approximated through the rotation of the arm with respect to the trunk, without including the motions of two anatomical parts articulating in between them, the scapula and the clavicle [62]. We computed the two joint angles of the shoulder and joint angle of the elbow following Pacilli et al. [63]. With reference to Fig 4, the angle formed by the vector connecting the shoulder center and the elbow center (vse) with the ztxt-plane of the trunk is termed as shoulder flexion/extension (Fl/Ex). The angle that the projection of vse on the ztxt-plane of the trunk forms with zt is the shoulder abduction/adduction (Ab/Ad). Finally, the elbow Fl/Ex is the angle between vse and the vector connecting the elbow and the wrist joint centers (vew).

A set of kinematic performance indices were selected as a tool for evaluating motor skills. These indices were chosen in terms of quantity and quality of the motion of the participant, as both these factors are considered crucial in quantifying effectiveness of physical rehabilitation [64, 65]. The following parameters were considered:

  1. ROM of shoulder and elbow, defined as the difference between the maximum and the minimum level of the angle reached during the trial. In particular, for each trial three values of the ROM were computed (shoulder Fl/Ex, shoulder Ab/Ad, and elbow Fl/Ex).
  2. Root mean square of angular velocity (AVRMS) of shoulder and elbow. The values of this new index were calculated as the root mean square of the derivative of the three computed joint angle signals. AVRMS is intended to quantify the peak-to-peak variation of joint angular velocity.
  3. The planarity motion index (PMI). We defined this index as (1) where σa, σb, and σc were the three eigenvalues, in descending order, of the 3 × 3 covariance matrix from the three components of the trajectory of the wrist in the xyz-coordinate system. A value close to 0 implies that the motion of the wrist is 2D as requested by the planar arm motion of the driving task. 3D motions revealed by values of PMI closer to 1 should be associated with low motion control levels.
  4. Trunk stabilization index (TSI), defined as the standard deviation of the time traces roll and pitch angles of the trunk during the trial, considering the roll-pitch-yaw rotation sequence of the xtytzt-coordinate system with respect to the xyz one. This metric scored the capability of the subject to isolate the gesture to the arm level without losing balance.
  5. The speed metric (SM), measured as the mean velocity of the wrist above a threshold value divided by the peak velocity, represents a score of the smoothness of the motion [66].
  6. The normalized number of submovements (NNSM) is a modified version of the number of submovements (NSM) [67], obtained by dividing the NSM by the time duration of the trial in seconds.

As a first step in the data analysis, we sought to verify if the Kinect platform represents a valid solution for performing water quality data collection. Specifically, we evaluated the solution both from a subjective and an objective point of view. To assess the statistical significance of the collected survey measures, we used the chi-square test [68], evaluating if the obtained score distributions for engagement factor and maneuverability factor in the three groups were different from an expected value due to chance. The chi-square test was also used to assess differences in engagement factor and maneuverability factor between groups.

Beyond the subjective evaluation provided by the survey, we considered the completion time of the task as an objective measure. In particular, we analyzed differences among the three groups with a Mann-Whitney U test [69], as the measurement sample was non-normally distributed, as seen from a Kolmogorov-Smirnov test [70] (p < 0.001).

To evaluate the validity of the scientific task as a physical exercise, we looked for a correlation between physical performance and skill in controlling the boat. Such a relationship is important to predict whether future improvement of subjects in their driving skills, due to continuous practice over time, could be associated with an improvement in their motion quality. Only kinematic data of the targeted body segments from the two Kinect groups (KC and KNC) were used, since no rehabilitative gesture was performed by the joystick group (J). The Pearson’s correlation coefficients (and probability levels associated) were calculated between the time of completion for all the trials of both the Kinect groups, and the corresponding performance indices.

A chi-square test was executed on the interest score obtained for the KC participants to test whether their response was different from chance. Finally, a one-way ANOVA, with the data collection (KC and KNC) as the independent variable and all the motion performance indices as dependent variables was performed. This analysis sought to elucidate the effects of the awareness of the scientific activity.

For all the tests, statistical significance level was set at p = 0.05.

Results

Participants from all groups felt engaged

Participants considered the two control strategies to be highly engaging, and no differences were found between them. Specifically, the total score for the engagement factor was higher than the neutral level of 100 for the three groups, being very close to the maximum level of 160 with a distribution of the answers different from chance (J: χ2(6) = 188.667, p < 0.001; KC: χ2(6) = 188.667, p < 0.001; and KNC: χ2(6) = 213.067, p < 0.001) as shown in Table 2. The engagement factor values of the three groups were indistinguishable (J vs. KC: χ2(6) = 0, p = 1; J vs. KNC: χ2(6) = 0.23, p = 0.999; and KC vs. KNC: χ2(6) = 0.23, p = 0.999).

thumbnail
Table 2. Engagement factor, maneuverability factor, and lap time for the three groups.

Groups include participants controlling the boat via the joystick (J), the Kinect in data collecting trials (KC), and the kinect in non-collecting trials (KNC). For engagement and maneuverability factors, the reported values represent the total score obtained from all subjects. For the completion time, instead, mean values (standard deviation) are reported. Values sharing the same superscript in the same column are not statistically different.

https://doi.org/10.1371/journal.pone.0172587.t002

Both the joystick and the Kinect were effective for controlling the boat

As shown in Table 2, participants in each of the three groups perceived the boat to be maneuverable, whereby the total score was higher than the neutral value of 100 with a distribution of the answers different from chance for all three groups (J: χ2(6) = 212.667, p < 0.001; KC: χ2(6) = 24.667, p < 0.001; and KNC: χ2(6) = 20.667, p = 0.002). In addition, participants felt that the boat was more maneuverable with the joystick than the Kinect (J vs. KC: χ2(6) = 19.9, p = 0.003 and J vs. KNC: χ2(6) = 26.22, p < 0.001). No difference was found due to the awareness of the scientific task (KC vs. KNC: χ2(6) = 2.22, p = 0.894). As evidenced in Table 2, Kinect users completed the path in a longer time than remote control users (J vs. KC: U = 99.50, p = 0.007 and J vs. KNC: U = 129.00, p = 0.055). No difference in the time completion was noted between the two Kinect groups (KC vs. KNC: U = 180.00, p = 0.588). The minimum trial duration was 51 s and the maximum was 411 s.

Motion performance was correlated with the time of completion of the task

Most of the proposed motion indices were correlated with the time of task completion. Table 3 reports the Pearson’s correlation coefficients, and the associated level of probability, between each of the indices and the lap time. A moderate correlation was found between the ROM of all the considered angles and the driving performance (shoulder Fl/Ex: R = 0.532, p < 0.001; shoulder Ab/Ad: R = 0.512, p = 0.001; and elbow Fl/Ex: R = 0.512, p = 0.001). A correlation was found for AVRMS of elbow Fl/Ex (R = 0.341, p = 0.031). The TSI showed a weak correlation for the roll plane (R = 0.392, p = 0.012), and a moderate correlation for the pitch plane (R = 0.500, p = 0.001). A strong negative correlation was found for the SM index (R = −0.608, p < 0.001), displayed for clarity in Fig 5. No significant correlation, instead, was found for the AVRMS of the two shoulder joint angles (Fl/Ex: R = 0.231, p = 0.152; Ab/Ad: R = 0.138, p = 0.394), for the PMI (R = 0.124, p = 0.444) and for the NNSM (R = 0.040, p = 0.807), as reported in Table 3.

thumbnail
Table 3. Correlation coefficients and associated probability values between the time of completion of the task and the motion performance indices.

https://doi.org/10.1371/journal.pone.0172587.t003

thumbnail
Fig 5. Scatter plot of the completion time of the trial with respect to the speed metric (SM) scores.

The regression line is depicted in red. The Pearson’s correlation coefficient (R) and the probability associated (p) are reported above.

https://doi.org/10.1371/journal.pone.0172587.g005

Participants were interested in the citizen science activity

Participants maneuvering the boat via the Kinect who were made aware of the scientific component of the test (KC) exhibited a remarkably high interest in the citizen science project (χ2(9) = 91.253, p < 0.001), reaching a total score of 202 on a range of possible values from 60 to 240, where the neutral value was of 150, with a distribution of the answers different than chance.

Awareness of the scientific task increased the motion performance

Table 4 reports mean and standard deviation values for the indices in the two Kinect groups. ANOVA results showed a significant effect of the awareness of the scientific component on the Fl/Ex AVRMS of the elbow (F = 4.901, p = 0.033) and on the TSI in the pitch plane (F = 7.345, p = 0.010), whereby KC participants exhibited a lower mean Fl/Ex AVRMS of the elbow and a higher trunk stability (TSI), as depicted in Fig 6. For all other indices considered, mean values of the groups were not statistically different.

thumbnail
Table 4. Mean (standard deviation) values for the motion performance indices among all the subjects of the two Kinect groups.

Statistically significant differences are marked with stars (p(*) = 0.033, and p(**) = 0.010).

https://doi.org/10.1371/journal.pone.0172587.t004

thumbnail
Fig 6. Bar plots of two kinematic indices in the two Kinect groups.

Root mean square of the angular velocity (AVRMS) for the elbow Fl/Ex and pitch trunk stabilization index (TSI) in the two Kinect groups (KC and KNC) are depicted. Error bars indicate the 95% confidence interval. Statistically significant differences are indicated with different letters.

https://doi.org/10.1371/journal.pone.0172587.g006

Discussion

In this paper, we utilized a new low-cost technology to bridge citizen science and physical exercise, linking two activities that can benefit each other. Specifically, we proposed the use of a Kinect motion system to perform citizen science tasks of environmental data collection through an instrumented boat, during physical exercises tailored for future rehabilitation studies. Our results demonstrate that the use of this off-the-shelf device offers an engaging means to contribute to a scientific task without compromising the maneuverability that could be achieved with a more traditional remote control joystick. The markerless human motion tracker device allows for delivering relevant exercises, which can be characterized by wide and unrestricted 3D movements, while recording the body kinematics in time and scoring the subjects’ motion performance.

Although several Kinect-based strategies have been already established to foster physical exercise [50, 51], their integration in citizen science activity has not been explored. The novelty of our study lies in the original use of Kinect for controlling a mechatronic device that collects data for a scientific cause. Unlike other studies based on motion capture, we leveraged participation in citizen science to strengthen motivational factors in physical exercise. Testing this novel concept required an experimental study to its acceptance by users, whose response to Kinect-based citizen science activities could not have been directly anticipated from the literature on gaming [4547]. Virtual environments utilized in most gaming-based systems do not present the intricacies that characterize our mechatronic system, such as inertia, communication delays, and unstable or unpredictable behaviors due to interactions with objects. These features might have rendered the proposed solution ineffective or unattractive, thereby requiring a dedicated experimental study, which is at the core of this work. Just as the reception of our technology by users was difficult to predict, the specific role of the motivational aspects of citizen science was also elusive and warranted a focused study.

As hypothesized, the use of an unconventional strategy to drive a radio controlled device was perceived as less maneuverable with respect to a traditional remote control, a perception confirmed by a higher completion time of the task. However, the mean maneuverability was significantly higher than the satisfactory level, suggesting that the opportunity for physical exercise offered by the proposed system does not hinder the efficiency in completing the environmental monitoring task. These positive results suggest that the scientific purpose will not be regarded only as a recreational content, but also as a concrete environmental monitoring activity, an important driver for the success of the platform developed here.

Subjects who displayed a higher motion coordination and body stability were also found to be more efficient in controlling the instrumented boat. This relationship suggests that the awareness of the scientific contribution could lead to higher levels of care and accuracy in the task. The need of further developing this link is strengthened by the observed scientific interest and motion performance of participants aware of the environmental data collection they were contributing to. Participants informed of the scientific value of their effort performed better and indicated a strong interest in the overall citizen science project.

Our results confirm the hypothesis that physical activity involving a set of body motions is perceived as engaging, despite the fact that it may limit maneuverability [71]. It is possible that the gestures performed by the participants could offer an additional motivational driver, similar to instances of gamification in citizen science [72, 73]. Such drivers could thus supplement the intrinsic motivations associated with citizen science [74, 75] through recreational elements that sustain the participants’ involvement. This recreational aspect compensated for the increased difficulty as compared to the regular joystick and, as a result, the participants did not feel the use of gestures to be limiting in the completion of their task. Even if Kinect users performed the task in a time period about 50% longer than those who used the traditional remote control, they felt that the Kinect-based control was effective in maneuvering the boat. The volunteers’ perception of the effectiveness of their effort has often been proposed as a valid indicator of their future contribution to the project [16].

Engagement and maneuverability scores obtained in this study may be considered as predictors for the success of a potential rehabilitation therapy that would leverage citizen science and Kinect. In future applications of the platform, we will enhance objectivity of the assessment of participant interest, beyond survey administration. Through a different design, based on multiple subsequent sessions, we plan to investigate participants’ adherence to the project, overcoming some of the potential limitations of traditional survey instruments [76]. The endurance of participants may be evaluated by the amount that continue to follow the activity to the last sessions, a more representative index of long-lasting interest, free from ephemeral enthusiasm due to curiosity.

Beyond assessing the possibility of obtaining quantitative indices of motion performance via a low-cost device, our study explored the feasibility of a novel type of physical exercise with potential for future rehabilitation applications. Further studies on subjects with pathologies will be grounded on the positive findings of this work, where a correlation between the proposed motion metrics and participants’ ability to complete the path was observed. Such a result offers a promising basis for future efforts to investigate the suitability of the activity tested in this study as a rehabilitation task. Traditional upper limb rehabilitation tasks train patients to directly increase their precision in movements via the execution of a precise arm motion [77, 78]. The subjects of this study, instead, were instructed to drive a miniature boat along a path in the minimum possible time. We expect citizen science activities to increase both engagement and motivation of rehabilitation patients in future application of the proposed platform, expanding on the gaming applications tested herein. Our methodology expands behavioral rehabilitation, by providing an engaging and fun natural user interface, a tangible scientific contribution, and an attractive low-cost markerless technology for human motion capture.

Future efforts on subjects with pathologies will be informed by prior studies which have pointed at the potential viability of using the Kinect for stroke rehabilitation and elder care [50]. Some of these efforts posit that the inclusion of gaming elements [49, 79] and visual feedback [80] may induce higher levels of enjoyment, and consequently, improve patient adherence to rehabilitation regimens. Although these efforts offer compelling evidence in favor of the Kinect, long-term studies necessary for demonstrating its actual effectiveness for motor recovery have yet to be undertaken. Building on the findings of this study, future studies should contrast the proposed platform to both traditional physical rehabilitation and computer-based rehabilitation via the Kinect. Comparing motor recovery in an experimental group that uses the proposed methodology to control groups, employing traditional or computer-based rehabilitation, will shed light on the potential of the Kinect. This comparison will offer evidence for the feasibility of computer-based rehabilitation via the Kinect and help testing whether the motivational factors tapped by our approach would translate into a higher rehabilitation effectiveness.

Our preliminary evidence suggests that motivation of patients to improve their performance in driving the boat could be an additional element to be leveraged toward the delivery of effective rehabilitation treatments. Specifically, the correlation found between ROM of shoulder and elbow angles and the time completion of the task indicates that a smaller and more controlled gesture permits a better control of the boat. Similarly, the correlation between the TSI and the lap time suggests that subjects with a better balance are more skilled in performing the trial. The smoothness of the motion is a further feature that correlates with the lap time, whereby subjects exhibiting higher SM execute the trial in a shorter time. Thus, similar to virtual gaming elements considered in Kinect-based rehabilitation treatments [49, 50, 81], the possibility to remotely drive physical objects might offer an additional venue for improving the delivery of rehabilitation treatments.

Based on our results, the set of selected motion performance indices is expected to represent a valid tool in quantifying the motor deficit on a population with pathology. The correlation of some of these indices with driving capabilities, found on healthy subjects, is predictive of the efficacy of the water quality data collection activity to improve subjects’ motor skills. We would anticipate a comparable scenario from future application of this methodology on a population with motor deficit. The higher variation of motor skills in such a population would probably result in a wider spread of the performance indices values, as patients with upper limb impairment often present limited ROM [64], a reduced stability of the trunk when moving the arm [82], and a lower motion smoothness [66].

In agreement with our expectations, our results indicate that the awareness of the scientific task enhances subjects’ performance. This hypothesis rested upon findings of a previous study, where the inclusion of scientific content was shown to increase the level of satisfaction in a rehabilitation activity based on a virtual environment [26]. Here, we increased the level of commitment of the participants by direct involvement in authentic environmental data collection as part of a citizen science project. The task of following the path was designed to invoke a similar set of predefined movements between the two Kinect groups, regardless of awareness of environmental data collection. Subjects involved in the scientific task exhibited a significantly higher TSI and a reduced ROM of the shoulder Fl/Ex, which are both valid indicators of improved motion performance. It is thus tenable that motivational drivers addressed by citizen science, such as personal interest and the desire to contribute to science [75], reinforce the commitment of the participant to accurately maneuver the instrumented boat and collect useful environmental data. We would expect that these motivational drivers could be also useful in enhancing patients’ adherence to rigid exercise regimens, which are often involved in rehabilitation treatments and perceived as tedious [83, 84].

Our approach builds on the clinical advancements enabled by RMT in evaluating motion performance [66] toward an affordable, versatile, and portable technology for rehabilitation. While RMT often entails the use of complex and expensive devices that can only be accessed in specialized facilities, the proposed framework requires minimal clinical and technical infrastructure, which should facilitate rapid and simple implementations in diverse environments, including the patients’ home [85]. In such home settings, the patients could drive the miniature instrumented boat from their personal computers and contribute to citizen science remotely. Differently from the testing scenario presented in this study, a real rehabilitation session could be programmed by asking participants to drive the sensor-equipped boat through several target locations in the canal, and requiring a time duration comparable with traditional rehabilitation sessions. Such a design will afford the delivery of the requisite quantity (duration and frequency) and quality (task-specificity) of treatment that is needed for physical rehabilitation [86, 87]. The scalability of the tested platform, enabled by the use of motion capture rather than a mechatronic device, might allow to rapidly re-map the gesture under recommendation of specialized clinicians, and include subjects in need of various typologies of physical exercise or rehabilitation.

Our approach might particularly benefit patients with a motor impairment of appropriated severity, who can exhibit sufficient abilities to attempt the citizen science task. In such case, future works should involve rehabilitation outpatients after stroke and use the results of this study with healthy subjects as a baseline. Patients with disabilities too severe would likely require more traditional treatments or treatments with robotic devices to provide haptic assistance [88]. In particular, we envisage including patients, based on clinical evaluation of their impairment by medical specialists. The clinician’s case-by-case opinion, grounded both on the clinical status and risk evaluation, will be the final driver for inclusion. However, scores obtained via clinical motor impairment scales widely adopted in stroke evaluation, such as the Fugl-Mayer score [89], are likely to offer a first metric for the inclusion of patients. Specifically, adult patients, with a Fugl-Mayer score <50/66, sufficient cognitive skills, and able to sit and stand independently are expected to benefit from the platform proposed here.

Due to the high scalability of our natural user interface, this solution may be adapted to a different rehabilitation approach for post-stroke hemiparesis, based on the bilateral manipulation of objects [90]. Modification of the implemented gestures to bilateral, under clinicians’ supervision, would adapt the tested solution to patients with a moderate level of impairment severity, who can leverage the strength of the unaffected side to help the affected one, activating motor synergies between limbs [91].

Similar to systems used in RMT, the Kinect platform allows for evaluating an array of motion performance measures which could aid in assessing the patient recovery, without the continuous presence and direct supervision by a physical therapist, who could simultaneously supervise several patients at the same time. The simplicity of the setup tested in this work, only based on a low-cost markerless motion tracker, allows for its integration with a set of other sensors typically adopted in the evaluation of motor impairment.

In an extended version of the proposed paradigm, the inclusion of recently developed wearable force or torque sensors [92, 93] may provide a more comprehensive and valuable assessment of motor impairment, not only based on kinematics. By further integrating other devices, future incarnations of the proposed platform could extend exercise and evaluation indices to the lower limb as well. The inclusion of recent wearable solutions that have been introduced and validated for the lower limb [9497], may extend the evaluation of kinematics and dynamics to the lower extremity. This is a crucial body region for a recovering patient’s autonomy, and the possibility of including inertial and force sensors may considerably empower the effectiveness of the Kinect platform.

Supporting information

S1 Table. Processed dataset.

The table contains classification data of all anonymous participants (gender, used device, water quality data collection), along with time of completion of the task, kinematic indices calculated from raw data, and survey answers (Q4 to Q11).

https://doi.org/10.1371/journal.pone.0172587.s001

(XLSX)

S1 Folder. Kinect raw data.

Kinect raw data of each participant in the two kinect groups (KC and KNC). Data of each participant are saved in a binary file. Trajectories of points have been extracted form the binary file and saved in a text file.

https://doi.org/10.1371/journal.pone.0172587.s002

(ZIP)

Acknowledgments

Paolo Cappa passed away before the submission of the final version of this manuscript. Maurizio Porfiri accepts responsibility for the integrity and validity of the data collected and analyzed.

The authors would like to gratefully acknowledge Mauro De Bellis and Francesco Cappa for their assistance with the experiments and The Gowanus Dredgers for providing canoes.

Author Contributions

  1. Conceptualization: EP JL ON PC MP.
  2. Data curation: EP JL MP.
  3. Formal analysis: EP JL ON PC MP.
  4. Funding acquisition: ON MP.
  5. Investigation: EP JL.
  6. Methodology: EP JL ON PC MP.
  7. Project administration: ON MP.
  8. Resources: MP.
  9. Software: EP JL.
  10. Supervision: ON MP.
  11. Validation: EP JL.
  12. Visualization: EP.
  13. Writing – original draft: EP JL MP.
  14. Writing – review & editing: EP JL ON PC MP.

References

  1. 1. Silvertown J. A new dawn for citizen science. Trends in Ecology & Evolution. 2009;24(9):467–471.
  2. 2. Dickinson JL, Zuckerberg B, Bonter DN. Citizen science as an ecological research tool: challenges and benefits. Annual Review of Ecology, Evolution, and Systematics. 2010;41:149–172.
  3. 3. Haklay M. Citizen science and volunteered geographic information: overview and typology of participation. In: Crowdsourcing Geographic Knowledge. Springer; 2013. p. 105–122.
  4. 4. Lintott CJ, Schawinski K, Slosar A, Land K, Bamford S, Thomas D, et al. Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society. 2008;389(3):1179–1189.
  5. 5. Fortson, L, Masters K, Nichol R, Borne K, Edmondson E, Lintott C, et al. Galaxy Zoo: morphological classification and citizen science. arXiv preprint arXiv:11045513. 2011;.
  6. 6. Becker M, Caminiti S, Fiorella D, Francis L, Gravino P, Haklay MM, et al. Awareness and learning in participatory noise sensing. PLoS ONE. 2013;8(12):e81638. pmid:24349102
  7. 7. Conrad CC, Hilchey KG. A review of citizen science and community-based environmental monitoring: issues and opportunities. Environmental Monitoring and Assessment. 2011;176(1–4):273–291. pmid:20640506
  8. 8. Dickinson JL, Shirk J, Bonter D, Bonney R, Crain RL, Martin J, et al. The current state of citizen science as a tool for ecological research and public engagement. Frontiers in Ecology and the Environment. 2012;10(6):291–297.
  9. 9. Cooper CB, Shirk J, Zuckerberg B. The Invisible Prevalence of Citizen Science in Global Research: migratory Birds and Climate Change. PLoS ONE. 2014;9(9):e106508. pmid:25184755
  10. 10. Gouveia C, Fonseca A, Câmara A, Ferreira F. Promoting the use of environmental data collected by concerned citizens through information and communication technologies. Journal of Environmental Management. 2004;71(2):135–154. pmid:15135948
  11. 11. Foster-Smith J, Evans SM. The value of marine ecological data collected by volunteers. Biological Conservation. 2003;113(2):199–213.
  12. 12. Scott D, Barnett C. Something in the air: civic science and contentious environmental politics in post-apartheid South Africa. Geoforum. 2009;40(3):373–382.
  13. 13. Laut J, Nelson Wright S, Nov O, Porfiri M. Gowanus Voyage: where Mechatronics, Public Art, Community Members, and Environmental Science Meet. IEEE Control Systems. 2014;34(1):60–64.
  14. 14. Laut J, Henry E, Nov O, Porfiri M. Development of a Mechatronics-Based Citizen Science Platform for Aquatic Environmental Monitoring. IEEE/ASME Transactions on Mechatronics. 2014;19(5):1541–1551.
  15. 15. Pearsall H. Superfund me: a study of resistance to gentrification in New York City. Urban Studies. 2013;50(11):2293–2310.
  16. 16. Nov O, Arazy O, Anderson D. Scientists@Home: what drives the quantity and quality of online citizen science participation? PLoS ONE. 2014;9(4):e90375. pmid:24690612
  17. 17. Cappa F, Laut J, Nov O, Giustiniano L, Porfiri M. Activating social strategies: Face-to-face interaction in technology-mediated citizen science. Journal of Environmental Management. 2016;182:374–384. pmid:27498272
  18. 18. Laut J, Cappa F, Nov O, Porfiri M. Increasing citizen science contribution using a virtual peer. Journal of the Association for Information Science and Technology;.
  19. 19. Bütefisch C, Hummelsheim H, Denzler P, Mauritz KH. Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. Journal of the Neurological Sciences. 1995;130(1):59–68. pmid:7650532
  20. 20. Duncan PW, Horner RD, Reker DM, Samsa GP, Hoenig H, Hamilton B, et al. Adherence to postacute rehabilitation guidelines is associated with functional recovery in stroke. Stroke. 2002;33(1):167–178. pmid:11779907
  21. 21. Connolly TM, Boyle EA, MacArthur E, Hainey T, Boyle JM. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education. 2012;59(2):661–686.
  22. 22. O’Luanaigh P. Game design complete. Paraglyph Press; 2005.
  23. 23. Awad M, Ferguson S, Craig C. Designing games for older adults: an affordance based approach. In: IEEE 3rd International Conference on Serious Games and Applications for Health (SeGAH). IEEE; 2014. p. 1–7.
  24. 24. Flores E, Tobon G, Cavallaro E, Cavallaro FI, Perry JC, Keller T. Improving patient motivation in game development for motor deficit rehabilitation. In: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology. ACM; 2008. p. 381–384.
  25. 25. Cappa P, Clerico A, Nov O, Porfiri M. Can Force Feedback and Science Learning Enhance the Effectiveness of Neuro-Rehabilitation? An Experimental Study on Using a Low-Cost 3D Joystick and a Virtual Visit to a Zoo. PLoS ONE. 2013;8(12):e83945. pmid:24349562
  26. 26. Laut J, Cappa F, Nov O, Porfiri M. Increasing patient engagement in rehabilitation exercises using computer-based citizen science. PLoS ONE. 2015;10(3):e0117013. pmid:25793867
  27. 27. Gupta A, O’Malley MK. Design of a haptic arm exoskeleton for training and rehabilitation. IEEE/ASME Transactions on Mechatronics. 2006;11(3):280–289.
  28. 28. Malfait N, Ostry DJ. Is interlimb transfer of force-field adaptation a cognitive response to the sudden introduction of load? The Journal of Neuroscience. 2004;24(37):8084–8089. pmid:15371509
  29. 29. Patton JL, Mussa-Ivaldi FA. Robot-assisted adaptive training: custom force fields for teaching movement patterns. IEEE Transactions on Biomedical Engineering. 2004;51(4):636–646. pmid:15072218
  30. 30. Bouzit M, Burdea G, Popescu G, Boian R. The Rutgers Master II-new design force-feedback glove. IEEE/ASME Transactions on Mechatronics. 2002;7(2):256–263.
  31. 31. Masia L, Frascarelli F, Morasso P, Di Rosa G, Petrarca M, Castelli E, et al. Reduced short term adaptation to robot generated dynamic environment in children affected by Cerebral Palsy. Journal of Neuroengineering and Rehabilitation. 2011;8(28). pmid:21600031
  32. 32. Rastgaar M, Lee H, Ficanha E, Ho P, Krebs HI, Hogan N. Multi-directional Dynamic Mechanical Impedance of the Human Ankle; A Key to Anthropomorphism in Lower Extremity Assistive Robots. In: Neuro-Robotics. Springer; 2014. p. 157–178.
  33. 33. Kwakkel G, Kollen BJ, Krebs HI. Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabilitation and Neural Repair. 2008;22(2):111–121. pmid:17876068
  34. 34. Coote S, Murphy B, Harwin W, Stokes E. The effect of the GENTLE/s robot-mediated therapy system on arm function after stroke. Clinical Rehabilitation. 2008;22(5):395–405. pmid:18441036
  35. 35. Díaz I, Gil JJ, Sánchez E. Lower-limb robotic rehabilitation: literature review and challenges. Journal of Robotics. 2011;2011, Article ID 759764.
  36. 36. Patanè F, Cappa P. A 3-DOF parallel robot with spherical motion for the rehabilitation and evaluation of balance performance. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2011;19(2):157–166. pmid:20977987
  37. 37. Rossi S, Patane F, Del Sette F, Cappa P. WAKE-up: a wearable ankle knee exoskeleton. In: 5th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics. IEEE; 2014. p. 504–507.
  38. 38. Dong S, Lu KQ, Sun JQ, Rudolph K. Adaptive force regulation of muscle strengthening rehabilitation device with magnetorheological fluids. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2006;14(1):55–63. pmid:16562632
  39. 39. Bach-y Rita P, Wood S, Leder R, Paredes O, Bahr D, Bach-y Rita EW, et al. Computer-assisted motivating rehabilitation (CAMR) for institutional, home, and educational late stroke programs. Topics in Stroke Rehabilitation. 2002;8(4):1–10. pmid:14523726
  40. 40. Johnson MJ. Recent trends in robot-assisted therapy environments to improve real-life functional performance after stroke. Journal of NeuroEngineering and Rehabilitation. 2006;3:29. pmid:17176474
  41. 41. Morrow K, Docan C, Burdea G, Merians A. Low-cost virtual rehabilitation of the hand for patients post-stroke. In: IEEE International Workshop on Virtual Rehabilitation; 2006. p. 6–10.
  42. 42. Attygalle S, Duff M, Rikakis T, He J. Low-cost, at-home assessment system with Wii Remote based motion capture. In: IEEE Virtual Rehabilitation; 2008. p. 168–174.
  43. 43. Charles J, Gordon AM. A critical review of constraint-induced movement therapy and forced use in children with hemiplegia. Neural plasticity. 2005;12(2–3):245–261. pmid:16097492
  44. 44. Saposnik G, Teasell R, Mamdani M, Hall J, McIlroy W, Cheung D, et al. Effectiveness of virtual reality using Wii gaming technology in stroke rehabilitation a pilot randomized clinical trial and proof of principle. Stroke. 2010;41(7):1477–1484. pmid:20508185
  45. 45. Han J, Shao L, Xu D, Shotton J. Enhanced computer vision with microsoft kinect sensor: a review. IEEE Transactions on Cybernetics. 2013;43(5):1318–1334. pmid:23807480
  46. 46. Parker J. Buttons, simplicity, and natural interfaces. Loading. 2008;2(2).
  47. 47. Mitra S, Acharya T. Gesture recognition: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews. 2007;37(3):311–324.
  48. 48. Phamduy P, DeBellis M, Porfiri M. Controlling a Robotic Fish Via a Natural User Interface for Informal Science Education. IEEE Transactions on Multimedia. 2015;17(12):2328–2337.
  49. 49. Lange B, Chang CY, Suma E, Newman B, Rizzo AS, Bolas M. Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC; 2011. p. 1831–1834.
  50. 50. Webster D, Celik O. Systematic review of Kinect applications in elderly care and stroke rehabilitation. Journal of Neuroengineering and Rehabilitation. 2014;11(108).
  51. 51. Kitsunezaki N, Adachi E, Masuda T, Mizusawa Ji. Kinect applications for the physical rehabilitation. In: IEEE International Symposium on Medical Measurements and Applications Proceedings (MeMeA); 2013. p. 294–299.
  52. 52. Fern’ndez-Baena A, Susín A, Lligadas X. Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments. In: 4th IEEE International Conference on Intelligent Networking and Collaborative Systems (INCoS); 2012. p. 656–661.
  53. 53. González A, Hayashibe M, Bonnet V, Fraisse P. Whole Body Center of Mass Estimation with Portable Sensors: using the Statically Equivalent Serial Chain and a Kinect. Sensors. 2014;14(9):16955–16971. pmid:25215943
  54. 54. Microsoft. Kinect SDK;. http://msdn.microsoft.com/en-us/library/hh855347.aspx.
  55. 55. Obdrzalek S, Kurillo G, Ofli F, Bajcsy R, Seto E, Jimison H, et al. Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2012. p. 1188–1193.
  56. 56. Taub E, Griffin A, Uswatte G, Gammons K, Nick J, Law CR. Treatment of congenital hemiparesis with pediatric constraint-induced movement therapy. Journal of Child Neurology. 2011;26(9):1163–1173. pmid:21771948
  57. 57. Taub E, Uswatte G, Mark VW, Morris DM, Barman J, Bowman MH, et al. Method for enhancing real-world use of a more affected arm in chronic stroke transfer package of constraint-induced movement therapy. Stroke. 2013;44(5):1383–1388. pmid:23520237
  58. 58. Takahashi C, Scheidt RA, Reinkensmeyer D. Impedance control and internal model formation when reaching in a randomly varying dynamical environment. Journal of Neurophysiology. 2001;86(2):1047–1051. pmid:11495973
  59. 59. NYU. Brooklyn Atlantis;. http://www.brooklynatlantis.org.
  60. 60. Allen IE, Seaman CA. Likert scales and data analyses. Quality Progress. 2007;40(7):64–65.
  61. 61. Armstrong RL. The midpoint on a five-point Likert-type scale. Perceptual and Motor Skills. 1987;64(2):359–362.
  62. 62. Wu G, Van Der Helm FC, Veeger H, Makhsous M, Van Roy P, Anglin C, et al. ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion—Part II: shoulder, elbow, wrist and hand. Journal of Biomechanics. 2005;38(5):981–992. pmid:15844264
  63. 63. Pacilli A, Germanotta M, Rossi S, Cappa P. Quantification of age-related differences in reaching and circle-drawing using a robotic rehabilitation device. Applied Bionics and Biomechanics. 2014;11(3):91–104.
  64. 64. Highland TR, Dreisinger TE, Vie LL, Russell GS. Changes in isometric strength and range of motion of the isolated cervical spine after eight weeks of clinical rehabilitation. Spine. 1992;17(6):S77–S82. pmid:1631723
  65. 65. Celik O, O’Malley MK, Boake C, Levin HS, Yozbatiran N, Reistetter TA. Normalized movement quality measures for therapeutic robots strongly correlate with clinical motor impairment measures. IEEE Transaction on Neural Systems and Rehabilitation Engineering. 2010;18(4):433–444.
  66. 66. Rohrer B, Fasoli S, Krebs HI, Hughes R, Volpe B, Frontera WR, et al. Movement smoothness changes during stroke recovery. The Journal of Neuroscience. 2002;22(18):8297–8304. pmid:12223584
  67. 67. Dipietro L, Krebs HI, Fasoli SE, Volpe BT, Hogan N. Submovement changes characterize generalization of motor recovery after stroke. Cortex. 2009;45(3):318–324. pmid:18640668
  68. 68. Pett MA. Nonparametric statistics in health care research: Statistics for small samples and unusual distributions. Sage; 1997.
  69. 69. Casella G, Berger RL. Statistical inference. vol. 2. Duxbury Pacific Grove, CA; 2002.
  70. 70. Massey FJ Jr. The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association. 1951;46(253):68–78.
  71. 71. Gregersen A. Genre, technology and embodied interaction: The evolution of digital game genres and motion gaming. MedieKultur Journal of Media and Communication Research. 2011;27(51):94–109.
  72. 72. Crowley DN, Breslin JG, Corcoran P, Young K. Gamification of citizen sensing through mobile social reporting. In: IEEE International Games Innovation Conference (IGIC); 2012. p. 1–5.
  73. 73. Bowser A, Hansen D, Preece J, He Y, Boston C, Hammock J. Gamifying citizen science: a study of two user groups. In: Proceedings of the Companion Publication of the 17th ACM Conference on Computer Supported cooperative Work & Social Computing; 2014. p. 137–140.
  74. 74. Nov O, Anderson D, Arazy O. Volunteer computing: a model of the factors determining contribution to community-based scientific research. In: Proceedings of the 19th International Conference on World Wide Web. ACM; 2010. p. 741–750.
  75. 75. Nov O, Arazy O, Anderson D. Dusting for science: motivation and participation of digital citizen science volunteers. In: Proceedings of the 2011 iConference. ACM; 2011. p. 68–74.
  76. 76. Burton S, Blair E. Task conditions, response formulation processes, and response accuracy for behavioral frequency questions in surveys. Public Opinion Quarterly. 1991;55(1):50–79.
  77. 77. Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. The Lancet. 2011;377(9778):1693–1702.
  78. 78. Johansson B. Current trends in stroke rehabilitation. A review with focus on brain plasticity. Acta Neurologica Scandinavica. 2011;123(3):147–159. pmid:20726844
  79. 79. Gerling K, Livingston I, Nacke L, Mandryk R. Full-body motion-based game interaction for older adults. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2012. p. 1873–1882.
  80. 80. Da Gama A, Chaves T, Figueiredo L, Teichrieb V. Poster: improving motor rehabilitation process through a natural interaction based system using kinect sensor. In: 2012 IEEE Symposium on 3D User Interfaces (3DUI); 2012. p. 145–146.
  81. 81. Saini S, Rambli DRA, Sulaiman S, Zakaria MN, Shukri SM. A low-cost game framework for a home-based stroke rehabilitation system. In: International Conference on Computer & Information Science (ICCIS), 2012. vol. 1; 2012. p. 55–60.
  82. 82. Acar M, Karatas GK. The effect of arm sling on balance in patients with hemiplegia. Gait & Posture. 2010;32(4):641–644.
  83. 83. Chang CM, Chang YC, Chang HY, Chou LW. An interactive game-based shoulder wheel system for rehabilitation. Patient Preference and Adherence. 2012;6:821–828. pmid:23226005
  84. 84. Lloyd-Jones D, Adams RJ, Brown TM, Carnethon M, Dai S, De Simone G, et al. Heart disease and stroke statistics–2010 update A report from the American Heart Association. Circulation. 2010;121(7):e46–e215. pmid:20019324
  85. 85. Holden MK, Dyar TA, Dayan-Cimadoro L. Telerehabilitation using a virtual environment improves upper extremity function in patients with stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2007;15(1):36–42. pmid:17436874
  86. 86. Volpe BT, Krebs HI, Hogan N. Is robot-aided sensorimotor training in stroke rehabilitation a realistic option? Current Opinion in Neurology. 2001;14(6):745–752. pmid:11723383
  87. 87. Patritti BL, Straudi S, Deming LC, Benedetti MG, Nimec DL, Bonato P. Robotic gait training in an adult with cerebral palsy: a case report. PM&R. 2010;2(1):71–75.
  88. 88. Liao WW, Wu CY, Hsieh YW, Lin KC, Chang WY. Effects of robot-assisted upper limb rehabilitation on daily function and real-world arm activity in patients with chronic stroke: a randomized controlled trial. Clinical Rehabilitation. 2012;26(2):111–120. pmid:21840917
  89. 89. Gladstone DJ, Danells CJ, Black SE. The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabilitation and Neural Repair. 2002;16(3):232–240. pmid:12234086
  90. 90. Stewart KC, Cauraugh JH, Summers JJ. Bilateral movement training and stroke rehabilitation: a systematic review and meta-analysis. Journal of the Neurological Sciences. 2006;244(1):89–95. pmid:16476449
  91. 91. Swinnen SP. Intermanual coordination: from behavioural principles to neural-network interactions. Nature Reviews Neuroscience. 2002;3(5):348–359. pmid:11988774
  92. 92. Lukowicz P, Hanser F, Szubski C, Schobersberger W. Detecting and interpreting muscle activity with wearable force sensors. In: International Conference on Pervasive Computing. Springer; 2006. p. 101–116.
  93. 93. Liu T, Inoue Y, Shibata K. A small and low-cost 3-D tactile sensor for a wearable force plate. IEEE Sensors Journal. 2009;9(9):1103–1110.
  94. 94. Zhang W, Tomizuka M, Byl N. A wireless human motion monitoring system for smart rehabilitation. Journal of Dynamic Systems, Measurement, and Control. 2016;138(11):111004.
  95. 95. Tao W, Liu T, Zheng R, Feng H. Gait analysis using wearable sensors. Sensors. 2012;12(2):2255–2283. pmid:22438763
  96. 96. Bamberg SJM, Benbasat AY, Scarborough DM, Krebs DE, Paradiso JA. Gait analysis using a shoe-integrated wireless sensor system. IEEE Transactions on Information Technology in Biomedicine. 2008;12(4):413–423. pmid:18632321
  97. 97. Liu T, Inoue Y, Shibata K. A wearable ground reaction force sensor system and its application to the measurement of extrinsic gait variability. Sensors. 2010;10(11):10240–10255. pmid:22163468