ArmTrak: tracking arm postures with a smartwatch

We aim to track the 3D posture of the entire arm - both wrist and elbow - using the motion and magnetic sensors on smartwatches. We do not intend to employ machine learning to train the system on a specific set of gestures. Instead, we aim to trace the geometric motion of the arm, which can then be used as a generic platform for gesture-based applications. The problem is challenging because the arm posture is a function of both elbow and shoulder motions, whereas the watch is only a single point of (noisy) measurement from the wrist. Moreover, while other tracking systems (like indoor/outdoor localization) often benefit from maps or landmarks to occasionally reset their estimates, such opportunities are almost absent here. While this appears to be an under-constrained problem, we find that the pointing direction of the forearm is strongly coupled to the arm's posture. If the gyroscope and compass on the watch can be made to estimate this direction, the 3D search space can become smaller; the IMU sensors can then be applied to mitigate the remaining uncertainty. We leverage this observation to design ArmTrak, a system that fuses the IMU sensors and the anatomy of arm joints into a modified hidden Markov model (HMM) to continuously estimate state variables. Using Kinect 2.0 as ground truth, we achieve around 9.2 cm of median error for free-form postures; the errors increase to 13.3 cm for a real-time version. We believe this is a step forward in posture tracking, and with some additional work, could become a generic underlay to various practical applications.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2017

Author dc:creator

• Shen, Sheng

Contributors dc:contributor

• Choudhury, Romit Roy

• Gesture
• Tracking
• Smartwatch

Statement dc:rights

• Copyright 2017 Sheng Shen

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/97546

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/97546