3D Hand Pose Estimation Via a Lightweight Deep Learning Model

Deep Learning with depth cameras has enabled 3D hand pose estimation from RGBD images. Commercial solutions like Leap Motion and Intel RealSense™ use stereoscopic sensors or IR illumination-based methods to capture the depth in a photograph and further estimate pose using Deep Learning (DL) methods. These hand pose estimation work has not considered the use of virtual reality (VR) apps on mobile devices because this requires extensive computational resources including hardware for processing the acquired depth. Previous works in 3D hand pose estimation are based on the large pre-trained DL models in the pose estimation pipeline which are not suitable to run on mobile devices. In this work, we address the problem of hand pose estimation from monocular RGB images (instead of RGBD images) and making DL models suitable to run on mobile VR. This task is so challenging due to the missing depth information, we propose a deep neural network (DNN) that learns a 3D hand articulated prior to estimating the 3D pose from RGB images. Our approach comprises of (1) Localization network predicts the location of hands in the image, (2) sparse adversarial auto-encoders trained on hand RGB images, and (3) adversarial auto-encoder for capturing 3D hand pose distributions. Finally, the proposed model yielded the accuracy comparable to state-of-the-art 3D hand pose estimation. However, our model is much smaller than the existing models so that we significantly accelerated the model execution and greatly reduced the run-time 2.6X faster than the current solutions.

Name thesis:degree_name

M.S. (Master of Science)

Level thesis:degree_level

Masters

Discipline thesis:degree_discipline

Computer Science (UMKC)

Grantor dc:publisher

University of Missouri -- Kansas City

Year dc:date.issued

2018

Author dc:creator

• Suggala, Prudhvi Sai

Advisor dc:contributor.advisor

• Lee, Yugyung, 1960-

Handle dc:identifier.uri

https://hdl.handle.net/10355/65996

OAI identifier oai:identifier

oai:mospace.umsystem.edu:10355/65996