Non-Invasive Vision-Based Measurement of Hand Kinematics and Interaction

The ability to manipulate and interact with the world is a key part of what distinguishes humanity from other animals, and the human hand is perhaps the most powerful tool we have to do so. As such, a great deal of research goes towards better understanding how the hand behaves during interaction tasks. Study of physical interaction requires measurement of hand kinematics and interaction forces. Unfortunately, current methods involve cumbersome sensors or external forces that inherently change the way that the subject behaves. In order to avoid these confounding factors, this thesis presents an approach to measuring hand kinematics, dynamics, and physical interaction using a non-encumbering vision-based tool. The proposed tool consists of (1) Vision-based hand tracking of hand kinematics in joint space (2) Synergy extraction and synergy space projection (3) Visual soft tissue deformation based contact detection (4) An exploration of force estimation at the fingertips. This pipeline is applied to a piano-based experiment for validation and comparison with existing tools. The results indicate that vision-based kinematics measurement is largely comparable to and at times shows more sensitivity to joint angle variation than traditionally instrumented approaches. However, force estimation is not yet a consistent alternative to physical sensor interfaces.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2023

Author dc:creator

• Wang, Margaret

Advisor dc:contributor.advisor

• Hogan, Neville

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/151615

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/151615