Motion sculptures : automating artistic visualization of shape and time

We present a method for automatically visualizing complex human and object motion via 3D motion sculptures -- a representation that conveys the 3D structure swept by the human's body as it moves through space. Given only a monocular RGB video as input, our algorithm computes the motion sculpture by estimating the object's 3D geometry over time, and then renders the motion sculpture under different rendering styles such as sculpture material, scene lighting, and floor reflections. We develop a 3D-aware image-based rendering approach to insert motion sculptures into a synthetic scene or back into the original video. This results in high-quality artistic visualizations of motion. Because motion sculptures are 3D, they can be viewed from arbitrary viewpoints and even physically printed. As such, they may reveal space-time information that is undetected by the naked eyes and allow the viewer to interpret how different parts of the object interact over time. Our method automates the process of motion sculpture creation, making this manual process typically done only by professional artists accessible to novice users and applicable to standard videos. We show results on various scenes involving complex motions such as sports actions and dancing.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2018

Author dc:creator

• Zhang, Xiuming (Scientist in electrical engineering and computer science) Massachusetts Institute of Technology

Advisor dc:contributor.advisor

• William T. Freeman.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/117815

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/117815