BiDi screen : depth and lighting aware interaction and display

In this thesis, I describe a new type of interactive display that supports both on-screen multi-touch interactions and off-screen hover-based gestures. This BiDirectional (BiDi) screen, capable of both image capture and display, is inspired by emerging LCDs that use embedded optical sensors to detect multiple points of direct contact. The key contribution of this thesis is to exploit the spatial light modulation capability of LCDs to allow dynamic mask-based scene capture without interfering with display functionality. A large-format image sensor is placed slightly behind the liquid crystal layer. By alternatly switching the liquid crystal between a display mode showing traditional graphics and a capture mode in which the backlight is disabled and a pinhole array or an equivalent tiled-broadband code is displayed, the BiDi Screen can recover multi-view orthographic imagery while functioning as a 2D display. The recovered imagery is used to passively estimate the depth of scene points from focus. I discuss the design and construction of a prototype to demonstrate these capabilities in two motivating applications: a hybrid touch plus gesture interaction and a light-gun mode for interacting with external light-emitting widgets. The working prototype simulates the large format light sensor with a camera and diffuser, supporting interaction up to 50 cm in front of a modified 20.1 inch LCD.

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Architecture. Program in Media Arts and Sciences.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2009

Author dc:creator

• Hirsch, Matthew Waggener

Advisor dc:contributor.advisor

• Henry Holtzman.

• Architecture. Program in Media Arts and Sciences.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/55261

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/55261