A multimodal Ouija board for aircraft carrier deck operations

In this thesis, we present improvements to DeckAssistant, a system that provides a traditional Ouija board interface by displaying a digital rendering of an aircraft carrier deck that assists deck handlers in planning deck operations. DeckAssistant has a large digital tabletop display that shows the status of the deck and has an understanding of certain deck actions for scenario planning. To preserve the conventional way of interacting with the old-school Ouija board where deck handlers move aircraft by hand, the system takes advantage of multiple modes of interaction. Deck handlers plan strategies by pointing at aircraft, gesturing and talking to the system. The system responds with its own speech and gestures, and it updates the display to show the consequences of the actions taken by the handlers. The system can also be used to simulate certain scenarios during the planning process. The multimodal interaction described here creates a communication of sorts between deck handlers and the system. Our contributions include improvements in hand-tracking, speech synthesis and speech recognition.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2016

Author dc:creator

• Uzun, Birkan

Advisor dc:contributor.advisor

• Randall Davis.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/106388

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/106388