Embodied object schemas for grounding language use

This thesis presents the Object Schema Model (OSM) for grounded language interaction. Dynamic representations of objects are used as the central point of coordination between actions, sensations, planning, and language use. Objects are modeled as object schemas -- sets of multimodal, object-directed behavior processes -- each of which can make predictions, take actions, and collate sensations, in the modalities of touch, vision, and motor control. This process-centered view allows the system to respond continuously to real-world activity, while still viewing objects as stabilized representations for planning and speech interaction. The model can be described from four perspectives, each organizing and manipulating behavior processes in a different way. The first perspective views behavior processes like thread objects, running concurrently to carry out their respective functions. The second perspective organizes the behavior processes into object schemas. The third perspective organizes the behavior processes into plan hierarchies to coordinate actions. The fourth perspective creates new behavior processes in response to language input.

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Architecture. Program In Media Arts and Sciences

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2007

Author dc:creator

• Hsiao, Kai-yuh, 1977-

Advisor dc:contributor.advisor

• Deb K. Roy.

• Architecture. Program In Media Arts and Sciences

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/39258
• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Identifier URI

http://dspace.mit.edu/handle/1721.1/39258

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/39258