A taxonomy of situated language in natural contexts

This thesis develops a multi-modal dataset consisting of transcribed speech along with the locations in which that speech took place. Speech with location attached is called situated language, and is represented here as spatial distributions, or two-dimensional histograms over locations in a home. These histograms are organized in the form of a taxonomy, where one can explore, compare, and contrast various slices along several axes of interest. This dataset is derived from raw data collected as part of the Human Speechome Project, and consists of semi-automatically transcribed spoken language and time-aligned overhead video collected over 15 months in a typical home environment. As part of this thesis, the vocabulary of the child before the age of two is derived from transcription, as well as the age at which the child first produced each of the 658 words in his vocabulary. Locations are derived using an efficient tracking algorithm, developed as part of this thesis, called 2C. This system maintains high accuracy when compared to similar systems, while dramatically reducing processing time, an essential feature when processing a corpus of this size. Spatial distributions are produced for many different cuts through the data, including temporal segments (i.e. morning, day, and night), speaker identities (i.e. mother, father, child), and linguistic content (i.e. per-word, aggregate by word type). Several visualization types and statistics are developed, which prove useful for organizing and exploring the dataset. It will then be shown that spatial distributions contain a wealth of information, and that this information can be exploited in various ways to derive meaningful insights and numerical results from the data.

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Architecture. Program in Media Arts and Sciences.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2011

Author dc:creator

• Shaw, George Macaulay

Advisor dc:contributor.advisor

• Deb Roy.

• Architecture. Program in Media Arts and Sciences.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/69808

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/69808