Infixer: A Method for Segmenting Non-Concatenative Morphology in Tagalog

<p>In this paper, I present a method for coercing a widely-used morphological segmentation algorithm, Morfessor (Creutz and Lagus 2005), into accurately segmenting non-concatenative morphological patterns. The non-concatenative patterns targeted—infixation and partial-reduplication—present problems for many segmentation algorithms, and tools that can successfully identify and segment those patterns can improve a number of downstream natural language processing tasks, including keyword search and machine translation.</p> <p>Included with this project is an implementation of the segmentation method described in the form of a Python library called Infixer. This approach involves a preprocessing step that re- structures non-concatenative patterns into concatenative ones using regular expressions, which allows the algorithm to operate on the input data as it would any other data. The target language for this project is Tagalog, a major language of the Philippines that makes extensive use of non- concatenative morphology, and the training data is built from data from the IARPA Babel Program and the Tagalog Wikipedia.</p> <p>The results for this test were promising, especially for the more straightforward cases of infixation tested. For the data tested, the Infixer implementation using affix regular expressions showed performance gains over those without, demonstrating an improved ability to segment data containing non-concatenative morphological forms. In the future it is hoped that this project can lead to the development of tools that can be used effectively on languages besides Tagalog and on a more diverse array of phenomena.</p>

Name thesis:degree_name

Master of Arts

Level thesis:degree_level

Master

Discipline thesis:degree_discipline

Linguistics

Grantor

The Graduate School and University Center of The City University of New York

Year dc:date.available

2016

Author dc:creator

• Butler, Steven R

Advisor dc:contributor.advisor

• William Sakas

• morphological segmentation
• Tagalog
• low-resource languages
• infixation

Repository record dc:identifier

https://academicworks.cuny.edu/gc_etds/1308

OAI identifier oai:identifier

oai:academicworks.cuny.edu:gc_etds-2344