Finite-state canonicalization techniques for historical German

This work addresses issues in the automatic preprocessing of historical German input text for use by conventional natural language processing techniques. Conventional techniques cannot adequately account for historical input text due to conventional tools' reliance on a fixed application-specific lexicon keyed by contemporary orthographic surface form on the one hand, and the lack of consistent orthographic conventions in historical input text on the other. Historical spelling variation is treated here as an error-correction problem or "canonicalization" task: an attempt to automatically assign each (historical) input word a unique extant canonical cognate, thus allowing direct application-specific processing (tagging, parsing, etc.) of the returned canonical forms without need for any additional application-specific modifications. In the course of the work, various methods for automatic canonicalization are investigated and empirically evaluated, including conflation by phonetic identity, conflation by lemma instantiation heuristics, canonicalization by weighted finite-state rewrite cascade, and token-wise disambiguation by a dynamic Hidden Markov Model.

Level thesis:degree_level

thesis.doctoral

Grantor dc:publisher

Universität Potsdam

Year

2011

Author dc:creator

• Jurish, Bryan

Contributors dc:contributor

• Staudacher, Peter

• Computerlinguistik
• Orthographie
• historischer Text
• Rechtschreibkorrektur
• computational linguistics
• orthography
• historical text
• spelling correction

Statement dc:rights

• Keine öffentliche Lizenz: Unter Urheberrechtsschutz

Repository record source_url

https://publishup.uni-potsdam.de/frontdoor/index/index/docId/5562

OAI identifier oai:identifier

oai:kobv.de-opus4-uni-potsdam:5562