Approximate nearest neighbor problem in high dimensions

We investigate the problem of finding the approximate nearest neighbor when the data set points are the substrings of a given text T. The exact version of this problem is defined as follows. Given a text T of length n, we want to build a data structure that supports the following operation: given a pattern P, find the substring of T that is the closest to P. Since the exact version of this problem is surprisingly difficult, we address the approximate version, in which we are allowed to return a substring of T that is at most c times further than the actual closest substring of T. This problem occurs, for example, in computational biology [4, 5]. In particular, we study the case where the length of the pattern P, denoted by m, is not known in advance, which is the most natural scenario. We present a data structure that uses O(n1+1/c) space and has 0 (nl/c + mn⁰(l)) query time' when the distance between two strings is the Hamming distance. These bounds essentially match the earlier bounds of [12], which assumed that the pattern length m is fixed in advance. Furthermore, our data structure can be constructed in O (n1+1/c + n1+⁰(1)M1/3) time, where M is an upper bound for m. This time essentially matches the preprocessing time of [12] as long as the term O(n1+1/c) dominates the running time, which is the case when, for example, c < 3. We also extend our results to the case where the distances are measured according to the lI distance. The query time and the space bound are essentially the same, while the preprocessing time becomes 6 (n'+/c + nl+o(l)M2/3) (We use notation f(n) = O(g(n)) to denote f(n) = g(n) logO(1) g(n)).

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2005

Author dc:creator

• Andoni, Alexandr

Advisor dc:contributor.advisor

• Piotr Indyk.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/33106

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/33106