Density estimation with Kullback-Leibler loss

Probability density functions are estimated by the method of maximum likelihood in sequences of regular exponential families. The approximation families of log-densities that we consider are polynomials, splines, and trigonometric series. Bounds on the relative entropy (Kullback-Leibler number) between the true density and the estimator are obtained and rates of convergence are established for log-density functions assumed to have square integrable derivatives. The relative entropy risk between true probability density function and the estimator is shown to converge to zero at a desired rate. The idea is to select n samples from the true distribution and choose the estimator which is the maximum posterior likelihood estimator in certain regular m-parameter exponential families, given that a Gaussian distribution is the prior on the parameter space. The implications for universal source coding and portfolio selection are discussed.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Statistics

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2011

Author dc:creator

• Sheu, Chyong-Hwa

Contributors dc:contributor

• Barron, Andrew

• Statistics

Statement dc:rights

• Copyright 1990 Sheu, Chyong-Hwa

Language dc:language

eng

Identifier

AAI9026321
(UMI)AAI9026321

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/20798