A robust optimization approach to statistical estimation problems by Apostolos G. Fertis.

There have long been intuitive connections between robustness and regularization in statistical estimation, for example, in lasso and support vector machines. In the first part of the thesis, we formalize these connections using robust optimization. Specifically (a) We show that in classical regression, regularized estimators like lasso can be derived by applying robust optimization to the classical least squares problem. We discover the explicit connection between the size and the structure of the uncertainty set used in the robust estimator, with the coefficient and the kind of norm used in regularization. We compare the out-of-sample performance of the nominal and the robust estimators in computer generated and real data. (b) We prove that the support vector machines estimator is also a robust estimator of some nominal classification estimator (this last fact was also observed independently and simultaneously by Xu, Caramanis, and Mannor [52]). We generalize the support vector machines estimator by considering several sizes and structures for the uncertainty sets, and proving that the respective max-min optimization problems can be expressed as regularization problems. In the second part of the thesis, we turn our attention to constructing robust maximum likelihood estimators. Specifically (a) We define robust estimators for the logistic regression model, taking into consideration uncertainty in the independent variables, in the response variable, and in both. We consider several structures for the uncertainty sets, and prove that, in all cases, they lead to convex optimization problems. We provide efficient algorithms to compute the estimates in all cases.

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2009

Author dc:creator

• Fertis, Apostolos

Advisor dc:contributor.advisor

• Dimitris J. Bertsimas.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/53270

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/53270