Statistical Learning with Discrete Structures: Statistical and Computational Perspectives

In various statistical tasks it is of interest to learn estimators with discrete structures (e.g., sparsity, low-rank, shared model parameters, etc)---they are appealing for their interpretability and compactness. However, learning with discrete structures can be computationally challenging. In this thesis, we explore statistical and computational aspects of statistics estimators (some classical and some new) that can be formulated as discrete optimization problems. In Chapters 2 and 3, we study two well-known problems in high-dimensional statistics: sparse Principal Component Analysis (PCA) and Gaussian Graphical Models. These are notoriously hard optimization problems---we explore computationally friendlier estimators based on Mixed Integer Programming (MIP) under suitable statistical assumptions. We study the statistical and computational properties of our estimators. In the fourth chapter, we study the multi-task learning problem with sparse linear estimators. Motivated by applications in biomedical sciences, we propose a new modeling framework to jointly learn sparse linear estimators for different tasks by sharing support information. Our theoretical results show that our joint estimation framework can lead to better statistical properties compared to independently fitting models for each task. We develop scalable approximate solvers for our MIP-based formulation. In the fifth chapter, we study the problem of sparse Nonnegative Matrix Factorization (NMF) using Cutler and Breiman's archetypal regularization. We explore the utility of our methods in the context of applications in the biomedical sciences, computer vision and computational finance.

Name thesis:degree_name

Doctoral

Department dc:contributor.department

Massachusetts Institute of Technology. Operations Research Center

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Behdin, Kayhan

Advisor dc:contributor.advisor

• Mazumder, Rahul

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/155499

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/155499