{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/151535"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/151535","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Topics in Sparsity and Compression: From High dimensional statistics to Overparametrized Neural Networks","abstract":"This thesis presents applications of sparsity in three different areas: covariance estimation in time-series data, linear regression with categorical variables, and neural network compression. In the first chapter, motivated by problems in computational finance, we consider a framework for jointly learning time-varying covariance matrices under different structural assumptions (e.g., low-rank, sparsity or a combination of both). We propose novel algorithms for learning these covariance matrices simultaneously across all time blocks and show improved computational efficiency and performance across different tasks. In the second chapter, we study the problem of linear regression with categorical variables, where every categorical variable can have a large number of levels. We seek to reduce or cluster the number of levels for statistical and interpretability reasons. To this end, we propose a new estimator and study its computational and statistical properties. And in the third chapter, we explore the problem of pruning or sparsifying the weights of a neural network. Modern neural networks tend to have a large number of parameters, which makes their storage and deployment expensive, especially in resource-constrained environments. One solution to this is compressing the network by pruning or removing some parameters, while trying to maintain a similar level of performance compared to the dense network. To achieve this, we propose a new optimization-based pruning algorithm, and show how it leads to significantly better sparsity-accuracy trade-offs compared to existing pruning methods.","abstract_html":"This thesis presents applications of sparsity in three different areas: covariance estimation in time-series data, linear regression with categorical variables, and neural network compression. In the first chapter, motivated by problems in computational finance, we consider a framework for jointly learning time-varying covariance matrices under different structural assumptions (e.g., low-rank, sparsity or a combination of both). We propose novel algorithms for learning these covariance matrices simultaneously across all time blocks and show improved computational efficiency and performance across different tasks. In the second chapter, we study the problem of linear regression with categorical variables, where every categorical variable can have a large number of levels. We seek to reduce or cluster the number of levels for statistical and interpretability reasons. To this end, we propose a new estimator and study its computational and statistical properties. And in the third chapter, we explore the problem of pruning or sparsifying the weights of a neural network. Modern neural networks tend to have a large number of parameters, which makes their storage and deployment expensive, especially in resource-constrained environments. One solution to this is compressing the network by pruning or removing some parameters, while trying to maintain a similar level of performance compared to the dense network. To achieve this, we propose a new optimization-based pruning algorithm, and show how it leads to significantly better sparsity-accuracy trade-offs compared to existing pruning methods.","abstract_has_math":false,"creators":["Benbaki, Riade"],"institution":"Massachusetts Institute of Technology","degree_name":"Master","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Operations Research Center","school":null,"contributors":[],"advisors":["Mazumder, Rahul"],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-06","date_published":"2023-06","updated_at":"2026-07-22T22:21:32Z","subjects":[],"languages":[],"rights":["Attribution 4.0 International (CC BY 4.0)","Copyright retained by author(s)"],"rights_urls":["https://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/151535","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Mazumder, Rahul"]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Operations Research Center"]},{"key":"dc:creator","label":"Author","values":["Benbaki, Riade"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2023-07-31T19:46:56Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2023-07-31T19:46:56Z"]},{"key":"dc:date.issued","label":"Date","values":["2023-06"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master","Master of Science in Operations Research"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["Attribution 4.0 International (CC BY 4.0)","Copyright retained by author(s)"]},{"key":"dc:rights.uri","label":"Rights URI","values":["https://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/151535"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This thesis presents applications of sparsity in three different areas: covariance estimation in time-series data, linear regression with categorical variables, and neural network compression. In the first chapter, motivated by problems in computational finance, we consider a framework for jointly learning time-varying covariance matrices under different structural assumptions (e.g., low-rank, sparsity or a combination of both). We propose novel algorithms for learning these covariance matrices simultaneously across all time blocks and show improved computational efficiency and performance across different tasks. In the second chapter, we study the problem of linear regression with categorical variables, where every categorical variable can have a large number of levels. We seek to reduce or cluster the number of levels for statistical and interpretability reasons. To this end, we propose a new estimator and study its computational and statistical properties. And in the third chapter, we explore the problem of pruning or sparsifying the weights of a neural network. Modern neural networks tend to have a large number of parameters, which makes their storage and deployment expensive, especially in resource-constrained environments. One solution to this is compressing the network by pruning or removing some parameters, while trying to maintain a similar level of performance compared to the dense network. To achieve this, we propose a new optimization-based pruning algorithm, and show how it leads to significantly better sparsity-accuracy trade-offs compared to existing pruning methods."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["S.M."]},{"key":"dc:title","label":"Title","values":["Topics in Sparsity and Compression: From High dimensional statistics to Overparametrized Neural Networks"]}]}],"canonical_facts":{"dc:contributor.advisor":["Mazumder, Rahul"],"dc:contributor.department":["Massachusetts Institute of Technology. Operations Research Center"],"dc:creator":["Benbaki, Riade"],"dc:date.accessioned":["2023-07-31T19:46:56Z"],"dc:date.available":["2023-07-31T19:46:56Z"],"dc:date.issued":["2023-06"],"dc:description.abstract":["This thesis presents applications of sparsity in three different areas: covariance estimation in time-series data, linear regression with categorical variables, and neural network compression. In the first chapter, motivated by problems in computational finance, we consider a framework for jointly learning time-varying covariance matrices under different structural assumptions (e.g., low-rank, sparsity or a combination of both). We propose novel algorithms for learning these covariance matrices simultaneously across all time blocks and show improved computational efficiency and performance across different tasks. In the second chapter, we study the problem of linear regression with categorical variables, where every categorical variable can have a large number of levels. We seek to reduce or cluster the number of levels for statistical and interpretability reasons. To this end, we propose a new estimator and study its computational and statistical properties. And in the third chapter, we explore the problem of pruning or sparsifying the weights of a neural network. Modern neural networks tend to have a large number of parameters, which makes their storage and deployment expensive, especially in resource-constrained environments. One solution to this is compressing the network by pruning or removing some parameters, while trying to maintain a similar level of performance compared to the dense network. To achieve this, we propose a new optimization-based pruning algorithm, and show how it leads to significantly better sparsity-accuracy trade-offs compared to existing pruning methods."],"dc:description.degree":["S.M."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/151535"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["Attribution 4.0 International (CC BY 4.0)","Copyright retained by author(s)"],"dc:rights.uri":["https://creativecommons.org/licenses/by/4.0/"],"dc:title":["Topics in Sparsity and Compression: From High dimensional statistics to Overparametrized Neural Networks"],"dc:type":["Thesis"],"thesis:degree_name":["Master","Master of Science in Operations Research"]},"updated_at":"2026-07-22T22:21:32Z"}