{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/113893"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/113893","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"The role of explicit regularization in overparameterized neural networks","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-04-06 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-04-06 without embargo terms","abstract_has_math":false,"creators":["Liang, Shiyu"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Srikant, Rayadurgam","Viswanath, Pramod","Raginsky, Maxim","Sun, Ruoyu","Lee, Jason D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-04-29T21:34:46Z","date_published":"2022-04-29T21:34:46Z","updated_at":"2026-07-22T22:24:53Z","subjects":["Engineering"],"languages":["en","eng"],"rights":["Copyright 2021 Shiyu Liang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/113893","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Srikant, Rayadurgam","Viswanath, Pramod","Raginsky, Maxim","Sun, Ruoyu","Lee, Jason D."]},{"key":"dc:creator","label":"Author","values":["Liang, Shiyu"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-04-29T21:34:46Z","2021-12","2021-12-01"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Engineering"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2021 Shiyu Liang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/113893"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-04-06 without embargo terms","The student, Shiyu Liang, accepted the attached license on 2021-12-01 at 11:06.","The student, Shiyu Liang, submitted this Dissertation for approval on 2021-12-01 at 11:06.","This Dissertation was approved for publication on 2021-12-01 at 16:30.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17333 on 2022-04-06 at 17:10:34","Made available in DSpace on 2022-04-29T21:34:46Z (GMT). No. of bitstreams: 3 LIANG-DISSERTATION-2021.pdf: 658537 bytes, checksum: e0c47e791cae3180b0683ed2aee50f29 (MD5) LICENSE.txt: 4208 bytes, checksum: 7a8a3f092c6bdf10766efc8ce2f5049b (MD5) PROQUEST_LICENSE.txt: 4554 bytes, checksum: 020958fc887f59bb41f74a25430b239e (MD5) Previous issue date: 2021-12-01","Recent theoretical works on over-parameterized neural nets have focused on two aspects: optimization and generalization. Many existing works that study optimization and generalization together are based on the neural tangent kernel and require a very large width. In this dissertation, we are interested in the following two questions: for a binary classification problem with two-layer mildly over-parameterized ReLU network, (1) does every local minimum memorize and generalize well? and (2) can we find a set of parameters that result in small test error in polynomial time? We first show that the landscape of loss functions with explicit regularization has the following property: all local minima, and certain other points which are only stationary in certain directions, achieve small test error. We then prove that, for convolutional neural nets, there is an algorithm which finds one of these points in polynomial time (in the input dimension and the number of data points). In addition, we prove that for a fully connected neural net, with an additional assumption on the data distribution, there is a polynomial-time algorithm to find one of these points."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["The role of explicit regularization in overparameterized neural networks"]}]}],"canonical_facts":{"dc:contributor":["Srikant, Rayadurgam","Viswanath, Pramod","Raginsky, Maxim","Sun, Ruoyu","Lee, Jason D."],"dc:creator":["Liang, Shiyu"],"dc:date":["2022-04-29T21:34:46Z","2021-12","2021-12-01"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-04-06 without embargo terms","The student, Shiyu Liang, accepted the attached license on 2021-12-01 at 11:06.","The student, Shiyu Liang, submitted this Dissertation for approval on 2021-12-01 at 11:06.","This Dissertation was approved for publication on 2021-12-01 at 16:30.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17333 on 2022-04-06 at 17:10:34","Made available in DSpace on 2022-04-29T21:34:46Z (GMT). No. of bitstreams: 3 LIANG-DISSERTATION-2021.pdf: 658537 bytes, checksum: e0c47e791cae3180b0683ed2aee50f29 (MD5) LICENSE.txt: 4208 bytes, checksum: 7a8a3f092c6bdf10766efc8ce2f5049b (MD5) PROQUEST_LICENSE.txt: 4554 bytes, checksum: 020958fc887f59bb41f74a25430b239e (MD5) Previous issue date: 2021-12-01","Recent theoretical works on over-parameterized neural nets have focused on two aspects: optimization and generalization. Many existing works that study optimization and generalization together are based on the neural tangent kernel and require a very large width. In this dissertation, we are interested in the following two questions: for a binary classification problem with two-layer mildly over-parameterized ReLU network, (1) does every local minimum memorize and generalize well? and (2) can we find a set of parameters that result in small test error in polynomial time? We first show that the landscape of loss functions with explicit regularization has the following property: all local minima, and certain other points which are only stationary in certain directions, achieve small test error. We then prove that, for convolutional neural nets, there is an algorithm which finds one of these points in polynomial time (in the input dimension and the number of data points). In addition, we prove that for a fully connected neural net, with an additional assumption on the data distribution, there is a polynomial-time algorithm to find one of these points."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/113893"],"dc:language":["en","eng"],"dc:rights":["Copyright 2021 Shiyu Liang"],"dc:subject":["Engineering"],"dc:title":["The role of explicit regularization in overparameterized neural networks"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:53Z"}