{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/107979"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/107979","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Hyperparameter tuning and its effects on deep learning performance and generalization","abstract":"Hyperparameter tuning is an integral part of deep learning research. Finding hyperparameter values that effectively leverage the strengths of network architectures and training procedures is crucial to maximizing performance. However, extensive hyperparameter searches raise concerns about overfitting to re-used evaluation datasets. In this thesis, we perform a case study of hyperparameter search methods on SqueezeNet v1.0 with refinements added to the training procedure. We show that random search allows for improvement over baseline performance in few trials, achieve around a 2% increase in SqueezeNet accuracy on ImageNet, and provide evidence that contrary to the common notion of adaptive overfitting, accuracy gains achieved on a validation set through hyperparameter tuning result in larger gains on a held-out test set.","abstract_html":"Hyperparameter tuning is an integral part of deep learning research. Finding hyperparameter values that effectively leverage the strengths of network architectures and training procedures is crucial to maximizing performance. However, extensive hyperparameter searches raise concerns about overfitting to re-used evaluation datasets. In this thesis, we perform a case study of hyperparameter search methods on SqueezeNet v1.0 with refinements added to the training procedure. We show that random search allows for improvement over baseline performance in few trials, achieve around a 2% increase in SqueezeNet accuracy on ImageNet, and provide evidence that contrary to the common notion of adaptive overfitting, accuracy gains achieved on a validation set through hyperparameter tuning result in larger gains on a held-out test set.","abstract_has_math":false,"creators":["Rabe, Benjamin"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Kindratenko, Volodymyr"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-08-26T21:54:50Z","date_published":"2020-08-26T21:54:50Z","updated_at":"2026-07-22T22:24:47Z","subjects":["machine learning","hyperparameter search"],"languages":["en"],"rights":["Copyright 2020 Benjamin Rabe"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/107979","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Kindratenko, Volodymyr"]},{"key":"dc:creator","label":"Author","values":["Rabe, Benjamin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-08-26T21:54:50Z","2020-05-06","2020-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["machine learning","hyperparameter search"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Benjamin Rabe"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/107979"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Hyperparameter tuning is an integral part of deep learning research. Finding hyperparameter values that effectively leverage the strengths of network architectures and training procedures is crucial to maximizing performance. However, extensive hyperparameter searches raise concerns about overfitting to re-used evaluation datasets. In this thesis, we perform a case study of hyperparameter search methods on SqueezeNet v1.0 with refinements added to the training procedure. We show that random search allows for improvement over baseline performance in few trials, achieve around a 2% increase in SqueezeNet accuracy on ImageNet, and provide evidence that contrary to the common notion of adaptive overfitting, accuracy gains achieved on a validation set through hyperparameter tuning result in larger gains on a held-out test set.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-08-25 without embargo terms","The student, Benjamin Rabe, accepted the attached license on 2020-05-04 at 10:47.","The student, Benjamin Rabe, submitted this Thesis for approval on 2020-05-04 at 10:57.","This Thesis was approved for publication on 2020-05-06 at 17:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15187 on 2020-08-25 at 17:11:49","Made available in DSpace on 2020-08-26T21:54:50Z (GMT). No. of bitstreams: 4 RABE-THESIS-2020.pdf: 1789624 bytes, checksum: 9614222d6ba05bbec09aac4e18a95376 (MD5) LICENSE.txt: 4210 bytes, checksum: d3bee1cb84f39a60d4583d1eb8f59fd3 (MD5) bengio_permission.pdf: 82275 bytes, checksum: f4201627cbb32f8c0a7b7de0bcc6a9ae (MD5) bergstra_permission.pdf: 49224 bytes, checksum: 33a755afedb49b7ec5d25627eeec7afd (MD5) Previous issue date: 2020-05-06"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Hyperparameter tuning and its effects on deep learning performance and generalization"]}]}],"canonical_facts":{"dc:contributor":["Kindratenko, Volodymyr"],"dc:creator":["Rabe, Benjamin"],"dc:date":["2020-08-26T21:54:50Z","2020-05-06","2020-05"],"dc:description":["Hyperparameter tuning is an integral part of deep learning research. Finding hyperparameter values that effectively leverage the strengths of network architectures and training procedures is crucial to maximizing performance. However, extensive hyperparameter searches raise concerns about overfitting to re-used evaluation datasets. In this thesis, we perform a case study of hyperparameter search methods on SqueezeNet v1.0 with refinements added to the training procedure. We show that random search allows for improvement over baseline performance in few trials, achieve around a 2% increase in SqueezeNet accuracy on ImageNet, and provide evidence that contrary to the common notion of adaptive overfitting, accuracy gains achieved on a validation set through hyperparameter tuning result in larger gains on a held-out test set.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-08-25 without embargo terms","The student, Benjamin Rabe, accepted the attached license on 2020-05-04 at 10:47.","The student, Benjamin Rabe, submitted this Thesis for approval on 2020-05-04 at 10:57.","This Thesis was approved for publication on 2020-05-06 at 17:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15187 on 2020-08-25 at 17:11:49","Made available in DSpace on 2020-08-26T21:54:50Z (GMT). No. of bitstreams: 4 RABE-THESIS-2020.pdf: 1789624 bytes, checksum: 9614222d6ba05bbec09aac4e18a95376 (MD5) LICENSE.txt: 4210 bytes, checksum: d3bee1cb84f39a60d4583d1eb8f59fd3 (MD5) bengio_permission.pdf: 82275 bytes, checksum: f4201627cbb32f8c0a7b7de0bcc6a9ae (MD5) bergstra_permission.pdf: 49224 bytes, checksum: 33a755afedb49b7ec5d25627eeec7afd (MD5) Previous issue date: 2020-05-06"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/107979"],"dc:language":["en"],"dc:rights":["Copyright 2020 Benjamin Rabe"],"dc:subject":["machine learning","hyperparameter search"],"dc:title":["Hyperparameter tuning and its effects on deep learning performance and generalization"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:47Z"}