{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/95560"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/95560","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Task-specific and interpretable feature learning","abstract":"Deep learning models have had tremendous impacts in recent years, while a question has been raised by many: Is deep learning just a triumph of empiricism? There has been emerging interest in reducing the gap between the theoretical soundness and interpretability, and the empirical success of deep models. This dissertation provides a comprehensive discussion on bridging traditional model-based learning approaches that emphasize problem-specific reasoning, and deep models that allow for larger learning capacity. The overall goal is to devise the next-generation feature learning architectures that are: 1) task-specific, namely, optimizing the entire pipeline from end to end while taking advantage of available prior knowledge and domain expertise; and 2) interpretable, namely, being able to learn a representation consisting of semantically sensible variables, and to display predictable behaviors. This dissertation starts by showing how the classical sparse coding models could be improved in a task-specific way, by formulating the entire pipeline as bi-level optimization. Then, it mainly illustrates how to incorporate the structure of classical learning models, e.g., sparse coding, into the design of deep architectures. A few concrete model examples are presented, ranging from the $\\ell_0$ and $\\ell_1$ sparse approximation models, to the $\\ell_\\infty$ constrained model and the dual-sparsity model. The analytic tools in the optimization problems can be translated to guide the architecture design and performance analysis of deep models. As a result, those customized deep models demonstrate improved performance, intuitive interpretation, and efficient parameter initialization. On the other hand, deep networks are shown to be analogous to brain mechanisms. They exhibit the ability to describe semantic content from the primitive level to the abstract level. This dissertation thus also presents a preliminary investigation of the synergy between feature learning with cognitive science and neuroscience. Two novel application domains, image aesthetics assessment and brain encoding, are explored, with promising preliminary results achieved.","abstract_html":"Deep learning models have had tremendous impacts in recent years, while a question has been raised by many: Is deep learning just a triumph of empiricism? There has been emerging interest in reducing the gap between the theoretical soundness and interpretability, and the empirical success of deep models. This dissertation provides a comprehensive discussion on bridging traditional model-based learning approaches that emphasize problem-specific reasoning, and deep models that allow for larger learning capacity. The overall goal is to devise the next-generation feature learning architectures that are: 1) task-specific, namely, optimizing the entire pipeline from end to end while taking advantage of available prior knowledge and domain expertise; and 2) interpretable, namely, being able to learn a representation consisting of semantically sensible variables, and to display predictable behaviors. This dissertation starts by showing how the classical sparse coding models could be improved in a task-specific way, by formulating the entire pipeline as bi-level optimization. Then, it mainly illustrates how to incorporate the structure of classical learning models, e.g., sparse coding, into the design of deep architectures. A few concrete model examples are presented, ranging from the <span class=\"etd-inline-math\">\\ell<sub>0</sub></span> and <span class=\"etd-inline-math\">\\ell<sub>1</sub></span> sparse approximation models, to the <span class=\"etd-inline-math\">\\ell<sub>\\</sub>infty</span> constrained model and the dual-sparsity model. The analytic tools in the optimization problems can be translated to guide the architecture design and performance analysis of deep models. As a result, those customized deep models demonstrate improved performance, intuitive interpretation, and efficient parameter initialization. On the other hand, deep networks are shown to be analogous to brain mechanisms. They exhibit the ability to describe semantic content from the primitive level to the abstract level. This dissertation thus also presents a preliminary investigation of the synergy between feature learning with cognitive science and neuroscience. Two novel application domains, image aesthetics assessment and brain encoding, are explored, with promising preliminary results achieved.","abstract_has_math":true,"creators":["Wang, Zhangyang"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Huang, Thomas","Hasegawa-Johnson, Mark","Liang, Zhi-Pei","Dolcos, Florin","Yang, Jianchao"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2017,"date_issued":"2017-03-01T17:01:07Z","date_published":"2017-03-01T17:01:07Z","updated_at":"2026-07-22T22:26:37Z","subjects":["deep learning","sparse representation"],"languages":["en"],"rights":["Copyright 2016 Zhangyang Wang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/95560","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Huang, Thomas","Hasegawa-Johnson, Mark","Liang, Zhi-Pei","Dolcos, Florin","Yang, Jianchao"]},{"key":"dc:creator","label":"Author","values":["Wang, Zhangyang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2017-03-01T17:01:07Z","2019-03-02T10:15:07Z","2016-10-28","2016-12"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["deep learning","sparse representation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2016 Zhangyang Wang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/95560"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Deep learning models have had tremendous impacts in recent years, while a question has been raised by many: Is deep learning just a triumph of empiricism? There has been emerging interest in reducing the gap between the theoretical soundness and interpretability, and the empirical success of deep models. This dissertation provides a comprehensive discussion on bridging traditional model-based learning approaches that emphasize problem-specific reasoning, and deep models that allow for larger learning capacity. The overall goal is to devise the next-generation feature learning architectures that are: 1) task-specific, namely, optimizing the entire pipeline from end to end while taking advantage of available prior knowledge and domain expertise; and 2) interpretable, namely, being able to learn a representation consisting of semantically sensible variables, and to display predictable behaviors. This dissertation starts by showing how the classical sparse coding models could be improved in a task-specific way, by formulating the entire pipeline as bi-level optimization. Then, it mainly illustrates how to incorporate the structure of classical learning models, e.g., sparse coding, into the design of deep architectures. A few concrete model examples are presented, ranging from the $\\ell_0$ and $\\ell_1$ sparse approximation models, to the $\\ell_\\infty$ constrained model and the dual-sparsity model. The analytic tools in the optimization problems can be translated to guide the architecture design and performance analysis of deep models. As a result, those customized deep models demonstrate improved performance, intuitive interpretation, and efficient parameter initialization. On the other hand, deep networks are shown to be analogous to brain mechanisms. They exhibit the ability to describe semantic content from the primitive level to the abstract level. This dissertation thus also presents a preliminary investigation of the synergy between feature learning with cognitive science and neuroscience. Two novel application domains, image aesthetics assessment and brain encoding, are explored, with promising preliminary results achieved.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2018-12-01","The student, Zhangyang Wang, accepted the attached license on 2016-10-28 at 09:36.","The student, Zhangyang Wang, submitted this Dissertation for approval on 2016-10-28 at 09:44.","This Dissertation was approved for publication on 2016-10-28 at 13:39.","DSpace SAF Submission Ingestion Package generated from Vireo submission #10206 on 2017-02-28 at 14:41:16","Made available in DSpace on 2017-03-01T17:01:07Z (GMT). No. of bitstreams: 2 WANG-DISSERTATION-2016.pdf: 6234466 bytes, checksum: 718edda7179a23890ccf902cf3832f76 (MD5) LICENSE.txt: 4211 bytes, checksum: b33aae51cb4fe6f7425f8c4a19912ec7 (MD5) Previous issue date: 2016-10-28","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:02:22Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:03:32Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:05:02Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:06:55Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited Restriction Lifted for Item 98676 on 2019-03-02T10:15:07Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Task-specific and interpretable feature learning"]}]}],"canonical_facts":{"dc:contributor":["Huang, Thomas","Hasegawa-Johnson, Mark","Liang, Zhi-Pei","Dolcos, Florin","Yang, Jianchao"],"dc:creator":["Wang, Zhangyang"],"dc:date":["2017-03-01T17:01:07Z","2019-03-02T10:15:07Z","2016-10-28","2016-12"],"dc:description":["Deep learning models have had tremendous impacts in recent years, while a question has been raised by many: Is deep learning just a triumph of empiricism? There has been emerging interest in reducing the gap between the theoretical soundness and interpretability, and the empirical success of deep models. This dissertation provides a comprehensive discussion on bridging traditional model-based learning approaches that emphasize problem-specific reasoning, and deep models that allow for larger learning capacity. The overall goal is to devise the next-generation feature learning architectures that are: 1) task-specific, namely, optimizing the entire pipeline from end to end while taking advantage of available prior knowledge and domain expertise; and 2) interpretable, namely, being able to learn a representation consisting of semantically sensible variables, and to display predictable behaviors. This dissertation starts by showing how the classical sparse coding models could be improved in a task-specific way, by formulating the entire pipeline as bi-level optimization. Then, it mainly illustrates how to incorporate the structure of classical learning models, e.g., sparse coding, into the design of deep architectures. A few concrete model examples are presented, ranging from the $\\ell_0$ and $\\ell_1$ sparse approximation models, to the $\\ell_\\infty$ constrained model and the dual-sparsity model. The analytic tools in the optimization problems can be translated to guide the architecture design and performance analysis of deep models. As a result, those customized deep models demonstrate improved performance, intuitive interpretation, and efficient parameter initialization. On the other hand, deep networks are shown to be analogous to brain mechanisms. They exhibit the ability to describe semantic content from the primitive level to the abstract level. This dissertation thus also presents a preliminary investigation of the synergy between feature learning with cognitive science and neuroscience. Two novel application domains, image aesthetics assessment and brain encoding, are explored, with promising preliminary results achieved.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2018-12-01","The student, Zhangyang Wang, accepted the attached license on 2016-10-28 at 09:36.","The student, Zhangyang Wang, submitted this Dissertation for approval on 2016-10-28 at 09:44.","This Dissertation was approved for publication on 2016-10-28 at 13:39.","DSpace SAF Submission Ingestion Package generated from Vireo submission #10206 on 2017-02-28 at 14:41:16","Made available in DSpace on 2017-03-01T17:01:07Z (GMT). No. of bitstreams: 2 WANG-DISSERTATION-2016.pdf: 6234466 bytes, checksum: 718edda7179a23890ccf902cf3832f76 (MD5) LICENSE.txt: 4211 bytes, checksum: b33aae51cb4fe6f7425f8c4a19912ec7 (MD5) Previous issue date: 2016-10-28","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:02:22Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:03:32Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:05:02Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 98676 Lift date: 2019-03-01T17:06:55Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited Restriction Lifted for Item 98676 on 2019-03-02T10:15:07Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/95560"],"dc:language":["en"],"dc:rights":["Copyright 2016 Zhangyang Wang"],"dc:subject":["deep learning","sparse representation"],"dc:title":["Task-specific and interpretable feature learning"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:26:37Z"}