{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/95379"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/95379","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Similarity modeling for machine learning","abstract":"Similarity is the extent to which two objects resemble each other. Modeling similarity is an important topic for both machine learning and computer vision. In this dissertation, we first propose a discriminative similarity learning method, then introduce two novel sparse similarity modeling methods for high dimensional data from the perspective of manifold learning and subspace learning. Our sparse similarity modeling methods learn sparse similarity and consequently generate a sparse graph over the data. The generated sparse graph leads to superior performance in clustering and semi-supervised learning, compared to existing sparse graph based methods such as $\\ell^{1}$-graph and Sparse Subspace Clustering (SSC). More concretely, our discriminative similarity learning method adopts a novel pairwise clustering framework by bridging the gap between clustering and multi-class classification. This pairwise clustering framework learns an unsupervised nonparametric classifier from each data partition, and searches for the optimal partition of the data by minimizing the generalization error of the learned classifiers associated with the data partitions. Regarding to our sparse similarity modeling methods, we propose a novel $\\ell^{0}$ regularized $\\ell^{1}$-graph ($\\ell^{0}$-$\\ell^{1}$-graph) to improve $\\ell^{1}$-graph from the perspective of manifold learning. Our $\\ell^{0}$-$\\ell^{1}$-graph generates a sparse graph that is aligned to the manifold structure of the data for better clustering performance. From the perspective of learning the subspace structures of the high dimensional data, we propose $\\ell^{0}$-graph that generates a subspace-consistent sparse graph for clustering and semi-supervised learning. Subspace-consistent sparse graph is a sparse graph where a data point is only connected to other data that lie in the same subspace, and the representative method Sparse Subspace Clustering (SSC) proves to generate subspace-consistent sparse graph under certain assumptions on the subspaces and the data, e.g. independent/disjoint subspaces and subspace incoherence/affinity. In contrast, our $\\ell^{0}$-graph can generate subspace-consistent sparse graph for arbitrary distinct underlying subspaces under far less restrictive assumptions, i.e. only i.i.d. random data generation according to arbitrary continuous distribution. Extensive experimental results on various data sets demonstrate the superiority of $\\ell^{0}$-graph compared to other methods including SSC for both clustering and semi-supervised learning. The proposed sparse similarity modeling methods require sparse coding using the entire data as the dictionary, which can be inefficient especially in case of large-scale data. In order to overcome this challenge, we propose Support Regularized Sparse Coding (SRSC) where a compact dictionary is learned. The data similarity induced by the support regularized sparse codes leads to compelling clustering performance. Moreover, a feed-forward neural network, termed Deep-SRSC, is designed as a fast encoder to approximate the codes generated by SRSC, further improving the efficiency of SRSC.","abstract_html":"Similarity is the extent to which two objects resemble each other. Modeling similarity is an important topic for both machine learning and computer vision. In this dissertation, we first propose a discriminative similarity learning method, then introduce two novel sparse similarity modeling methods for high dimensional data from the perspective of manifold learning and subspace learning. Our sparse similarity modeling methods learn sparse similarity and consequently generate a sparse graph over the data. The generated sparse graph leads to superior performance in clustering and semi-supervised learning, compared to existing sparse graph based methods such as <span class=\"etd-inline-math\">\\ell<sup>1</sup></span>-graph and Sparse Subspace Clustering (SSC). More concretely, our discriminative similarity learning method adopts a novel pairwise clustering framework by bridging the gap between clustering and multi-class classification. This pairwise clustering framework learns an unsupervised nonparametric classifier from each data partition, and searches for the optimal partition of the data by minimizing the generalization error of the learned classifiers associated with the data partitions. Regarding to our sparse similarity modeling methods, we propose a novel <span class=\"etd-inline-math\">\\ell<sup>0</sup></span> regularized <span class=\"etd-inline-math\">\\ell<sup>1</sup></span>-graph (<span class=\"etd-inline-math\">\\ell<sup>0</sup></span>-<span class=\"etd-inline-math\">\\ell<sup>1</sup></span>-graph) to improve <span class=\"etd-inline-math\">\\ell<sup>1</sup></span>-graph from the perspective of manifold learning. Our <span class=\"etd-inline-math\">\\ell<sup>0</sup></span>-<span class=\"etd-inline-math\">\\ell<sup>1</sup></span>-graph generates a sparse graph that is aligned to the manifold structure of the data for better clustering performance. From the perspective of learning the subspace structures of the high dimensional data, we propose <span class=\"etd-inline-math\">\\ell<sup>0</sup></span>-graph that generates a subspace-consistent sparse graph for clustering and semi-supervised learning. Subspace-consistent sparse graph is a sparse graph where a data point is only connected to other data that lie in the same subspace, and the representative method Sparse Subspace Clustering (SSC) proves to generate subspace-consistent sparse graph under certain assumptions on the subspaces and the data, e.g. independent/disjoint subspaces and subspace incoherence/affinity. In contrast, our <span class=\"etd-inline-math\">\\ell<sup>0</sup></span>-graph can generate subspace-consistent sparse graph for arbitrary distinct underlying subspaces under far less restrictive assumptions, i.e. only i.i.d. random data generation according to arbitrary continuous distribution. Extensive experimental results on various data sets demonstrate the superiority of <span class=\"etd-inline-math\">\\ell<sup>0</sup></span>-graph compared to other methods including SSC for both clustering and semi-supervised learning. The proposed sparse similarity modeling methods require sparse coding using the entire data as the dictionary, which can be inefficient especially in case of large-scale data. In order to overcome this challenge, we propose Support Regularized Sparse Coding (SRSC) where a compact dictionary is learned. The data similarity induced by the support regularized sparse codes leads to compelling clustering performance. Moreover, a feed-forward neural network, termed Deep-SRSC, is designed as a fast encoder to approximate the codes generated by SRSC, further improving the efficiency of SRSC.","abstract_has_math":true,"creators":["Yang, Yingzhen"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Huang, Thomas S.","Hasegawa-Johnson, Mark","Liang, Zhi-Pei","Yang, Jianchao"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2017,"date_issued":"2017-03-01T15:49:22Z","date_published":"2017-03-01T15:49:22Z","updated_at":"2026-07-22T22:26:37Z","subjects":["Similarity Modeling","Machine Learning"],"languages":["en"],"rights":["Copyright 2016 Yingzhen Yang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/95379","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Huang, Thomas S.","Hasegawa-Johnson, Mark","Liang, Zhi-Pei","Yang, Jianchao"]},{"key":"dc:creator","label":"Author","values":["Yang, Yingzhen"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2017-03-01T15:49:22Z","2016-12-02","2016-12"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Similarity Modeling","Machine Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2016 Yingzhen Yang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/95379"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Similarity is the extent to which two objects resemble each other. Modeling similarity is an important topic for both machine learning and computer vision. In this dissertation, we first propose a discriminative similarity learning method, then introduce two novel sparse similarity modeling methods for high dimensional data from the perspective of manifold learning and subspace learning. Our sparse similarity modeling methods learn sparse similarity and consequently generate a sparse graph over the data. The generated sparse graph leads to superior performance in clustering and semi-supervised learning, compared to existing sparse graph based methods such as $\\ell^{1}$-graph and Sparse Subspace Clustering (SSC). More concretely, our discriminative similarity learning method adopts a novel pairwise clustering framework by bridging the gap between clustering and multi-class classification. This pairwise clustering framework learns an unsupervised nonparametric classifier from each data partition, and searches for the optimal partition of the data by minimizing the generalization error of the learned classifiers associated with the data partitions. Regarding to our sparse similarity modeling methods, we propose a novel $\\ell^{0}$ regularized $\\ell^{1}$-graph ($\\ell^{0}$-$\\ell^{1}$-graph) to improve $\\ell^{1}$-graph from the perspective of manifold learning. Our $\\ell^{0}$-$\\ell^{1}$-graph generates a sparse graph that is aligned to the manifold structure of the data for better clustering performance. From the perspective of learning the subspace structures of the high dimensional data, we propose $\\ell^{0}$-graph that generates a subspace-consistent sparse graph for clustering and semi-supervised learning. Subspace-consistent sparse graph is a sparse graph where a data point is only connected to other data that lie in the same subspace, and the representative method Sparse Subspace Clustering (SSC) proves to generate subspace-consistent sparse graph under certain assumptions on the subspaces and the data, e.g. independent/disjoint subspaces and subspace incoherence/affinity. In contrast, our $\\ell^{0}$-graph can generate subspace-consistent sparse graph for arbitrary distinct underlying subspaces under far less restrictive assumptions, i.e. only i.i.d. random data generation according to arbitrary continuous distribution. Extensive experimental results on various data sets demonstrate the superiority of $\\ell^{0}$-graph compared to other methods including SSC for both clustering and semi-supervised learning. The proposed sparse similarity modeling methods require sparse coding using the entire data as the dictionary, which can be inefficient especially in case of large-scale data. In order to overcome this challenge, we propose Support Regularized Sparse Coding (SRSC) where a compact dictionary is learned. The data similarity induced by the support regularized sparse codes leads to compelling clustering performance. Moreover, a feed-forward neural network, termed Deep-SRSC, is designed as a fast encoder to approximate the codes generated by SRSC, further improving the efficiency of SRSC.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2017-02-28 without embargo terms","The student, Yingzhen Yang, accepted the attached license on 2016-12-01 at 02:53.","The student, Yingzhen Yang, submitted this Dissertation for approval on 2016-12-01 at 03:18.","This Dissertation was approved for publication on 2016-12-02 at 09:20.","DSpace SAF Submission Ingestion Package generated from Vireo submission #10387 on 2017-02-28 at 14:55:02","Made available in DSpace on 2017-03-01T15:49:22Z (GMT). No. of bitstreams: 2 YANG-DISSERTATION-2016.pdf: 1768484 bytes, checksum: f475c90c6650b9c776d7a6d3fe50255b (MD5) LICENSE.txt: 4210 bytes, checksum: 38c62f55e06e9a6f614846f0fabc4d8d (MD5) Previous issue date: 2016-12-02"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Similarity modeling for machine learning"]}]}],"canonical_facts":{"dc:contributor":["Huang, Thomas S.","Hasegawa-Johnson, Mark","Liang, Zhi-Pei","Yang, Jianchao"],"dc:creator":["Yang, Yingzhen"],"dc:date":["2017-03-01T15:49:22Z","2016-12-02","2016-12"],"dc:description":["Similarity is the extent to which two objects resemble each other. Modeling similarity is an important topic for both machine learning and computer vision. In this dissertation, we first propose a discriminative similarity learning method, then introduce two novel sparse similarity modeling methods for high dimensional data from the perspective of manifold learning and subspace learning. Our sparse similarity modeling methods learn sparse similarity and consequently generate a sparse graph over the data. The generated sparse graph leads to superior performance in clustering and semi-supervised learning, compared to existing sparse graph based methods such as $\\ell^{1}$-graph and Sparse Subspace Clustering (SSC). More concretely, our discriminative similarity learning method adopts a novel pairwise clustering framework by bridging the gap between clustering and multi-class classification. This pairwise clustering framework learns an unsupervised nonparametric classifier from each data partition, and searches for the optimal partition of the data by minimizing the generalization error of the learned classifiers associated with the data partitions. Regarding to our sparse similarity modeling methods, we propose a novel $\\ell^{0}$ regularized $\\ell^{1}$-graph ($\\ell^{0}$-$\\ell^{1}$-graph) to improve $\\ell^{1}$-graph from the perspective of manifold learning. Our $\\ell^{0}$-$\\ell^{1}$-graph generates a sparse graph that is aligned to the manifold structure of the data for better clustering performance. From the perspective of learning the subspace structures of the high dimensional data, we propose $\\ell^{0}$-graph that generates a subspace-consistent sparse graph for clustering and semi-supervised learning. Subspace-consistent sparse graph is a sparse graph where a data point is only connected to other data that lie in the same subspace, and the representative method Sparse Subspace Clustering (SSC) proves to generate subspace-consistent sparse graph under certain assumptions on the subspaces and the data, e.g. independent/disjoint subspaces and subspace incoherence/affinity. In contrast, our $\\ell^{0}$-graph can generate subspace-consistent sparse graph for arbitrary distinct underlying subspaces under far less restrictive assumptions, i.e. only i.i.d. random data generation according to arbitrary continuous distribution. Extensive experimental results on various data sets demonstrate the superiority of $\\ell^{0}$-graph compared to other methods including SSC for both clustering and semi-supervised learning. The proposed sparse similarity modeling methods require sparse coding using the entire data as the dictionary, which can be inefficient especially in case of large-scale data. In order to overcome this challenge, we propose Support Regularized Sparse Coding (SRSC) where a compact dictionary is learned. The data similarity induced by the support regularized sparse codes leads to compelling clustering performance. Moreover, a feed-forward neural network, termed Deep-SRSC, is designed as a fast encoder to approximate the codes generated by SRSC, further improving the efficiency of SRSC.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2017-02-28 without embargo terms","The student, Yingzhen Yang, accepted the attached license on 2016-12-01 at 02:53.","The student, Yingzhen Yang, submitted this Dissertation for approval on 2016-12-01 at 03:18.","This Dissertation was approved for publication on 2016-12-02 at 09:20.","DSpace SAF Submission Ingestion Package generated from Vireo submission #10387 on 2017-02-28 at 14:55:02","Made available in DSpace on 2017-03-01T15:49:22Z (GMT). No. of bitstreams: 2 YANG-DISSERTATION-2016.pdf: 1768484 bytes, checksum: f475c90c6650b9c776d7a6d3fe50255b (MD5) LICENSE.txt: 4210 bytes, checksum: 38c62f55e06e9a6f614846f0fabc4d8d (MD5) Previous issue date: 2016-12-02"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/95379"],"dc:language":["en"],"dc:rights":["Copyright 2016 Yingzhen Yang"],"dc:subject":["Similarity Modeling","Machine Learning"],"dc:title":["Similarity modeling for machine learning"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:26:37Z"}