{"id":{"repo_id":"siu-theses","oai_identifier":"oai:opensiuc.lib.siu.edu:dissertations-2418"},"canonical_url":"https://search.dev.ndltd.org/etd/siu-theses/oai:opensiuc.lib.siu.edu:dissertations-2418","repository":{"repo_id":"siu-theses","name":"Southern Illinois University","base_url":"https://opensiuc.lib.siu.edu/do/oai/"},"display":{"title":"Conditional Neural Networks for Speech and Language Processing","abstract":"Neural networks based deep learning methods have gained significant success in several real world tasks: from machine translation to web recommendation, and it is also greatly improving the computer vision and the natural language processing. Compared with conventional machine learning techniques, neural network based deep learning do not require careful engineering and consideration domain expertise to design a feature extractor that transformed the raw data to a suitable internal representation. Its extreme efficacy on multiple levels of representation and feature learning ensures this type of approaches can process high dimensional data. It integrates the feature representation, learning and recognition into a systematical framework, which allows the learning starts at one level (i.e., being with raw input) and end at a higher slightly more abstract level. By simply stacking enough such transformations, very complex functions can be obtained. In general, high level feature representation facilitate the discrimination of patterns, and additionally can reduce the impact of irrelevant variations. However, previous studies indicate that deep composition of the networks make the training errors become vanished. To overcome this weakness, several techniques have been developed, for instance, dropout, stochastic gradient decent and residual network structures. In this study, we incorporates latent information into different network structures (e.g., restricted Boltzmann machine, recursive neural networks, and long short term memory). The conditional latent information reflects the high dimensional correlation existed in the data structure, and the typical network structure may not learn this kind of features due to limitation of the initial design (i.e., the network size the parameters). Similarly to residual nets, the conditional neural networks jointly learns the global features and local features, and the specifically designed network structure helps to incorporate the modulation derived from the probability distribution. The proposed models have been widely tested in different datasets, for instance, the conditional RBM has been applied to detect the speech components, and a language model based gated RBM has been used to recognize speech related EEG patterns. The conditional RNN has been tested in both general natural language modeling and medical notes prediction tasks. The results indicate that by introducing conditional branches in the conventional network structures, the latent features can be globally and locally learned.","abstract_html":"Neural networks based deep learning methods have gained significant success in several real world tasks: from machine translation to web recommendation, and it is also greatly improving the computer vision and the natural language processing. Compared with conventional machine learning techniques, neural network based deep learning do not require careful engineering and consideration domain expertise to design a feature extractor that transformed the raw data to a suitable internal representation. Its extreme efficacy on multiple levels of representation and feature learning ensures this type of approaches can process high dimensional data. It integrates the feature representation, learning and recognition into a systematical framework, which allows the learning starts at one level (i.e., being with raw input) and end at a higher slightly more abstract level. By simply stacking enough such transformations, very complex functions can be obtained. In general, high level feature representation facilitate the discrimination of patterns, and additionally can reduce the impact of irrelevant variations. However, previous studies indicate that deep composition of the networks make the training errors become vanished. To overcome this weakness, several techniques have been developed, for instance, dropout, stochastic gradient decent and residual network structures. In this study, we incorporates latent information into different network structures (e.g., restricted Boltzmann machine, recursive neural networks, and long short term memory). The conditional latent information reflects the high dimensional correlation existed in the data structure, and the typical network structure may not learn this kind of features due to limitation of the initial design (i.e., the network size the parameters). Similarly to residual nets, the conditional neural networks jointly learns the global features and local features, and the specifically designed network structure helps to incorporate the modulation derived from the probability distribution. The proposed models have been widely tested in different datasets, for instance, the conditional RBM has been applied to detect the speech components, and a language model based gated RBM has been used to recognize speech related EEG patterns. The conditional RNN has been tested in both general natural language modeling and medical notes prediction tasks. The results indicate that by introducing conditional branches in the conventional network structures, the latent features can be globally and locally learned.","abstract_has_math":false,"creators":["Sun, Pengfei"],"institution":null,"degree_name":"Doctor of Philosophy","degree_level":"Campus Only Dissertation","degree_discipline":"Electrical and Computer Engineering","degree_department":null,"school":null,"contributors":["Qin, Jun"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2017,"date_issued":"2017-08-01T07:00:00Z","date_published":"2017-08-01T07:00:00Z","updated_at":"2026-07-24T04:35:23Z","subjects":["Conditional Neural network","Deep learning","feature representation","Natural language processing","Neural networks","Speech recognition"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://opensiuc.lib.siu.edu/dissertations/1414","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Qin, Jun"]},{"key":"dc:creator","label":"Author","values":["Sun, Pengfei"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2018-10-25T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical and Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Campus Only Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Conditional Neural network","Deep learning","feature representation","Natural language processing","Neural networks","Speech recognition"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://opensiuc.lib.siu.edu/dissertations/1414"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Neural networks based deep learning methods have gained significant success in several real world tasks: from machine translation to web recommendation, and it is also greatly improving the computer vision and the natural language processing. Compared with conventional machine learning techniques, neural network based deep learning do not require careful engineering and consideration domain expertise to design a feature extractor that transformed the raw data to a suitable internal representation. Its extreme efficacy on multiple levels of representation and feature learning ensures this type of approaches can process high dimensional data. It integrates the feature representation, learning and recognition into a systematical framework, which allows the learning starts at one level (i.e., being with raw input) and end at a higher slightly more abstract level. By simply stacking enough such transformations, very complex functions can be obtained. In general, high level feature representation facilitate the discrimination of patterns, and additionally can reduce the impact of irrelevant variations. However, previous studies indicate that deep composition of the networks make the training errors become vanished. To overcome this weakness, several techniques have been developed, for instance, dropout, stochastic gradient decent and residual network structures. In this study, we incorporates latent information into different network structures (e.g., restricted Boltzmann machine, recursive neural networks, and long short term memory). The conditional latent information reflects the high dimensional correlation existed in the data structure, and the typical network structure may not learn this kind of features due to limitation of the initial design (i.e., the network size the parameters). Similarly to residual nets, the conditional neural networks jointly learns the global features and local features, and the specifically designed network structure helps to incorporate the modulation derived from the probability distribution. The proposed models have been widely tested in different datasets, for instance, the conditional RBM has been applied to detect the speech components, and a language model based gated RBM has been used to recognize speech related EEG patterns. The conditional RNN has been tested in both general natural language modeling and medical notes prediction tasks. The results indicate that by introducing conditional branches in the conventional network structures, the latent features can be globally and locally learned."]},{"key":"dc:title","label":"Title","values":["Conditional Neural Networks for Speech and Language Processing"]}]}],"canonical_facts":{"dc:contributor":["Qin, Jun"],"dc:creator":["Sun, Pengfei"],"dc:date.available":["2018-10-25T07:00:00Z"],"dc:description.abstract":["Neural networks based deep learning methods have gained significant success in several real world tasks: from machine translation to web recommendation, and it is also greatly improving the computer vision and the natural language processing. Compared with conventional machine learning techniques, neural network based deep learning do not require careful engineering and consideration domain expertise to design a feature extractor that transformed the raw data to a suitable internal representation. Its extreme efficacy on multiple levels of representation and feature learning ensures this type of approaches can process high dimensional data. It integrates the feature representation, learning and recognition into a systematical framework, which allows the learning starts at one level (i.e., being with raw input) and end at a higher slightly more abstract level. By simply stacking enough such transformations, very complex functions can be obtained. In general, high level feature representation facilitate the discrimination of patterns, and additionally can reduce the impact of irrelevant variations. However, previous studies indicate that deep composition of the networks make the training errors become vanished. To overcome this weakness, several techniques have been developed, for instance, dropout, stochastic gradient decent and residual network structures. In this study, we incorporates latent information into different network structures (e.g., restricted Boltzmann machine, recursive neural networks, and long short term memory). The conditional latent information reflects the high dimensional correlation existed in the data structure, and the typical network structure may not learn this kind of features due to limitation of the initial design (i.e., the network size the parameters). Similarly to residual nets, the conditional neural networks jointly learns the global features and local features, and the specifically designed network structure helps to incorporate the modulation derived from the probability distribution. The proposed models have been widely tested in different datasets, for instance, the conditional RBM has been applied to detect the speech components, and a language model based gated RBM has been used to recognize speech related EEG patterns. The conditional RNN has been tested in both general natural language modeling and medical notes prediction tasks. The results indicate that by introducing conditional branches in the conventional network structures, the latent features can be globally and locally learned."],"dc:identifier":["https://opensiuc.lib.siu.edu/dissertations/1414"],"dc:subject":["Conditional Neural network","Deep learning","feature representation","Natural language processing","Neural networks","Speech recognition"],"dc:title":["Conditional Neural Networks for Speech and Language Processing"],"thesis:degree_discipline":["Electrical and Computer Engineering"],"thesis:degree_level":["Campus Only Dissertation"],"thesis:degree_name":["Doctor of Philosophy"]},"updated_at":"2026-07-24T04:35:23Z"}