{"id":{"repo_id":"auckland-ms","oai_identifier":"oai:researchspace.auckland.ac.nz:2292/61963"},"canonical_url":"https://search.dev.ndltd.org/etd/auckland-ms/oai:researchspace.auckland.ac.nz:2292/61963","repository":{"repo_id":"auckland-ms","name":"University of Auckland","base_url":"https://researchspace.auckland.ac.nz/server/oai/request"},"display":{"title":"Fuse and Adapt: Investigating the Use of Pre-Trained Self-Supervising Learning Models in Limited Data NLU problems","abstract":"Deep Learning (DL) has become a key element in the field of Artificial Intelligence (AI) over the last decade. DL has improved many applications related to different fields of Natural Language Processing (NLP), Computer Vision (CV), Speech Recognition (SR), and Reinforcement Learning (RL). A significant challenge in applying DL in most domains is the scarcity of labelled data. Usually, DL models need large amounts of annotated data to train models. The novel paradigm of Self Supervised Learning (SSL) has become a gamechanger in the field of Deep Learning due to its ability to answer the problem of scarcity of labeled data. SSL can utilize commonly available unlabelled data to large DL architectures. SSL usually consists of two stages, the pre-training phase, and the downstream phase. The pre-training phase usually needs a large amount of unlabelled data, and it is a computationally expensive process that could cost hundreds of thousands of dollars to millions. But it has become a common practice to open-source the model checkpoints of pre-trained SSL models that can represent different modalities of data such as text, vision, and speech. Usually, expensive and valuable pre-trained SSL models get open-sourced from tech giants like Google, Meta, Amazon, Nvidia, and Microsoft. These Pre-trained models have become a vital part of research communities and the industry due to their effectiveness in solving many downstream tasks. In this thesis, my focus is on exploring the utilization of pre-trained SSL models in the field of Natural Language Understanding (NLU). NLU is the ability of machines to understand human language. It has enabled many practical applications such as Emotion Recognition, Sentiment Analysis, Summarization, and Question Answering. This thesis mainly explores the utilization of pre-trained SSL models in the two main areas of multimodal fusion and domain adaptation. Under these two main topics, I explored four research question that introduces novel fusion and adaptation techniques.","abstract_html":"Deep Learning (DL) has become a key element in the field of Artificial Intelligence (AI) over the last decade. DL has improved many applications related to different fields of Natural Language Processing (NLP), Computer Vision (CV), Speech Recognition (SR), and Reinforcement Learning (RL). A significant challenge in applying DL in most domains is the scarcity of labelled data. Usually, DL models need large amounts of annotated data to train models. The novel paradigm of Self Supervised Learning (SSL) has become a gamechanger in the field of Deep Learning due to its ability to answer the problem of scarcity of labeled data. SSL can utilize commonly available unlabelled data to large DL architectures. SSL usually consists of two stages, the pre-training phase, and the downstream phase. The pre-training phase usually needs a large amount of unlabelled data, and it is a computationally expensive process that could cost hundreds of thousands of dollars to millions. But it has become a common practice to open-source the model checkpoints of pre-trained SSL models that can represent different modalities of data such as text, vision, and speech. Usually, expensive and valuable pre-trained SSL models get open-sourced from tech giants like Google, Meta, Amazon, Nvidia, and Microsoft. These Pre-trained models have become a vital part of research communities and the industry due to their effectiveness in solving many downstream tasks. In this thesis, my focus is on exploring the utilization of pre-trained SSL models in the field of Natural Language Understanding (NLU). NLU is the ability of machines to understand human language. It has enabled many practical applications such as Emotion Recognition, Sentiment Analysis, Summarization, and Question Answering. This thesis mainly explores the utilization of pre-trained SSL models in the two main areas of multimodal fusion and domain adaptation. Under these two main topics, I explored four research question that introduces novel fusion and adaptation techniques.","abstract_has_math":false,"creators":["Siriwardhana, Shamane"],"institution":"ResearchSpace@Auckland","degree_name":"PhD","degree_level":"Doctoral","degree_discipline":"Bioengineering","degree_department":null,"school":null,"contributors":[],"advisors":["Nanayakkara, Suranga","Billinghurst, Mark","Wen, Elliott"],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-11","date_published":"2022-11","updated_at":"2026-07-24T01:06:29Z","subjects":[],"languages":[],"rights":["Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher."],"rights_urls":["https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2292/61963","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Nanayakkara, Suranga","Billinghurst, Mark","Wen, Elliott"]},{"key":"dc:creator","label":"Author","values":["Siriwardhana, Shamane"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-12-01T22:07:00Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-12-01T22:07:00Z"]},{"key":"dc:date.issued","label":"Date","values":["2022-11"]},{"key":"dc:publisher","label":"Institution","values":["ResearchSpace@Auckland"]},{"key":"dc:relation.isreferencedby","label":"Dc Relation Isreferencedby","values":["UoA"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Bioengineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["PhD"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["The University of Auckland"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher."]},{"key":"dc:rights.uri","label":"Rights URI","values":["https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/2292/61963"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Deep Learning (DL) has become a key element in the field of Artificial Intelligence (AI) over the last decade. DL has improved many applications related to different fields of Natural Language Processing (NLP), Computer Vision (CV), Speech Recognition (SR), and Reinforcement Learning (RL). A significant challenge in applying DL in most domains is the scarcity of labelled data. Usually, DL models need large amounts of annotated data to train models. The novel paradigm of Self Supervised Learning (SSL) has become a gamechanger in the field of Deep Learning due to its ability to answer the problem of scarcity of labeled data. SSL can utilize commonly available unlabelled data to large DL architectures. SSL usually consists of two stages, the pre-training phase, and the downstream phase. The pre-training phase usually needs a large amount of unlabelled data, and it is a computationally expensive process that could cost hundreds of thousands of dollars to millions. But it has become a common practice to open-source the model checkpoints of pre-trained SSL models that can represent different modalities of data such as text, vision, and speech. Usually, expensive and valuable pre-trained SSL models get open-sourced from tech giants like Google, Meta, Amazon, Nvidia, and Microsoft. These Pre-trained models have become a vital part of research communities and the industry due to their effectiveness in solving many downstream tasks. In this thesis, my focus is on exploring the utilization of pre-trained SSL models in the field of Natural Language Understanding (NLU). NLU is the ability of machines to understand human language. It has enabled many practical applications such as Emotion Recognition, Sentiment Analysis, Summarization, and Question Answering. This thesis mainly explores the utilization of pre-trained SSL models in the two main areas of multimodal fusion and domain adaptation. Under these two main topics, I explored four research question that introduces novel fusion and adaptation techniques."]},{"key":"dc:title","label":"Title","values":["Fuse and Adapt: Investigating the Use of Pre-Trained Self-Supervising Learning Models in Limited Data NLU problems"]}]}],"canonical_facts":{"dc:contributor.advisor":["Nanayakkara, Suranga","Billinghurst, Mark","Wen, Elliott"],"dc:creator":["Siriwardhana, Shamane"],"dc:date.accessioned":["2022-12-01T22:07:00Z"],"dc:date.available":["2022-12-01T22:07:00Z"],"dc:date.issued":["2022-11"],"dc:description.abstract":["Deep Learning (DL) has become a key element in the field of Artificial Intelligence (AI) over the last decade. DL has improved many applications related to different fields of Natural Language Processing (NLP), Computer Vision (CV), Speech Recognition (SR), and Reinforcement Learning (RL). A significant challenge in applying DL in most domains is the scarcity of labelled data. Usually, DL models need large amounts of annotated data to train models. The novel paradigm of Self Supervised Learning (SSL) has become a gamechanger in the field of Deep Learning due to its ability to answer the problem of scarcity of labeled data. SSL can utilize commonly available unlabelled data to large DL architectures. SSL usually consists of two stages, the pre-training phase, and the downstream phase. The pre-training phase usually needs a large amount of unlabelled data, and it is a computationally expensive process that could cost hundreds of thousands of dollars to millions. But it has become a common practice to open-source the model checkpoints of pre-trained SSL models that can represent different modalities of data such as text, vision, and speech. Usually, expensive and valuable pre-trained SSL models get open-sourced from tech giants like Google, Meta, Amazon, Nvidia, and Microsoft. These Pre-trained models have become a vital part of research communities and the industry due to their effectiveness in solving many downstream tasks. In this thesis, my focus is on exploring the utilization of pre-trained SSL models in the field of Natural Language Understanding (NLU). NLU is the ability of machines to understand human language. It has enabled many practical applications such as Emotion Recognition, Sentiment Analysis, Summarization, and Question Answering. This thesis mainly explores the utilization of pre-trained SSL models in the two main areas of multimodal fusion and domain adaptation. Under these two main topics, I explored four research question that introduces novel fusion and adaptation techniques."],"dc:identifier.uri":["https://hdl.handle.net/2292/61963"],"dc:publisher":["ResearchSpace@Auckland"],"dc:relation.isreferencedby":["UoA"],"dc:rights":["Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher."],"dc:rights.uri":["https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm"],"dc:title":["Fuse and Adapt: Investigating the Use of Pre-Trained Self-Supervising Learning Models in Limited Data NLU problems"],"dc:type":["Thesis"],"thesis:degree_discipline":["Bioengineering"],"thesis:degree_level":["Doctoral"],"thesis:degree_name":["PhD"],"thesis:institution_name":["The University of Auckland"]},"updated_at":"2026-07-24T01:06:29Z"}