Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 27 for “"Transformer-based models"”.
-
Transformer-based models for answer extraction in text-based question/answering
The success of transformer-based language models has led to a surge of research in various natural language processing tasks, among which extractive question-answering/answer span detection, has received considerable attention in recent years. However, to date, no comprehensive studies have been …
-
IMPROVING MULTI-VARIATE TIME SERIES FORECASTING WITH DYNAMIC MULTI-HEAD ATTENTION ADJACENCY MATRIX
… Existing research, including neural network-based models and transformer-based models, has demonstrated high performance in learning temporal information. However, capturing spatial information within time series data remains a significant challenge. In this project, we explored whether the …
-
Empirical Analysis of Neural Architectures and Side Information in Financial Time Series Forecasting
… price data, compare recurrent architectures and transformer-based models, and evaluate multiple training strategies. Our key contributions include: (1) evidence that options-derived input features improve both error metrics and directional accuracy; (2) a comparison study of four training methods …
-
Making Sense of Training Large AI Models
… of optimization is the training of large AI models. But currently such models are trained with ad-hoc heuristics at a very large computational cost, mainly due to lack of understanding of their working mechanisms. In this thesis, we conduct a systematic study of large-model optimization, …
-
Evaluating convolutional neural networks and transformer architectures for image-based prediction of protein localization in eukaryotic cells
… deep learning have enabled high-throughput image-based methods to tackle this problem by leveraging large-scale immunofluorescence microscopy datasets. The aim of this study is to comparatively evaluate convolutional neural network (CNN) architectures and Transformer- based models for the …
-
TOWARDS EFFICIENT TRANSFORMER SCALING
Transformer-based models have achieved exceptional performance across various tasks but face resource limitations when scaling. This thesis explores strategies to enhance Transformer efficiency. First, we propose WideNet, which optimizes parameter efficiency using parameter-sharing and …
-
Balancing Memory Efficiency and Accuracy in Spectral-Based Graph Transformers
The transformer architecture has been a significant driving force behind advancements in deep learning, yet transformer-based models for graph representation learning have not caught up to mainstream Graph Neural Network (GNN) variants. A major limitation is the large O(𝑛2) memory consumption of …
-
Vigilis: Leveraging Language Models for Fraud Detection in Mobile Communications and Financial Transactions
… application that employs advanced language models to counter such attacks in calls, texts, and payments. We first collect and make available a corpus of fraudulent calls from the Internet and train lightweight transformer-based models that achieve fraud detection accuracies of up to 94% and …
-
Combining the Power of Attention Models and Many-objective Computational Intelligence Algorithms for Drug Design
AI-based approaches have been recently applied to in silico drug design. However, existing approaches and protocols consider the absorption, distribution, metabolism, excretion, and toxicity (ADMET) pharmacokinetic properties of drug candidates in a later stage of drug design processes, where …
-
Accelerating the Design Process Through Natural Language Processing-based Idea Filtering
… We demonstrate the ability of machine learning models to predict design metrics from the design itself and textual survey information. Our results show that incorporating NLP improves prediction results across design metrics, and that clear distinctions in the predictability of certain metrics …
-
Structural Robustness of Transformer Models for Clinical Text Summarization on MIMIC-III
Transformer-based models are increasingly used to summarize clinical documents, yet their performance is evaluated exclusively on well-formatted text. In practice, clinical notes undergo structural degradation through hospital mergers, EHR migrations, and copy-paste practices, conditions that no …
-
Modeling with Attention in Demand Forecasting and Beyond
… Traditionally there have been many simple models that extrapolate trends and seasonal patterns from individual time series in order to forecast future values, but not until recently have DNNs (Deep Neural Networks) been leveraged to capture complex relationships between time series as well …
-
High-dimensional Multimodal Bayesian Learning
… Functional data, and (3) Bayesian Inference in Transformer-based Models. Chapter 2 in our work examines a two-tiered data structure; to simultaneously explore the variable selection and identify dependency structures among both higher and lower-level variables, we propose a multi-level …
-
Building small domain-specific masked language models vs. large generative models for clinical decision support and their effects on users.
… Traditional explicit knowledge-representation based AI involves reasoning over symbolic representation of statements standing for such “justified true beliefs” [1], the modern connectionist methodology however replaces explicit reasoning with making a prediction based on a set of computations …
-
Innovating the Study of Self-Regulated Learning: An Exploration through NLP, Generative AI, and LLMs
… language processing (NLP) and large language models (LLMs) to analyze student self-regulated learning (SRL) strategies in response to exam wrappers. Exam wrappers are structured reflection activities that prompt students to practice SRL after they get their graded exams back. The dissertation …
-
Adapting Transformers for Structured Data Domains
… to enhance the adaptability and effectiveness of Transformers in structured data domains beyond their traditional use in natural language processing (NLP). We revisit key elements of the transformer framework - including input representations, attention formulations, auxiliary tasks, prediction …
-
eMARLIN: Addressing Coordination and Partial Observability in Distributed Reinforcement Learning for Traffic Signal Control
… (ATSC). ATSC adapts traffic signal timing based on real-time traffic data, but the increasing granularity and decentralization of control introduce new challenges. These include coordination among decentralized controllers, in-field limitations in distributive communication, and partial …
-
BERT-Based Intrusion Detection System for RF Jamming Attacks in Vehicular Network
… a comprehensive study on the application of transformer-based models for RF jamming detection in intelligent transportation systems. Specifically, we evaluate the performance of four pre-trained transformer architectures—BERT, RoBERTa, DistilBERT, and ALBERT—fine-tuned for the multi-class …
-
Next-Generation Intelligent Portfolio Management
… portfolio management framework that leverages Transformer-based models and Large Language Models (LLMs) to enhance return predictions and sentiment extraction from extensive financial texts coupled with robust DRL trading agents to optimize portfolio performance. We introduce an adaptive …
Page 1 of 2