Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 70 for “"Attention Mechanism"”.
-
Attention mechanism in deep neural networks for computer vision tasks
<p>“Attention mechanism, which is one of the most important algorithms in the deep Learning community, was initially designed in the natural language processing for enhancing the feature representation of key sentence fragments over the context. In recent years, the attention mechanism has been …
-
A Spatial Attention Mechanism for Motion Prediction in Autonomous Driving
Η ακριβής και αποδοτική πρόβλεψη κίνησης αποτελεί βασικό στοιχείο για την ασφαλή λειτουργία αυτόνομων οχημάτων, τα οποία πρέπει να προβλέπουν τις μελλοντικές τροχιές των γύρω χρηστών του δρόμου σε σύνθετα αστικά περιβάλλοντα. Πρόσφατες αρχιτεκτονικές μετασχηματιστών, όπως το HPTR, προσφέρουν ένα …
-
Long-range Genomics Benchmark Technology and More
… the quadratic computational complexity of the attention mechanism, a core feature of the transformer architecture, poses a significant bottleneck. With the goal of creating a genomics foundation model (FM), this paper aims to address challenges associated long range dependencies in genomics. …
-
Multi-character prediction using attention
We propose a computational attention approach to localize and classify characters in a sequence in a given image. Our approach combines spatial soft-attention with attention regularization and learns “where-to-look” to carry out the sequence classification task. The image is first passed through a …
-
Relation extraction: exploring syntax parsing and constructing it as attention-like structure
Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, …
-
Predicting blood pressure response to fluid bolus therapy in the ICU using attention-based stacked neural networks for clinical interpretability
… this thesis, we utilized regression models and attention-based recurrent neural network (RNN) algorithms to predict the response of hypotensive patients to FBT from a multi-clinical information system large-scale database. We investigated time-series modeling with the use of the stacked long …
-
A Multi-head Attention Approach with Complementary Multimodal Fusion for Vehicle Detection
… distinguished by the integration of a multi-head attention layer. This key enhancement significantly refines the network's capability to process and integrate multimodal sensor data, an aspect that becomes crucial in the face of challenging weather conditions. The effectiveness of this upgraded …
-
A Scene Schema Memory Model for Spatial Relations
… strength which is determined by the amount of attention it receives, and the degree to which it fits the contents of a scene schema. The memory predictions for the integrated model are derived from both the strength and degree of fit of a relation. There are two special cases of the integrated …
-
EEG-based Brain Computer Interface with Deep Learning
… a Long Short-Term Memory (LSTM) network with an attention mechanism to learn the importance of EEG information varying through time, where discriminative information with higher importance is assigned higher scores to better contribute to the classification performance. Our model significantly …
-
AWE: Attention Word Embedding
… We tackle this inefficiency by introducing the Attention Word Embedding (AWE) model, which integrates the attention mechanism into the CBOW model. We also propose AWE-S, which incorporates subword information. We demonstrate that AWE and AWE-S outperform the state-of-the-art word embedding …
-
Improving LLM Long Context Understanding via Synthetic Data and Adaptive Compression
… constrained by the quadratic scaling of the self-attention mechanism, which restricts most popular LLMs to a context length of several thousand tokens. Many methods have been introduced to extend the context of LLMs, including the Activation Beacon approach. In this work, we propose two key …
-
Balancing Memory Efficiency and Accuracy in Spectral-Based Graph Transformers
… Specifically, we reduce the memory use in the attention mechanism and add a random-walk positional encoding to improve upon the SAN graph transformer architecture. We evaluate our model on standard node classification benchmarks: Cora, Citeseer, and Chameleon. Unlike SAN, which runs out of …
-
Interpretable neural networks via alignment and dpstribution Propagation
… object type. During object classification, an attention mechanism can be used to highlight the area of the image that the model focuses on thus offering a narrow view into the mechanism of classification.
-
ScPEFT : a parameter-efficient fine-tuning framework for enhancing single-cell large language models in out-of-context
… under-characterized cell population tasks. Its attention-mechanism analysis identified COVID-related genes associated with specific cell states and uncovered unique blood cell subpopulations, demonstrating scPEFT's capacity for conditionspecific interpretations. These findings position scPEFT as …
-
Self-attention policy architectures for reinforcement learning under partial observability
… necessarily constitutes noise. We explore self-attention-based policy architectures as a solution to this problem, demonstrating their robustness under conditions of high partial observability on different rein-forcement learning benchmark tasks, and explore the advantages and disadvantages …
-
Probing Language Models for Contextual ScaleUnderstanding
… into the object’s embedding through the LM’s attention mechanism.
-
Gated Transformer-Based Architecture for Automatic Modulation Classification
… architecture incorporating a multi-head self-attention mechanism. We train our architecture extensively across a diverse range of signal-to-noise ratios (SNRs) from the RadioML 2018.01A dataset. We introduce a novel transformer-based architecture with a gated mechanism, designed as a runtime …
-
Graph-based Multi-ODE Neural Networks for Spatio-Temporal Traffic Forecasting
There is a recent surge in the development of spatio-temporal forecasting models in many applications, and traffic forecasting is one of the most important ones. Long-range traffic forecasting, however, remains a challenging task due to the intricate and extensive spatio-temporal correlations …
-
IMPROVING MULTI-VARIATE TIME SERIES FORECASTING WITH DYNAMIC MULTI-HEAD ATTENTION ADJACENCY MATRIX
… In this project, we explored whether the attention mechanism can be effectively integrated into non-transformer-based models to enhance their ability to learn spatial information. To achieve this goal, we propose a novel framework that uses a dynamically learned adjacency matrix based on …
Page 1 of 4