DLRNA-BERTa: A transformer approach for RNA-drug interaction prediction

RNA-based therapies are gaining more and more attention due to their ability to target a variety of diseases, including many rare conditions. In this evolving landscape, Bidirectional Encoder Representations from Transformers (BERT) models offer a promising, cost-effective, and efficient approach to accelerate RNA-targeted drug discovery. In this thesis, we propose a RoBERTa-based model (DLRNA-BERTa), a dual language model architecture designed to predict the interactions between small molecules and RNA targets using only textual information across six distinct RNA classes: aptamers, repeats, ribosomal RNAs, riboswitches, miRNAs and viral RNAs. A model was built for each RNA type, with a seventh general-purpose model combining all the data. We use a cross-attention layer and a linear layer computation to allow for interpretation of each token’s contribution to the prediction. Our model outperformed existing RNA-drug interaction prediction approaches. The Pearson correlation coefficients were: 0.94 for aptamers, 0.95 for repeats, 0.93 for ribosomal RNAs, 0.94 for riboswitches, 0.95 for viral RNAs, 0.98 for miRNAs, and 0.94 for the general model, demonstrating strong predictive power across RNA categories. We then tested the performance of our model against four datasets from the ROBIN repository. As our work is computational, we acknowledge that experimental validation remains necessary. Overall, our architectures provide a promising resource to accelerate RNA-targeted drug discovery and contribute to the development of more precise treatments for a broad range of diseases. GitHub repository for the project here.

Grantor dc:publisher

Helsingin yliopisto

Year dc:date.issued

2025

Author dc:creator

• Lobascio, Pasquale

• RNA
• interaction
• machine learning
• transformer
• drug discovery
• drug-target interaction
• DTI
• deep learning
• binding affinity
• pKd

Statement dc:rights

• CC BY-NC-ND 4.0

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/10138/602380

OAI identifier oai:identifier

oai:helda.helsinki.fi:10138/602380