Structure, Function, and Interaction in Protein Language Models

In recent years, transformer architectures have shown remarkable capabilities in learning meaningful representations from text and images. This approach has been extended to the realm of protein sequences through pretrained protein language models, which have excelled in various protein engineering tasks. In this thesis, we investigate a pre-trained protein language model’s ability to predict protein structure and the effects of mutations. For many advanced protein understanding tasks, such as predicting protein function and protein-protein interactions, fine-tuning of the model is essential. We explore methods to fine-tune the Evolutionary Scale Modeling (ESM2) model, a pretrained protein language model, for predicting protein functions structured as Gene Ontology terms and predicting protein-protein interactions. Notably, we develop a novel method of modeling the hierarchy constraint in GO term prediction that improves training convergence and test performance while making the model hierarchically consistent with GO. This research aims to enhance our understanding of protein language models in decoding complex biological information, thereby contributing to advancements in computational biology.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2025

Author dc:creator

• Zheng, Jared

Advisors dc:contributor.advisor

• Zhang, Bin
• Jiang, Peng

Statement dc:rights

• Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
• Copyright retained by author(s)

Licence dc:rights.uri

• https://creativecommons.org/licenses/by-nc-nd/4.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/159126

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/159126