Pre-trained Language Models for Clinical Systematic Literature Reviews

Although systematic literature reviews play a critical role in clinical-based decision making, manual methods for information extraction can sometimes take prohibitively long. In this work, we first describe the construction of datasets in two distinct clinical domains containing randomized trials and observational studies. We then utilize these two datasets to benchmark the performance of Pretrained Language Model (PLM) based entity and relation extraction models as well as the effect of domain specific pre-training prior to their fine-tuning. Our results show evidence to the effectiveness of pre-training using masked language modeling (MLM), a sentence-level proxy task, on boosting the performance of fine-tuned models on both inter- and intra-sentence level information extraction tasks.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2022

Author dc:creator

• Ortiz, Juan M. Ochoa

Advisor dc:contributor.advisor

• Barzilay, Regina

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright MIT

Licence dc:rights.uri

• http://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/143177

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/143177