{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/124135"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/124135","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Automatic rare disease extraction based on large language models","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","abstract_has_math":false,"creators":["Cao, Lang"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Sun, Jimeng"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-05","date_published":"2024-05","updated_at":"2026-07-22T22:25:00Z","subjects":["Ai For Healthcare","Large Language Model","Natural Language Processing"],"languages":["en","eng"],"rights":["Copyright 2024 Lang Cao"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/124135","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Sun, Jimeng"]},{"key":"dc:creator","label":"Author","values":["Cao, Lang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-05","2024-04-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Ai For Healthcare","Large Language Model","Natural Language Processing"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Lang Cao"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/124135"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","The student, Lang Cao, accepted the attached license on 2024-04-04 at 16:31.","The student, Lang Cao, submitted this Thesis for approval on 2024-04-04 at 16:39.","This Thesis was approved for publication on 2024-04-05 at 15:19.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20212 on 2024-09-16 at 00:33:04","Identifying and extracting information on rare diseases is crucial in various medical contexts. However, mature rare disease extraction methods are lacking in low-resource settings. In this paper, we aim to create an end-to-end system called AutoRD, which automates extracting information from clinical text about rare diseases. We achieve this using large language models and medical knowledge graphs developed from open-source medical ontologies. Large language models (LLMs) aid in language analysis, while knowledge graphs provide content-specific facts, thus filling in any information gaps. Our system, AutoRD, is a software pipeline involving data preprocessing, entity extraction, relation extraction, entity calibration, and knowledge graph construction. Large language models and open-source data are leveraged throughout the pipeline. We have conducted various tests to evaluate the performance of AutoRD and highlighted its strengths and limitations in this paper. We quantitatively evaluate our system in terms of entity extraction, relation extraction, and the performance of knowledge graph construction. AutoRD achieves an overall F1 score of 47.3%, an improvement of 0.8% compared to the fine-tuned model, and a 14.4% improvement compared to the base LLM. In detail, AutoRD achieves an overall entity extraction F1 score of 56.1% (rare_disease: 83.5%, disease: 35.8%, symptom_and_sign: 46.1%, anaphor: 67.5%) and an overall relation extraction F1 score of 38.6% (produces: 34.7%, increases_risk_of: 12.4%, is_a: 37.4%, is_acronym: 44.1%, is_synonym: 16.3%, anaphora: 57.5%). Our qualitative experiment also demonstrates that the performance in constructing the knowledge graph is commendable. Several designs, including the incorporation of ontologies-enhanced LLMs, contribute to the improvement of AutoRD. AutoRD demonstrates superior performance compared to other methods, demonstrating the potential of LLM applications in rare disease detection and AI for healthcare."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Automatic rare disease extraction based on large language models"]}]}],"canonical_facts":{"dc:contributor":["Sun, Jimeng"],"dc:creator":["Cao, Lang"],"dc:date":["2024-05","2024-04-05"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","The student, Lang Cao, accepted the attached license on 2024-04-04 at 16:31.","The student, Lang Cao, submitted this Thesis for approval on 2024-04-04 at 16:39.","This Thesis was approved for publication on 2024-04-05 at 15:19.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20212 on 2024-09-16 at 00:33:04","Identifying and extracting information on rare diseases is crucial in various medical contexts. However, mature rare disease extraction methods are lacking in low-resource settings. In this paper, we aim to create an end-to-end system called AutoRD, which automates extracting information from clinical text about rare diseases. We achieve this using large language models and medical knowledge graphs developed from open-source medical ontologies. Large language models (LLMs) aid in language analysis, while knowledge graphs provide content-specific facts, thus filling in any information gaps. Our system, AutoRD, is a software pipeline involving data preprocessing, entity extraction, relation extraction, entity calibration, and knowledge graph construction. Large language models and open-source data are leveraged throughout the pipeline. We have conducted various tests to evaluate the performance of AutoRD and highlighted its strengths and limitations in this paper. We quantitatively evaluate our system in terms of entity extraction, relation extraction, and the performance of knowledge graph construction. AutoRD achieves an overall F1 score of 47.3%, an improvement of 0.8% compared to the fine-tuned model, and a 14.4% improvement compared to the base LLM. In detail, AutoRD achieves an overall entity extraction F1 score of 56.1% (rare_disease: 83.5%, disease: 35.8%, symptom_and_sign: 46.1%, anaphor: 67.5%) and an overall relation extraction F1 score of 38.6% (produces: 34.7%, increases_risk_of: 12.4%, is_a: 37.4%, is_acronym: 44.1%, is_synonym: 16.3%, anaphora: 57.5%). Our qualitative experiment also demonstrates that the performance in constructing the knowledge graph is commendable. Several designs, including the incorporation of ontologies-enhanced LLMs, contribute to the improvement of AutoRD. AutoRD demonstrates superior performance compared to other methods, demonstrating the potential of LLM applications in rare disease detection and AI for healthcare."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/124135"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Lang Cao"],"dc:subject":["Ai For Healthcare","Large Language Model","Natural Language Processing"],"dc:title":["Automatic rare disease extraction based on large language models"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:00Z"}