{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/124709"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/124709","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Automated field compliance checking for improved construction safety using natural language processing and computer vision","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","abstract_has_math":false,"creators":["Wang, Xiyu"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Civil Engineering","degree_department":null,"school":null,"contributors":["El-Gohary, Nora","Golparvar-Fard, Mani","El-Rayes, Khaled","Zhai, ChengXiang"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-05","date_published":"2024-05","updated_at":"2026-07-22T22:25:02Z","subjects":["Construction Safety","Automated Field Compliance Checking","Construction Safety Regulations","Natural Language Processing","Information Extraction","Computer Vision","Visual Information Detection","Knowledge Graphs","Automated Reasoning","Large Language Models."],"languages":["en","eng"],"rights":["Copyright 2024 Xiyu Wang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/124709","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["El-Gohary, Nora","Golparvar-Fard, Mani","El-Rayes, Khaled","Zhai, ChengXiang"]},{"key":"dc:creator","label":"Author","values":["Wang, Xiyu"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-05","2024-04-26"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Civil Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Construction Safety","Automated Field Compliance Checking","Construction Safety Regulations","Natural Language Processing","Information Extraction","Computer Vision","Visual Information Detection","Knowledge Graphs","Automated Reasoning","Large Language Models."]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Xiyu Wang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/124709"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","The student, Xiyu Wang, accepted the attached license on 2024-04-25 at 19:07.","The student, Xiyu Wang, submitted this Dissertation for approval on 2024-04-25 at 19:11.","This Dissertation was approved for publication on 2024-04-26 at 08:36.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20651 on 2024-09-16 at 00:51:02","A large portion of construction-jobsite fatalities and injuries, including those caused by fall, can be attributed to field noncompliances with safety regulations, particularly the Occupational Safety and Health Administration (OSHA) regulations, which include a large number of fall protection requirements for different equipment, facilities, and operations. Current jobsite safety checking practices are time consuming and error prone, because they rely heavily on manual inspection to identify violations. To reduce the time, cost, and errors of such a manual and exhaustive process, there is a need for automated field compliance checking, which seeks to automate the process of extracting safety requirements from OSHA, capturing site condition information, and identifying violations. To address this need, this thesis proposes a new deep learning-based method that utilizes natural language processing (NLP) and computer vision techniques to automatically detect field noncompliances with OSHA. The proposed method includes three main components. First, a set of computational models and NLP methods to automatically extract requirements from safety regulations, including entity extraction, relation extraction, and knowledge graph-based representation. Second, a set of computational models and computer vision methods to automatically detect site condition information from site images, including object detection, attribute recognition, visual relation detection, and scene graph representation. Third, a knowledge graph-based classification and reasoning method to retrieve applicable clauses, detect noncompliances, and generate documentation. The research methodology included seven primary research tasks. First, conducting a comprehensive literature review on related topics such as construction safety, automated compliance checking, machine learning, information extraction, object detection, attribute recognition, visual relation detection, knowledge graphs, and generative models. Second, developing a machine learning-based method to extract entities that describe fall protection requirements from OSHA. Third, developing a machine learning-based method to extract relations that describe fall protection requirements from OSHA and represent the extracted requirements as query graphs. Fourth, developing a machine learning-based method to identify fall-related site objects, with multiple attributes, to provide intricate site condition information. Fifth, developing a machine learning-based method to detect visual relations among fall-related site objects and generate scene graphs to represent the detected site information in a similar structure to the query graphs. Sixth, developing a machine learning-based method to retrieve applicable safety clauses according to the site scenes and conduct knowledge graph-based compliance reasoning. And seventh, integrating the developed computational models and methods into a prototype system and comparing the proposed approach with a general end-to-end approach that utilizes multimodal foundation models. The aforementioned computational models and methods were individually tested, each achieving over 80.0% in precision and recall. The final integrated automated field compliance checking prototype system was evaluated using a set of fall-related site images, as well as fall-related requirements from OSHA. The prototype showed effective performance, achieving an average precision of 71.2%, recall of 73.9%, and F-1 measure of 72.5%, respectively, for noncompliance detection. The generated documentation showed reasonable chain-of-thoughts, precise summarization, and proper suggestions of corrective measures. These experimental results demonstrate the good potential of the proposed automated field compliance checking approach."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Automated field compliance checking for improved construction safety using natural language processing and computer vision"]}]}],"canonical_facts":{"dc:contributor":["El-Gohary, Nora","Golparvar-Fard, Mani","El-Rayes, Khaled","Zhai, ChengXiang"],"dc:creator":["Wang, Xiyu"],"dc:date":["2024-05","2024-04-26"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","The student, Xiyu Wang, accepted the attached license on 2024-04-25 at 19:07.","The student, Xiyu Wang, submitted this Dissertation for approval on 2024-04-25 at 19:11.","This Dissertation was approved for publication on 2024-04-26 at 08:36.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20651 on 2024-09-16 at 00:51:02","A large portion of construction-jobsite fatalities and injuries, including those caused by fall, can be attributed to field noncompliances with safety regulations, particularly the Occupational Safety and Health Administration (OSHA) regulations, which include a large number of fall protection requirements for different equipment, facilities, and operations. Current jobsite safety checking practices are time consuming and error prone, because they rely heavily on manual inspection to identify violations. To reduce the time, cost, and errors of such a manual and exhaustive process, there is a need for automated field compliance checking, which seeks to automate the process of extracting safety requirements from OSHA, capturing site condition information, and identifying violations. To address this need, this thesis proposes a new deep learning-based method that utilizes natural language processing (NLP) and computer vision techniques to automatically detect field noncompliances with OSHA. The proposed method includes three main components. First, a set of computational models and NLP methods to automatically extract requirements from safety regulations, including entity extraction, relation extraction, and knowledge graph-based representation. Second, a set of computational models and computer vision methods to automatically detect site condition information from site images, including object detection, attribute recognition, visual relation detection, and scene graph representation. Third, a knowledge graph-based classification and reasoning method to retrieve applicable clauses, detect noncompliances, and generate documentation. The research methodology included seven primary research tasks. First, conducting a comprehensive literature review on related topics such as construction safety, automated compliance checking, machine learning, information extraction, object detection, attribute recognition, visual relation detection, knowledge graphs, and generative models. Second, developing a machine learning-based method to extract entities that describe fall protection requirements from OSHA. Third, developing a machine learning-based method to extract relations that describe fall protection requirements from OSHA and represent the extracted requirements as query graphs. Fourth, developing a machine learning-based method to identify fall-related site objects, with multiple attributes, to provide intricate site condition information. Fifth, developing a machine learning-based method to detect visual relations among fall-related site objects and generate scene graphs to represent the detected site information in a similar structure to the query graphs. Sixth, developing a machine learning-based method to retrieve applicable safety clauses according to the site scenes and conduct knowledge graph-based compliance reasoning. And seventh, integrating the developed computational models and methods into a prototype system and comparing the proposed approach with a general end-to-end approach that utilizes multimodal foundation models. The aforementioned computational models and methods were individually tested, each achieving over 80.0% in precision and recall. The final integrated automated field compliance checking prototype system was evaluated using a set of fall-related site images, as well as fall-related requirements from OSHA. The prototype showed effective performance, achieving an average precision of 71.2%, recall of 73.9%, and F-1 measure of 72.5%, respectively, for noncompliance detection. The generated documentation showed reasonable chain-of-thoughts, precise summarization, and proper suggestions of corrective measures. These experimental results demonstrate the good potential of the proposed automated field compliance checking approach."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/124709"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Xiyu Wang"],"dc:subject":["Construction Safety","Automated Field Compliance Checking","Construction Safety Regulations","Natural Language Processing","Information Extraction","Computer Vision","Visual Information Detection","Knowledge Graphs","Automated Reasoning","Large Language Models."],"dc:title":["Automated field compliance checking for improved construction safety using natural language processing and computer vision"],"dc:type":["text"],"thesis:degree_discipline":["Civil Engineering"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:02Z"}