{"id":{"repo_id":"vt","oai_identifier":"oai:vtechworks.lib.vt.edu:10919/140538"},"canonical_url":"https://search.dev.ndltd.org/etd/vt/oai:vtechworks.lib.vt.edu:10919/140538","repository":{"repo_id":"vt","name":"Virginia Tech","base_url":"https://vtechworks.lib.vt.edu/oai/request"},"display":{"title":"Human-AI Handshaking: Supporting Extreme Sensemaking through Trustworthy Shared Perception","abstract":"Extreme sensemaking occurs when teams of people must build situational awareness in high-stakes, dynamic environments such as search and rescue, military, or security operations. These contexts are marked by uncertainty, fragmented information, and time-critical decisions that stretch human cognitive and physical limits. Artificial intelligence (AI) offers potential assistance, but distributed AI systems that rely on object detection, such as drone swarms, autonomous vehicles, and large-scale sensor networks, face their own challenges, including fluctuating accuracy, false detections, and fragile resilience under real-world dynamics. This dissertation introduces the \"Human-AI Handshake\", a novel human-AI interaction technique that incorporates human-in-the-loop (HITL) and crowd-in-the-loop (CITL) components to enhance object detection accuracy and shared perception in distributed systems. The Human-AI Handshake addresses core challenges by combining human feedback with AI model performance to mitigate uncertainties and improve trustworthiness. The foundation of the proposed concept comprises four key components: (1) evaluating HITL concepts for improving computer vision accuracy, (2) enhancing shared perception among distributed AI systems to enable better situational awareness, (3) ensuring AI trustworthiness through a synchronized perception trust framework, and (4) interpreting contextual awareness to help AI systems adapt to diverse, real-time scenarios. The technique is tested in extreme sensemaking applications, including augmented reality (AR)-assisted search and rescue, where accurate and reliable object detection is essential. Overall, the Human-AI Handshake provides an assured, scalable solution for improving AI assurance in distributed, dynamic environments, ensuring greater reliability, trust, and performance in critical operations.","abstract_html":"Extreme sensemaking occurs when teams of people must build situational awareness in high-stakes, dynamic environments such as search and rescue, military, or security operations. These contexts are marked by uncertainty, fragmented information, and time-critical decisions that stretch human cognitive and physical limits. Artificial intelligence (AI) offers potential assistance, but distributed AI systems that rely on object detection, such as drone swarms, autonomous vehicles, and large-scale sensor networks, face their own challenges, including fluctuating accuracy, false detections, and fragile resilience under real-world dynamics. This dissertation introduces the "Human-AI Handshake", a novel human-AI interaction technique that incorporates human-in-the-loop (HITL) and crowd-in-the-loop (CITL) components to enhance object detection accuracy and shared perception in distributed systems. The Human-AI Handshake addresses core challenges by combining human feedback with AI model performance to mitigate uncertainties and improve trustworthiness. The foundation of the proposed concept comprises four key components: (1) evaluating HITL concepts for improving computer vision accuracy, (2) enhancing shared perception among distributed AI systems to enable better situational awareness, (3) ensuring AI trustworthiness through a synchronized perception trust framework, and (4) interpreting contextual awareness to help AI systems adapt to diverse, real-time scenarios. The technique is tested in extreme sensemaking applications, including augmented reality (AR)-assisted search and rescue, where accurate and reliable object detection is essential. Overall, the Human-AI Handshake provides an assured, scalable solution for improving AI assurance in distributed, dynamic environments, ensuring greater reliability, trust, and performance in critical operations.","abstract_has_math":false,"creators":["Wilchek, Matthew Ryan"],"institution":"Virginia Tech","degree_name":"Doctor of Philosophy","degree_level":"doctoral","degree_discipline":"Computer Science & Applications","degree_department":"Computer Science and#38; Applications","school":null,"contributors":[],"advisors":[],"committee_chairs":["Luther, Kurt","Batarseh, Feras A."],"committee_members":["Freeman, Laura June","Caba, Wilson","Tilevich, Eli","Bowman, Douglas Andrew"],"year":2025,"date_issued":"2025-12-19","date_published":"2025-12-19","updated_at":"2026-07-22T22:20:11Z","subjects":["Human-Computer Interaction","Human-AI Collaboration","Computer Vision","Trustworthy AI","Object Detection","AI for Surface Water","Search and Rescue"],"languages":["en"],"rights":["Creative Commons Attribution-NonCommercial 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by-nc/4.0/"],"identifier_entries":[{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["vt_gsexam:45081"],"render_values":[{"text":"vt_gsexam:45081","href":null,"code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/10919/140538","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.committeechair","label":"Committee Chair","values":["Luther, Kurt","Batarseh, Feras A."]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Freeman, Laura June","Caba, Wilson","Tilevich, Eli","Bowman, Douglas Andrew"]},{"key":"dc:contributor.department","label":"Department","values":["Computer Science and#38; Applications"]},{"key":"dc:creator","label":"Author","values":["Wilchek, Matthew Ryan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-12-20T09:00:40Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-12-20T09:00:40Z"]},{"key":"dc:date.issued","label":"Date","values":["2025-12-19"]},{"key":"dc:publisher","label":"Institution","values":["Virginia Tech"]},{"key":"dc:type","label":"Dc Type","values":["Dissertation"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science & Applications"]},{"key":"thesis:degree_level","label":"Degree Level","values":["doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Virginia Polytechnic Institute and State University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Human-Computer Interaction","Human-AI Collaboration","Computer Vision","Trustworthy AI","Object Detection","AI for Surface Water","Search and Rescue"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Creative Commons Attribution-NonCommercial 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by-nc/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["vt_gsexam:45081"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/10919/140538"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Extreme sensemaking occurs when teams of people must build situational awareness in high-stakes, dynamic environments such as search and rescue, military, or security operations. These contexts are marked by uncertainty, fragmented information, and time-critical decisions that stretch human cognitive and physical limits. Artificial intelligence (AI) offers potential assistance, but distributed AI systems that rely on object detection, such as drone swarms, autonomous vehicles, and large-scale sensor networks, face their own challenges, including fluctuating accuracy, false detections, and fragile resilience under real-world dynamics. This dissertation introduces the \"Human-AI Handshake\", a novel human-AI interaction technique that incorporates human-in-the-loop (HITL) and crowd-in-the-loop (CITL) components to enhance object detection accuracy and shared perception in distributed systems. The Human-AI Handshake addresses core challenges by combining human feedback with AI model performance to mitigate uncertainties and improve trustworthiness. The foundation of the proposed concept comprises four key components: (1) evaluating HITL concepts for improving computer vision accuracy, (2) enhancing shared perception among distributed AI systems to enable better situational awareness, (3) ensuring AI trustworthiness through a synchronized perception trust framework, and (4) interpreting contextual awareness to help AI systems adapt to diverse, real-time scenarios. The technique is tested in extreme sensemaking applications, including augmented reality (AR)-assisted search and rescue, where accurate and reliable object detection is essential. Overall, the Human-AI Handshake provides an assured, scalable solution for improving AI assurance in distributed, dynamic environments, ensuring greater reliability, trust, and performance in critical operations."]},{"key":"dc:description.abstractgeneral","label":"General Abstract","values":["In high-stakes situations, such as search and rescue, law enforcement, military operations, or environmental monitoring, people must make sense of complex and uncertain environments, often under intense time pressure. Increasingly, these efforts are being augmented with artificial intelligence (AI), which can process large amounts of data quickly but still struggles when conditions are dynamic or unpredictable. People bring valuable judgment, contextual expertise, and adaptability, but these strengths can also introduce bias or inconsistency, raising the challenge of how best to combine them with AI's speed and scale. To bridge this gap, I developed the concept of a ``Human-AI Handshake'' technique. A process for humans and AI systems to share information, check each other's work, and build trust in results. This approach was tested in studies using augmented reality (AR) headsets, canine-mounted sensors, drones, and simulation environments. Results showed that combining human input with AI improved accuracy, reduced errors, and enabled teams to make faster and more reliable decisions. The contributions of this work include designing systems that enable humans to guide and verify AI results, enhancing shared awareness between people and machines, and developing frameworks to measure and strengthen trust in AI. Together, these contributions offer a foundation for designing AI systems that are not only powerful but also dependable and supportive of human decision-making. The Human-AI Handshake technique demonstrates that meaningful collaboration between people and AI can achieve more reliable outcomes in critical operations. This research also highlights the importance of designing AI to be transparent, trustworthy, and aligned with human needs."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Doctor of Philosophy"]},{"key":"dc:format.medium","label":"Dc Format Medium","values":["ETD"]},{"key":"dc:title","label":"Title","values":["Human-AI Handshaking: Supporting Extreme Sensemaking through Trustworthy Shared Perception"]}]}],"canonical_facts":{"dc:contributor.committeechair":["Luther, Kurt","Batarseh, Feras A."],"dc:contributor.committeemember":["Freeman, Laura June","Caba, Wilson","Tilevich, Eli","Bowman, Douglas Andrew"],"dc:contributor.department":["Computer Science and#38; Applications"],"dc:creator":["Wilchek, Matthew Ryan"],"dc:date.accessioned":["2025-12-20T09:00:40Z"],"dc:date.available":["2025-12-20T09:00:40Z"],"dc:date.issued":["2025-12-19"],"dc:description.abstract":["Extreme sensemaking occurs when teams of people must build situational awareness in high-stakes, dynamic environments such as search and rescue, military, or security operations. These contexts are marked by uncertainty, fragmented information, and time-critical decisions that stretch human cognitive and physical limits. Artificial intelligence (AI) offers potential assistance, but distributed AI systems that rely on object detection, such as drone swarms, autonomous vehicles, and large-scale sensor networks, face their own challenges, including fluctuating accuracy, false detections, and fragile resilience under real-world dynamics. This dissertation introduces the \"Human-AI Handshake\", a novel human-AI interaction technique that incorporates human-in-the-loop (HITL) and crowd-in-the-loop (CITL) components to enhance object detection accuracy and shared perception in distributed systems. The Human-AI Handshake addresses core challenges by combining human feedback with AI model performance to mitigate uncertainties and improve trustworthiness. The foundation of the proposed concept comprises four key components: (1) evaluating HITL concepts for improving computer vision accuracy, (2) enhancing shared perception among distributed AI systems to enable better situational awareness, (3) ensuring AI trustworthiness through a synchronized perception trust framework, and (4) interpreting contextual awareness to help AI systems adapt to diverse, real-time scenarios. The technique is tested in extreme sensemaking applications, including augmented reality (AR)-assisted search and rescue, where accurate and reliable object detection is essential. Overall, the Human-AI Handshake provides an assured, scalable solution for improving AI assurance in distributed, dynamic environments, ensuring greater reliability, trust, and performance in critical operations."],"dc:description.abstractgeneral":["In high-stakes situations, such as search and rescue, law enforcement, military operations, or environmental monitoring, people must make sense of complex and uncertain environments, often under intense time pressure. Increasingly, these efforts are being augmented with artificial intelligence (AI), which can process large amounts of data quickly but still struggles when conditions are dynamic or unpredictable. People bring valuable judgment, contextual expertise, and adaptability, but these strengths can also introduce bias or inconsistency, raising the challenge of how best to combine them with AI's speed and scale. To bridge this gap, I developed the concept of a ``Human-AI Handshake'' technique. A process for humans and AI systems to share information, check each other's work, and build trust in results. This approach was tested in studies using augmented reality (AR) headsets, canine-mounted sensors, drones, and simulation environments. Results showed that combining human input with AI improved accuracy, reduced errors, and enabled teams to make faster and more reliable decisions. The contributions of this work include designing systems that enable humans to guide and verify AI results, enhancing shared awareness between people and machines, and developing frameworks to measure and strengthen trust in AI. Together, these contributions offer a foundation for designing AI systems that are not only powerful but also dependable and supportive of human decision-making. The Human-AI Handshake technique demonstrates that meaningful collaboration between people and AI can achieve more reliable outcomes in critical operations. This research also highlights the importance of designing AI to be transparent, trustworthy, and aligned with human needs."],"dc:description.degree":["Doctor of Philosophy"],"dc:format.medium":["ETD"],"dc:identifier.other":["vt_gsexam:45081"],"dc:identifier.uri":["https://hdl.handle.net/10919/140538"],"dc:language.iso":["en"],"dc:publisher":["Virginia Tech"],"dc:rights":["Creative Commons Attribution-NonCommercial 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by-nc/4.0/"],"dc:subject":["Human-Computer Interaction","Human-AI Collaboration","Computer Vision","Trustworthy AI","Object Detection","AI for Surface Water","Search and Rescue"],"dc:title":["Human-AI Handshaking: Supporting Extreme Sensemaking through Trustworthy Shared Perception"],"dc:type":["Dissertation"],"thesis:degree_discipline":["Computer Science & Applications"],"thesis:degree_level":["doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["Virginia Polytechnic Institute and State University"]},"updated_at":"2026-07-22T22:20:11Z"}