Action detection in in-classroom firefighter training: A 360-degree video analytics service

In hazardous situations like firefighting, split-second decisions can differentiate between success and failure, between life and death. Understanding the actions taken by firefighters in various scenarios can help inform decision-making processes, enabling responders to make more informed choices under pressure. Identifying actions provides valuable feedback for training programs. By reviewing and analyzing recorded firefighting actions, the instructors at the firefighter training facility can identify areas for improvement and tailor training sessions to address specific challenges or shortcomings. After a firefighting operation, analyzing the actions taken can provide insights into what worked well and what could be improved for future incidents. This post-incident analysis is essential for continuous learning and refinement of firefighting tactics and procedures, and to avoid any dangerous mistakes in the actions being performed on the field. This research focuses on a system combining the advantage of 360-degree videos and deep learning to automatically detect important actions being performed on the field during training in the panoramic scene, assisting firefighting instructors in classroom teaching scenarios. Specifically, we summarize the salient actions and events relevant to firefighting through multiple interviews with experienced firefighting instructors. Using a unique dataset collected from a firefighting training institute, our approach successfully helps in identifying and detecting the important actions in firefighting like carrying a civilian, operating a hose, breaking a door or window, etc. For a better reviewing experience for the user, we have integrated our system with a latency-aware Viewing and Query Service (VQS) that allows the user to choose which action they want to focus on in the video of their choice, and also at which time stamp they want to jump to save time.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Tiwari, Aditi

Contributors dc:contributor

• Nahrstedt, Klara

• 360-degree Video
• Firefighting
• Action Detection

Statement dc:rights

• © 2024 Aditi Tiwari

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/124576

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F124576