Human-Centric Spatio-temporal Modeling and Analysis for Multimedia Data

With the burgeoning advancements in artificial intelligence and deep learning, the spatio-temporal modeling and analysis of multimedia data have ascended to unprecedented importance. My dissertation navigates through the complexities of this domain, particularly emphasizing object analysis and behavior recognition. Here, the accurate and efficient interpretation of human-centric data propels a multitude of intelligent technology applications, ranging from enhancing search and rescue operations in natural disasters to intricately understanding and predicting human behaviors in multifaceted environments. The core challenges in these tasks involve handling data of different scales, modalities, and dynamics and extracting meaningful spatio-temporal features from them. In the static data sphere, my research contributes a novel network specifically designed for 2D Unmanned Aerial Vehicle (UAV) imagery. This innovation adeptly tackles the challenges of varying weather and lighting conditions, paving the way for reliable action detection under a broad spectrum of environmental factors. Moving into the 3D realm, I introduced an Interpolation Graph Convolutional Network, a technique inspired by the foundational operations of convolutional neural networks, to efficiently distill and interpret features from 3D point cloud data. This foundation is crucial for building upon in the subsequent analysis of dynamic data. Venturing into the dynamic data landscape, my research unfolds two distinctive approaches to trajectory prediction. The first approach is an attention-aware social graph transformer network that leverages sequence models to predict motion trajectories in complex public spaces. The second method integrates distributional diffusion into trajectory prediction, embodying a generative model that intricately understands and predicts the uncertainty inherent in human movements. In the realm of 3D dynamic data, the research recognizes the intricate nature of two-person interactions and their profound implications in human social activities. Lastly, we suggest future research directions and conclude the thesis.

Grantor dc:publisher

UNSW, Sydney

Year dc:date

2024

Author dc:creator

• Liu, Yao

Statement dc:rights

• open access
• CC BY 4.0
• free_to_read

Licence

• https://purl.org/coar/access_right/c_abf2
• https://creativecommons.org/licenses/by/4.0/

Identifier

https://doi.org/10.26190/unsworks/30179

OAI identifier oai:identifier

oai:unsworks.library.unsw.edu.au:1959.4/102259