Saccadic decision-making in dynamic real-world scenes

With every waking hour, we make thousands of rapid eye movements, known as saccades, effortlessly shifting our gaze to perceive the world around us. Each saccade is, whether conscious or not, preceded by a decision-making process about where and when to look next. The complexity of dynamic real-world scenes makes it difficult to analyze how specific aspects of visual attention shape this decision-making process under natural conditions. This thesis investigates what visual attention operates on in complex scenes, how its underlying perceptual units are formed, and what guides visual exploration behavior, specifically in dynamic scenes. To this end, we developed a computational framework and experimental designs to disentangle the contributions of individual attentional mechanisms to the saccadic decision-making process when observing dynamic real-world scenes. In the first study, we systematically explored the role of object-based attention in saccadic decision-making. We introduced our computational framework that simulates the saccadic decision-making process and reproduces the temporal and spatial aspects of human gaze behavior. Its modular and mechanistic design allowed us to assess the implications of object- versus space-based attentional mechanisms by comparing the simulated gaze behavior with human eye-tracking data. The results suggest that object-level perceptual units play a crucial role in attentional processing and form the basis for saccadic selection. Building on this foundation, the second study examined how perceptual units are dynamically formed and refined during active exploration. By incorporating mechanisms that link saccadic decision-making with object segmentation, this model reduces uncertainty and refines its object representations through saccades. This approach reproduced human-like scanpaths by leveraging the active reduction of segmentation uncertainty, rather than relying on explicit inhibitory mechanisms as previous models did. Ablation studies of the interconnected model revealed the importance of semantic cues in object-based attention. The third study extended these insights by empirically investigating how top-down expectations about scene dynamics shape gaze behavior. We found that even in visually identical scenes, expectations about potential scene changes lead to systematic differences in viewing behavior. The computational models and experimental design presented in this thesis enable principled hypothesis testing while maintaining high ecological validity. By bridging experimental findings with their algorithmic implementations, this framework establishes a foundation for future studies of saccadic decision-making and the underlying attentional mechanisms in dynamic real-world scenes.

Author dc:creator

• Roth, Nicolas

Advisor dc:contributor.advisor

• Obermayer, Klaus

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Identifier URI

https://doi.org/10.14279/depositonce-23136

OAI identifier oai:identifier

oai:depositonce.tu-berlin.de:11303/24322