Digital Symbol Digit Test: Multimodal Behavior Detection and Visualization

Neurodegenerative diseases, such as Alzheimer’s, impact many people worldwide and currently have no cure, making early detection essential for effective symptom management and intervention. Traditional diagnostic practices often rely on subjective clinical evaluations that can vary between practitioners, highlighting the need for more objective methods. The digital Symbol Digit Test (dSDT), administered via the Cognitive Health App on an iPad and using the ETVision Eye Tracking System, aims to provide an automated, reliable method to analyze patient cognitive function to detect early signs of impairment through capturing handwriting and gaze data. This thesis builds upon previous work by automating the synchronization of these two data modalities, refining definitions of learning behaviors, and developing pipelines for data processing and visualization. By creating a synchronized multimodal dataset, we can visualize participant behavior for more intuitive interpretation and draw meaningful conclusions. These contributions provide an end-to-end framework for analyzing behavior during the cognitive assessment and lay the groundwork for future development of diagnostic models to detect early signs of neurodegenerative diseases.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2025

Author dc:creator

• Xu, Jessica J.

Advisor dc:contributor.advisor

• Davis, Randall

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/162968

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/162968