Machine Learning Methods for functional Near Infrared Spectroscopy

<p>Identification of user state is of interest in a wide range of disciplines that fall under the umbrella of human machine interaction. Functional Near Infra-Red Spectroscopy (fNIRS) device is a relatively new device that enables inference of brain activity through non-invasively pulsing infra-red light into the brain. The fNIRS device is particularly useful as it has a better spatial resolution than the Electroencephalograph (EEG) device that is most commonly used in Human Computer Interaction studies under ecologically valid settings. But this key advantage of fNIRS device is underutilized in current literature in the fNIRS domain.</p> <p>We propose machine learning methods that capture this spatial nature of the human brain activity using a novel preprocessing method that uses `Region of Interest' based feature extraction. Experiments show that this method outperforms the F1 score achieved previously in classifying `low' vs `high' valence state of a user. </p> <p>We further our analysis by applying a Convolutional Neural Network (CNN) to the fNIRS data, thus preserving the spatial structure of the data and treating the data similar to a series of images to be classified. Going further, we use a combination of CNN and Long Short-Term Memory (LSTM) to capture the spatial and temporal behavior of the fNIRS data, thus treating it similar to a video classification problem. We show that this method improves upon the accuracy previously obtained by valence classification methods using EEG or fNIRS devices. Finally, we apply the above model to a problem in classifying combined task-load and performance in an across-subject, across-task scenario of a Human Machine Teaming environment in order to achieve optimal productivity of the system.</p>

Name thesis:degree_name

Doctor of Philosophy (PhD)

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical Engineering and Computer Science

Year

2018

Author dc:creator

• Bandara, Danushka Sandaruwan

Contributors dc:contributor

• Senem Velipasalar
• Leanne M. Hirshfield

• Brain activity
• Deep Learning
• fNIRS
• Human Computer Interaction
• Machine Learning
• Multi channel sensor
• Engineering

Repository record dc:identifier

https://surface.syr.edu/etd/953

OAI identifier oai:identifier

oai:surface.syr.edu:etd-1954