Optimising image feature zero-shot-learning with EEG

In this thesis, brain-computer interfaces (BCIs) are proposed as the future of our communication with machines. Once our thoughts can be directly understood and actioned, huge gains in terms of productivity and usability from consumer to professional applications can be envisioned. However, in its current state, BCI systems are limited in use to research and medical applications (in assistive technologies and in diagnoses).<br/><br/>This project presents a BCI system for image retrieval where the system is able to identify the specific image that the user is looking at. In contrast to the majority of BCI research, we contribute work under several real world restrictions. Namely, our processing must be fully automated, use a recording technology available to consumers, and should be able to search for images not present during training, i.e. we apply zero-shot learning.<br/><br/>To achieve this goal, we first investigate a discriminative feature extraction process for EEG brain data and establish a baseline evaluation of exemplar and category-level decoding. Once competitive decoding rates are achieved, we extend this framework to address our zero-shot restriction by making use of computer vision, linguistic, and traditional machine learning techniques. Finally, we demonstrate improvement over these results by designing a set of metric-learning-based deep neural network architectures, representing a more data-driven and state-of-the-art approach. This results in the first time a multi-modal network based on metric learning has been applied to BCI image retrieval. Our network resulted in a peak decoding rate of 5.85\% which we demonstrate is well situated against existing studies in order to give context in a field where appropriate result comparison is difficult.

Name dc:type.qualificationname

Doctor of Philosophy

Level dc:type.qualificationlevel

Doctoral Thesis

Grantor dc:publisher.institution

Queen's University Belfast

Year dc:date.issued

2020

Author dc:creator

• McCartney, Ben

Advisors dc:contributor.advisor

• Martinez del Rincon, Jesus
• Devereux, Barry

• EEG
• Machine Learning
• BCI
• Zero-shot Learning

Language dc:language

eng

Identifier

oai:pure.qub.ac.uk/portal:studenttheses/d20ffc0c-fb5f-42d9-a7dc-c72bc609de4e

OAI identifier oai:identifier

oai:pure.qub.ac.uk/portal:studenttheses/d20ffc0c-fb5f-42d9-a7dc-c72bc609de4e