Fairness for affective and wellbeing computing

Recent advancements in machine learning (ML) as well as affective and wellbeing computing methodologies have enabled affective and wellbeing computing technologies to be increasingly used and integrated into daily human life. However, the problem of bias in machine-learning based tools and systems are becoming an increasing source of concern. Bias can be understood as discrimination against individuals based on certain sensitive attributes such as age, race and gender. Fairness conversely dictates that no individual or subgroup should be advantaged or disadvantaged based on their inherent characteristics. Such risks are also present in the field of affective computing as affect recognition tools are increasingly deployed in a wide range of high-stake use-cases such as mental wellbeing prediction and robotic mental wellbeing coaching. Till date, a thorough evaluation of the existing fairness evaluation measures and bias mitigation methods available, a deeper understanding of the primary sources of bias and an understanding of the factors hampering bias mitigation efforts in affective and wellbeing computing is still lacking. This thesis thus attempts to advance research on machine learning fairness for affective and wellbeing computing. We first conduct an evaluation of the existing research on fairness as well as the bias mitigation methods available. Subsequently, we further investigate factors hampering bias mitigation efforts in affective and wellbeing computing. We identify the challenges unique to affective and wellbeing computing and demonstrate how existing bias mitigation strategies are inadequate to fully address the challenges identified. For instance, they are unable to address challenges such as inappropriate data pre-processing and the inherent subjectivity of the expression and manifestation of affective states. This resulted in existing ML methods to either fail to address the bias present or even exacerbate the bias that they were trying to mitigate. Given that causality is posited to be able to address the gaps induced by the associational nature of traditional ML and that recent work in causal fairness has indicated that a causal approach towards task-specific ML fairness is promising, we investigate the utility of causality to achieve fairness via the use of counterfactuals and provide preliminary causality-based tools to analyse the source of bias. Our findings indicate that counterfactuals can be effective at improving fairness and counterfactual fairness is effective as an orthogonal form of fairness measure to assess the bias present. Our proof-of-concept using causal structure learning also indicates that it may be possible to identify to source of bias with the use of causal graphs and causal structure learning. The promising results across the two case studies inspired us to further investigate subsequent application for advancing bias mitigation efforts in affective and wellbeing computing with the use of causality-based tools. To overcome the limitations in existing multimodal methods, we propose FairReFuse, a novel referee-based multimodal framework with causal intervention, for fairer predictions in a multimodal classification setting. To achieve group-level gender fairness, we implement causal interventional debiasing using backdoor adjustment in order to achieve fairer representational learning for each modality. To achieve individual-level fairness, we propose a referee network which learns to combine the predictions of the modalities dynamically using the individual fairness scores. We demonstrate that our method was able to provide significant improvements in group and individual fairness across three different audio-visual depression datasets. Moreover, given that the overarching objective of this thesis is on ensuring a positive societal impact, we investigate the efficacy of the proposed method on three applied deployment of wellbeing recognition technology within wellbeing coaching settings. Existing research has indicated that resampling outperforms reweighting for correcting sampling bias. Given the above, we propose a simple and effective data augmentation strategy, MixFeat, to debias small datasets. Our results indicate that multimodal approaches typically outperform unimodal approaches across performance metrics. However, not all multimodal approaches lead to an automatic reduction in bias. Second, we find that models trained with the high-level features generally perform better across both the unimodal and multimodal setups. Third, our results show that the proposed data augmentation method more consistently improves fairness across both the uni and multimodal experiments compared to the baseline data balancing method. We further provide recommendations on how to achieve fairer outcomes when using machine learning algorithms for the real-world deployment of affective and wellbeing computing related technology. The investigation, proposed methods and frameworks presented in this dissertation initiate a novel field of enquiry exploring principled methods to achieve ML fairness for affective and wellbeing computing. The main contribution of this thesis is the development of real-world grounded and feasible methods in order to advance research on fairness closer to the real-world application of affective and wellbeing computing.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Cheong, Jiaee

Advisors dc:contributor.advisor

• Gunes, Hatice
• Kalkan, Sinan

• Affective and Wellbeing Computing
• Causality
• Machine Learning Fairness

Licence

• https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/ccc1c049-e3ef-4e0e-90ae-3d2370fe3bbf/download
• https://creativecommons.org/licenses/by/4.0/

Language dc:language

eng

Author Identifier

0000-0001-5964-2284

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/381213