The Comparison of Human and Machine Performance in Object Recognition

Deep learning algorithms have advanced significantly due to better hardware and large training sets, with some claiming they now match or exceed human performance. Recent findings indicate improvements in robustness against Out-Of-Distribution benchmark sets and alignment with human categorisation behaviour. This thesis evaluates various machine learning models to ensure a fair comparison with human performance using principles from comparative psychology. Chapter 2 introduces a psychologically grounded framework for comparing human and machine object categorisation performance, focusing on different presentation times and task complexities. The results indicate that while recent machine learning models have reached or exceeded human performance under optimised conditions, even the most recent advanced machine models struggle with generalisation, especially in certain real-world object categories—particularly in scenarios without fine-tuning or task-specific zero-shot classification. While fine-tuning can significantly enhance machine performance in constrained tasks, this advantage may overestimate machine capabilities relative to humans. In Chapter 3, I test the assumption that human performance remains relatively stable when shifting from N-way categorisation to a free-naming task, while machine performance declines without fine-tuning. The results support this assumption: machines showed a decline in free-naming tasks without fine-tuning, whereas human performance was less affected when compared to their performance in Chapter 2. Additionally, it explores the separation of detection and classification tasks, using bounding boxes to enhance evaluation fairness, revealing that object isolation improves machine performance—although humans also exhibited performance gains, albeit to a lesser extent. This chapter also investigates the role of contextual information in recognition, revealing that context could either assist or hinder human object recognition, depending on its congruence with the object. In contrast, machine models often rely disproportionately on superficial contextual regularities rather than intrinsic object features, supporting the claims that DNNs process images differently than humans. Chapter 4 expanded on previous findings beyond group-level aggregate accuracy by examining individual differences in human categorisation performances and evaluating whether machine models capture the qualitative ordinal relationships characterising human categorisation behaviour, particularly among subgroups exhibiting distinct performance trends. The findings reveal that only CoCa, a multimodal image-text model, aligns with human performance, replicating common ordinal patterns among participants. Finally, the Discussion Chapter integrates these findings to assess the implications for future research, propose methodological refinements for future comparative studies and highlight the importance of considering individual differences when comparing machines and humans in the object recognition task.

Author dc:creator

• Kul, Gokcek

Contributors dc:contributor

• Andy Wills, Peter Jones, Jeremy Goslin

• Artificial Neural Networks
• Object Recognition
• Cognitive Psychology
• Machine Learning

Language dc:language

eng

Repository record dc:identifier

https://pearl.plymouth.ac.uk/psy-theses/150

OAI identifier oai:identifier

oai:pearl.plymouth.ac.uk:psy-theses-1149