Understanding object recognition performance at scale in machines and humans

Machine performance on the object classication and detection tasks is remark- ably high today. On some datasets, such as ImageNet, it seems to surpass human performance according to recently published results. Yet when we run detectors over real videos we observe that machine performance is far inferior to human performance. We aim to resolve this disconnect and understand the true state of machine and human performance for object recognition. To do this we have gathered a new large image dataset, via the use of Amazon Mechanical Turk, with novel methodology and evaluation mechanisms to both answer questions about how well humans recognize objects and to carefully characterize machine performance. We have found that the performance of current state-of-the-art object detectors drops significantly when run on our dataset: from 71% accuracy to 25% accuracy accuracy. This drop in performance indicates that object detection is not a solved problem, despite previous benchmarks.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2019

Author dc:creator

• Mayo, David Isaac.

Advisor dc:contributor.advisor

• Boris Katz.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/121677

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/121677