Contrasting contrastive and supervised models interpretability

In this thesis, we compare the representations of an unsupervised contrastive model to those of an equivalent supervised model using several deep neural network interpretability methods: network dissection, sparsity experiments, and saliency maps. Network dissections of self-supervised contrastive and supervised models show that the neurons of the contrastive model tend to learn about different parts of an object (ie. top-half of a dog or left-half of a person) while the neurons of the supervised model tend to learn about the entire object (ie. a dog or a person). Sparsity experiments show that the representations learned by the contrastive model are less sparse than the representations learned by the supervised counterpart model. Saliency maps show that the contrastive model focuses more on specific parts of the input image. Finally, we find that the contrastive model representations transfer better to finegrained classification tasks than the supervised model representations.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2021

Author dc:creator

• You, Yejin.

Advisor dc:contributor.advisor

• Antonio Torralba and David Bau.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/130717

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/130717