Self-supervised Representation Learning in Computer Vision and Reinforcement Learning

This work is devoted to self-supervised representation learning (SSL). We consider both contrastive and non-contrastive methods and present a new loss function for SSL based on feature whitening. Our solution is conceptually simple and competitive with other methods. Self-supervised representations are beneficial for most areas of deep learning, and reinforcement learning is of particular interest because SSL can compensate for the sparsity of the training signal. We present two methods from this area. The first tackles the partial observability providing the agent with a history, represented with temporal alignment, and improves performance in most Atari environments. The second addresses the exploration problem. The method employs a world model of the SSL latent space, and the prediction error of this model indicates novel states required to explore. It shows strong performance on exploration-hard benchmarks, especially on the notorious Montezuma's Revenge. Finally, we consider the metric learning problem, which has much in common with SSL approaches. We present a new method based on hyperbolic embeddings, vision transformers and contrastive loss. We demonstrate the advantage of hyperbolic space over the widely used Euclidean space for metric learning. The method outperforms the current state-of-the-art by a significant margin.

Grantor dc:publisher

Università degli studi di Trento

Year dc:date

2022

Author dc:creator

• Ermolov, Aleksandr

Contributors dc:contributor

• Sebe, Niculae

• self-supervised learning, representation learning, contrastive loss, reinforcement learning, metric learning

Statement dc:rights

• info:eu-repo/semantics/openAccess
• license:Creative commons
• license uri:http://creativecommons.org/licenses/by-sa/4.0/

Language dc:language

eng

Identifier

http://dx.doi.org/10.15168/11572_360781
10.15168/11572_360781

OAI identifier oai:identifier

oai:iris.unitn.it:11572/360781