Understanding the rich world of outfits: a study of fashion compatibility, latent style, and outfit behavior

Many computer vision applications in the fashion domain require solving tasks where complex relationships between images, such as the notion of item compatibility, are being learned. We take a metric learning approach to representing compatibility between pairs of items. First, we introduce a model that learns compatibility relationships in dedicated embedding subspaces dependent on item type, which results in significant gains on established fashion compatibility prediction tasks. Second, we present a method for learning a richer notion of compatibility across multiple compatibility conditions whose contributions are learned as a latent variable, which provides better performance on established tasks while requiring fewer embedding subspaces to be learned. Third, we make the first published attempt at diagnosing the salient features of a pair of items that make them compatible, and linking them to human-interpretable concepts. Finally, we demonstrate that our representation of outfits enables diverse, novel, and practically-useful visual search queries for the fashion domain, and results in semantically-meaningful style summaries with several directions for future work.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2020

Author dc:creator

• Vasileva, Mariya Ivanova

Contributors dc:contributor

• Forsyth, David A
• Hoiem, Derek
• Schwing, Alexander
• Berg, Tamara L

• Computer vision
• machine learning applications
• explainability
• embedding models
• vision and language
• image search and retrieval
• style summarization
• fashion compatibility

Statement dc:rights

• Copyright 2020 Mariya I. Vasileva

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/108720

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/108720