Theme-Matters: Fashion Compatibility Learning via Theme Attention
Built the first real-world theme-aware fashion dataset comprising 14K around outfits labeled with 32 themes. In this dataset, there are more than 40K fashion items labeled with 152 fine-grained categories. We also propose an attention model learning fashion compatibility given a specific theme.
Abstract
Fashion compatibility learning is important to many fashion markets such as outfit composition and online fashion recommendation. Unlike previous work, we argue that fashion compatibility is not only a visual appearance compatible problem but also a theme-matters problem. An outfit, which consists of a set of fashion items (e.g., shirt, suit, shoes, etc.), is considered to be compatible for a “dating” event, yet maybe not for a “business” occasion. In this paper, we aim at solving the fashion compatibility problem given specific themes. To this end, we built the first real-world theme-aware fashion dataset comprising 14K around outfits labeled with 32 themes. In this dataset, there are more than 40K fashion items labeled with 152 fine-grained categories. We also propose an attention model learning fashion compatibility given a specific theme. It starts with a category-specific subspace learning, which projects compatible outfit items in certain categories to be close in the subspace. Thanks to strong connections between fashion themes and categories, we then build a theme-attention model over the category-specific embedding space. This model associates themes with the pairwise compatibility with attention, and thus compute the outfit-wise compatibility. To the best of our knowledge, this is the first attempt to estimate outfit compatibility conditional on a theme. We conduct extensive qualitative and quantitative experiments on our new dataset. Our method outperforms the state-of-the-art approaches.
Paper Download
Jui-Hsin(Larry) Lai , Bo Wu, Xin Wang, Dan Zeng, Tao Mei, and Jingen Liu, “ Theme-Matters: Fashion Compatibility Learning via Theme Attention ”, arXiv, 2019
Patents
Abstract
本公开提供一种搭配场景识别方法和装置。搭配场景识别装置在物品集合中，计算物品u与物品v的搭配值d(u,v)，在与每个主题场景p相对应的图Gp中，将搭配值d(u,v)作为节点U和节点V之间的边的值，在图Gp中，对各边的值进行加权处理，以得到相应的主题场景值，将具有超过预设门限的主题场景值的图所对应的主题场景作为物品集合对应的搭配场景。本公开能够为不同类型的服饰搭配确定相应的搭配场景。
Device and method for item recommendation based on visual elements
Abstract
A method, a device, and a non-transitory computer readable medium for item recommendation based on visual elements. The method includes: determining, by one or more processors, visual elements from an item image of an item; generating, by the one or more processors, an element descriptor for the item based on at least a part of the visual elements; and calculating, by the one or more processors, a compatibility value between the element descriptor and one or more other element descriptors for one or more other items.
Slide Presentation
Download the dataset
We provide the complete dataset and parts of dataset for validation purposes. Please download accordingly.
All of 13,914 outfits with garment pictures, model pictures, and explicit desciptions.
The fine-grained to coarse-grained mapping table.
11,040 outfits (sampled out of 13,914 outfits) with garment pictures, as the training dataset in our paper.
The json file listing the training outfits.
853 outfits (sampled out of 13,914 outfits) with garment pictures, as the validation dataset in our paper.
The json file listing the validation outfits.
2,021 outfits (sampled out of 13,914 outfits) with garment pictures, as the test dataset in our paper.
The json file listing the test outfits.
In our paper, we cleaned the Polyvore dataset and labeled it with fine-grained categories. Notice that the Polyvore dataset is not labeled with theme tags.
The training dataset of Polyvore dataset.
The validation dataset of Polyvore dataset.
The test dataset of Polyvore dataset.