simple site creator
Mobirise

Tag My Outfit

A multilabel CNN-based classifier for fashion attributes

The Tag My Outfit service predicts the category and the attributes of a clothing item present in a given image. The prediction model called Visual Semantic Attention Model (VSAM) pubished in [Ferreira et al 19] was presented at the Computer Vision for Fashion Workshop@ICCV2019.
VSAM is a compact framework with guided attention for multi-label classification in the fashion domain. This model is supervised by automatic pose extraction creating a discriminative feature space. 

Download and Documentation
Mobirise

Skeleton-driven attention

VSAM uses extra information from a pose detection system to learn the relevant image regions, focusing its discriminative power on distinctive areas. This attention mechanism helps VSAM avoid background artifacts, contours or other salient regions unrelated to the outfit.

Mobirise

Network Arquitecture

VSAM regularizes the VGG-16 backbone network with ground-truth heatmaps with relevant joints.
The regularization is performed on the feature combination from the VGG-16 convolutional blocks 2, 3 and 4, applying a 1x1 convolution for feature selection and re-weighting.
Finally, a spatial attention block highlights informative regions. The model is trained end-to-end by minimizing the cross-entropy loss, weighted by class frequency, for the category level, and the focal loss for the attributes level.
To this loss function, we add the regularization term with the L2-norm pixel-wise difference between the estimated heatmap at layer S and the ground truth heatmap S

Download and run Tag-My-Outfit 

Tag-My-Outfit is part of AI4EU's Resource Catalog and can be downloaded and used under MIT-license (requires registration in the platform)

The Tag-My-Outfit model is delivered as a docker container, runs in CPU (GPU version coming soon) and can be downloaded from the  AI4EU Resource Catalog (https://www.ai4eu.eu/resource/tag-my-outfit).  
Building this resource motivated the development of gRPC4ISR  , a set of code to implement a client, the server and interface modules which are also available for public use. To run Tag-My-Outfit download it from AI4EU Catalog and check the GitHub documentation from  gRPC4ISR
Contact  - Beatriz Quintino-Ferreira ( https://www.ai4eu.eu/user/beatriz-ferreira )
                      Duarte Alves ( https://www.ai4eu.eu/user/duarte-alves )

Learn about gRPC4ISTR