StyleBabel: Artistic Style Tagging and Captioning
Dan Ruta [1], Andrew Gilbert [1] Pranav Aggarwal [2] , Naveen Marri [2] , Ajinkya Kale [2] , Jo Briggs [3] , Chris Speed [4] , Hailin Jin [2] , Baldo Faieta [2] , Alex Filipkowski [2] , Zhe Lin [2] , and John Collomosse [1,2]
[1] University of Surrey, [2] Adobe Research, [3] University of Northumbria, [4] University of Edinburgh
In European Conference on computer Vision ECCV'22
[1] University of Surrey, [2] Adobe Research, [3] University of Northumbria, [4] University of Edinburgh
In European Conference on computer Vision ECCV'22
Abstract
We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools. StyleBabel was collected via an iterative method, inspired by `Grounded Theory': a qualitative approach that enables annotation while co-evolving a shared language for fine-grained artistic style attribute description. We demonstrate several downstream tasks for StyleBabel, adapting the recent ALADIN architecture for fine-grained style similarity, to train cross-modal embeddings for: 1) free-form tag generation; 2) natural language description of artistic style; 3) fine-grained text search of style. To do so, we extend ALADIN with recent advances in Visual Transformer (ViT) and cross-modal representation learning, achieving a state of the art accuracy in fine-grained style retrieval.
|
|
The StyleBabel Grounded Annotation process. Moodboards (examples, left) are annotated via an iterative process (right) that encourages groups of participants to evolve and converge to a shared language for describing style. See the paper for stage descriptions.
Paper
|
StyleBabel: Artistic Style Tagging and Captioning, Dan Ruta, Andrew Gilbert, Pranav Aggarwal, Naveen Marri, Ajinkya Kale, Chris Speed, Hailin Jin, Baldo Faieta, Alex Filipkowski, Zhe Lin, and John Collomosse, In Proc ECCV'22 2022
|
Video
|
|
Poster
Citation
@inproceedings{Ruta:ECCV:2022,
AUTHOR = Ruta, Dan and Gilbert, Andrew and Aggarwal, Pranav and Marri, Naveen and Kale, Ajinkya and Speed, Chris and Jin, Hailin and Faieta, Baldo and Filipkowski, Alex and Lin, Zhe and Collomosse, John",
TITLE = "StyleBabel: Artistic Style Tagging and Captioning",
BOOKTITLE = "European Conference on computer Vision ECCV'22 ",
YEAR = "2022",
}
AUTHOR = Ruta, Dan and Gilbert, Andrew and Aggarwal, Pranav and Marri, Naveen and Kale, Ajinkya and Speed, Chris and Jin, Hailin and Faieta, Baldo and Filipkowski, Alex and Lin, Zhe and Collomosse, John",
TITLE = "StyleBabel: Artistic Style Tagging and Captioning",
BOOKTITLE = "European Conference on computer Vision ECCV'22 ",
YEAR = "2022",
}
