Rethinking Genre Classification with Fine Grained semantic Clustering

Edward Fish, Jon Weinbren, Andrew Gilbert, 
Centre for Creative Arts and Technology (C-CATS) University of Surrey
In International Conference on Image Processing (ICIP'21) 2021

Abstract

Movie genre classification is an active research area in machine learning; however, the content of movies can vary widely within a single genre label. We expand these ‘coarse’ genre labels by identifying ‘fine-grained’ contextual relationships within the multi-modal content of videos. By leveraging pretrained
‘expert’ networks, we learn the influence of different combinations of modes for multi-label genre classification. Then, we continue to fine-tune this ‘coarse’ genre classification network self-supervised to sub-divide the genres based on the multi-modal content of the videos. Our approach is demonstrated
on a new multi-modal 37,866,450 frame, 8,800 movie trailer dataset, MMX-Trailer-20, which includes pre-computed audio, location, motion, and image embeddings.
An overview of the approach. Image and audio features from movie trailers are extracted and their influence is learnt via a collaborative weighting to classify broad genres such as Action, Adventure and Sci-Fi. A self-supervised network then compares these embeddings to generate contextually appropriate sub-genres.

Paper

Rethinking Genre Classification with Fine Grained semantic Clustering, 2021, Edward Fish, Jon Weinbren, Andrew Gilbert, In Proc International Conference on Image Processing (ICIP'21)

Video

Poster

​Citation

@inproceedings{Fish:ICIP:2021,
AUTHOR = Fish, Edward and Weinbren, Jon and Gilbert, Andrew",
TITLE = "Rethinking Genre Classification with Fine Grained semantic Clustering",
BOOKTITLE = "In Proc International Conference on Image Processing (ICIP'21)",
YEAR = "2021",
​}