Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras, 

Andrew Gilbert [1], Matthew Trumble [1], Adrian Hilton [1], John Collomosse [1,2]
[1] University of Surrey,  [2] Adobe Research
In British Machine Vision Conference (BMVC'20), 2020

Abstract

We aim to simultaneously estimate the 3D articulated pose and high fidelity volumetric occupancy of human performance, from multiple viewpoint video (MVV) with as few as two views. We use a multi-channel symmetric 3D convolutional encoder-decoder with a dual loss to enforce the learning of a latent embedding that enables inference of skeletal joint positions and a volumetric reconstruction of the performance. The inference is regularised via a prior learned over a dataset of view-ablated multi-view video footage of a wide range of subjects and actions, and show this to generalise well across unseen
subjects and actions. We demonstrate improved reconstruction accuracy and lower pose estimation error relative to prior work on two MVV performance capture datasets: Human 3.6M and TotalCapture.
Network architecture. The input is a low fidelity geometric proxy (VL) from two wide baseline camera views. This proxy is passed through a decoder-encoder (D;E) to produce a 3D human pose estimate (joint angles J(VL)) via the latent space and to output, a high-fidelity geometric proxy (VH) regularised via discriminator (D).

Paper

Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras, Andrew Gilbert, Matthew Trumble, Adrian Hilton, John Collomosse, In British Machine Vision Conference (BMVC'20), 2020

Videos

Presentation Slides
Results Showreal

​Citation

@inproceedings{Gilbert:BMVC:2020,
AUTHOR = Gilbert, Andrew and Trumble, Matthew and Hilton, Adrian and Collomosse, John",
TITLE = "Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras,",
BOOKTITLE = "In Proc British Machine Vision Conference (BMVC'20)",
YEAR = "2020",
​}