Multi-view Performance Capture of Surface DetailsReport as inadecuate

Multi-view Performance Capture of Surface Details - Download this document for free, or read online. Document in PDF available to download.

International Journal of Computer Vision

, Volume 124, Issue 1, pp 96–113

First Online: 21 January 2017Received: 22 May 2015Accepted: 29 November 2016


This paper presents a novel approach to recover true fine surface detail of deforming meshes reconstructed from multi-view video. Template-based methods for performance capture usually produce a coarse-to-medium scale detail 4D surface reconstruction which does not contain the real high-frequency geometric detail present in the original video footage. Fine scale deformation is often incorporated in a second pass by using stereo constraints, features, or shading-based refinement. In this paper, we propose an alternative solution to this second stage by formulating dense dynamic surface reconstruction as a global optimization problem of the densely deforming surface. Our main contribution is an implicit representation of a deformable mesh that uses a set of Gaussian functions on the surface to represent the initial coarse mesh, and a set of Gaussians for the images to represent the original captured multi-view images. We effectively find the fine scale deformations for all mesh vertices, which maximize photo-temporal-consistency, by densely optimizing our model-to-image consistency energy on all vertex positions. Our formulation yields a smooth closed form energy with implicit occlusion handling and analytic derivatives. Furthermore, it does not require error-prone correspondence finding or discrete sampling of surface displacement values. We demonstrate our approach on a variety of datasets of human subjects wearing loose clothing and performing different motions. We qualitatively and quantitatively demonstrate that our technique successfully reproduces finer detail than the input baseline geometry.

KeywordsPerformance capture Surface detail Sums of Gaussian Communicated by Lourdes Agapito, Hiroshi Kawasaki, Katsushi Ikeuchi, Martial Hebert.

Electronic supplementary materialThe online version of this article doi:10.1007-s11263-016-0979-1 contains supplementary material, which is available to authorized users.

Download fulltext PDF

Author: Nadia Robertini - Dan Casas - Edilson De Aguiar - Christian Theobalt



Related documents