MPI-INF/SWS Research Reports 1991-2021

2. Number - only D4


Enhanced dynamic reflectometry for relightable free-viewpoint video

Theobalt, Christian and Ahmed, Naveed and Lensch, Hendrik and Magnor, Marcus and Seidel, Hans-Peter

July 2006, 37 pages.

Status: available - back from printing

Free-Viewpoint Video of Human Actors allows photo- realistic rendering of real-world people under novel viewing conditions. Dynamic Reflectometry extends the concept of free-view point video and allows rendering in addition under novel lighting conditions. In this work, we present an enhanced method for capturing human shape and motion as well as dynamic surface reflectance properties from a sparse set of input video streams. We augment our initial method for model-based relightable free-viewpoint video in several ways. Firstly, a single-skin mesh is introduced for the continuous appearance of the model. Moreover an algorithm to detect and compensate lateral shifting of textiles in order to improve temporal texture registration is presented. Finally, a structured resampling approach is introduced which enables reliable estimation of spatially varying surface reflectance despite a static recording setup. The new algorithm ingredients along with the Relightable 3D Video framework enables us to realistically reproduce the appearance of animated virtual actors under different lighting conditions, as well as to interchange surface attributes among different people, e.g. for virtual dressing. Our contribution can be used to create 3D renditions of real-world people under arbitrary novel lighting conditions on standard graphics hardware.

  • MPI-I-2006-4-006.pdf
  • Attachement: 06799002.pdf (13752 KBytes)

URL to this document:

Hide details for BibTeXBibTeX
  AUTHOR = {Theobalt, Christian and Ahmed, Naveed and Lensch, Hendrik and Magnor, Marcus and Seidel, Hans-Peter},
  TITLE = {Enhanced dynamic reflectometry for relightable free-viewpoint video},
  TYPE = {Research Report},
  INSTITUTION = {Max-Planck-Institut f{\"u}r Informatik},
  ADDRESS = {Stuhlsatzenhausweg 85, 66123 Saarbr{\"u}cken, Germany},
  NUMBER = {MPI-I-2006-4-006},
  MONTH = {July},
  YEAR = {2006},
  ISSN = {0946-011X},