Max-Planck-Institut für Informatik
max planck institut
mpii logo Minerva of the Max Planck Society


How not to be seen – inpainting dynamic objects in crowded scenes

Granados, Miguel and Tompkin, James and In Kim, Kwang and Grau, Oliver and Kautz, Jan and Theobalt, Christian

MPI-I-2011-4-001. February 2011, 35 pages. | Status: available - back from printing | Next --> Entry | Previous <-- Entry

Abstract in LaTeX format:
Removal of dynamic scene elements from video is an extremely challenging prob-
lem that even movie professionals often solve through days of manual frame-by-
frame editing. The disoccluded regions in the video have to be inpainted in a
coherent way, even if originally occluded objects or background are dynamic. To
make this problem easier, we propose a new approach for video
inpainting that can
deal with complex scenes with dynamic backgrounds and many non-periodically
moving occluding scene elements. It is built on the idea that a spatio-temporal
hole created by a removed scene element can be filled by copying information
from other space-time locations in the video, where objects and background are
unoccluded. Inpainting is performed by solving a combinatorial optimization
problem that searches for the optimal pattern of pixel shifts. Solving
this problem
naively, even on short videos, quickly becomes infeasible. The primary contribu-
tions of this work are a new energy functional with desirable convergence prop-
erties, an efficient hierarchical solution strategy, and an effective
search space
reduction strategy that restricts potential pixel shifts to regions
around tracked ob-
jects in the scene. A simple interface enables the user to optionally
support the
algorithm in marking and tracking dynamic objects. Our approach can efficiently
inpaint holes even in HD videos with many occlusions, and requires only little
user input.
References to related material:

To download this research report, please select the type of document that fits best your needs.Attachement Size(s):
MPI-I-2011-4-001.pdfMPI-I-2011-4-001.pdf14168 KBytes
Please note: If you don't have a viewer for PostScript on your platform, try to install GhostScript and GhostView
URL to this document:
Hide details for BibTeXBibTeX
  AUTHOR = {Granados, Miguel and Tompkin, James and In Kim, Kwang and Grau, Oliver and Kautz, Jan and Theobalt, Christian},
  TITLE = {How not to be seen – inpainting dynamic objects in crowded scenes},
  TYPE = {Research Report},
  INSTITUTION = {Max-Planck-Institut f{\"u}r Informatik},
  ADDRESS = {Stuhlsatzenhausweg 85, 66123 Saarbr{\"u}cken, Germany},
  NUMBER = {MPI-I-2011-4-001},
  MONTH = {February},
  YEAR = {2011},
  ISSN = {0946-011X},