Augmented Image Synthesis

It is about the manipulation of real images with consistent illumination, for example how to
augment a real image with virtual objects that are indistinguishable from real objects.
Many applications exist for this kind of rendering, like virtual furniture in photographs of real rooms or virtual characters in real movies.

Two main research directions are investigated: The first objective is a fast augmentation by using the graphics hardware.
One example is a panoramic image viewer (PanoAR) with several possibilities for interactive user manipulations.
The second direction is a high quality augmentation based on a special rendering technique called "Differential Photon Mapping".
This enables to place reflective and refractive virtual objects with their resulting caustics in real images.

Image manipulations with correct lighting require high dynamic range (HDR) images, which are typically combined from a sequence of photographs.
Small camera and object movements during the recording are often unavoidable, resulting in blurry HDR images.
Therefore, I developed a fast and robust HDR generator for such image sequences.

The final part of the talk will explain the interactive augmentation of moving (live) images. Here, the varying illumination is
captured with an HDR video camera. One main aspect is a real-time description of the time-varying near-field illumination
which is often neglected in computer graphics. My vision is, that a user with a head-mounted display can walk
in an arbitrary environment, and the augmentations shown in the display always adjust to the current lighting
conditions.