MPI-INF Logo
Campus Event Calendar

Event Entry

What and Who

Varifocal Lenses for Focus-Supporting Near-Eye Displays & Learning domain-specific cameras: End-to-end optimization of optical sensing pipelines

Nitish Padmanaban & Vincent Sitzmann
Stanford Computational Imaging Lab
Talk

Nitish Padmanaban is a third year PhD candidate at Stanford EE, supported by a NSF Graduate Research Fellowship. He is advised by Prof. Gordon Wetzstein as part of the Stanford Computational Imaging Lab. His research is currently focused on optical and computational techniques for virtual and augmented reality. In particular he's spent the couple of years working on building and evaluating displays to alleviate the vergence–accommodation conflict, and has also looked into the role of the vestibular system conflicts in causing motion sickness in VR. Nitish received an MS from Stanford EE in 2017, and a BS in EECS from UC Berkeley in 2015.
Vincent Sitzmann is a Ph.D. student working in Professor Gordon Wetzstein’s Computational Imaging Laboratory in the Department of Electrical Engineering at Stanford University. He is doing research in computational imaging and vision systems, lying at the intersection of machine learning, optimization, optics and computer vision. He previously received his Master’s Degree in computer science from Stanford University and his Bachelor’s degree in Electrical Engineering from the Technical University of Munich.
AG 2, AG 4, RG1, MMCI  
Public Audience
English

Date, Time and Location

Monday, 19 March 2018
11:00
60 Minutes
E1 4
019
Saarbrücken

Abstract

Varifocal Lenses for Focus-Supporting Near-Eye Displays
Virtual and augmented reality (VR/AR) systems are gaining popularity due to their ability to provide highly immersive experiences. Furthermore, they are expected to become a major computing platform in the future. However, their current implementation with fixed focal elements in many devices lacks important focus cues that are usually present in the real world. This lack of ability to refocus, or accommodate, while still being able to fixate to different depths via convergence and divergence of the eyes leads to visual discomfort, eye strain, and headaches. A relatively newer technology that can be used to alleviate some of these concerns is the varifocal lens. We incorporate varifocal lenses into the optical pipeline of a prototype VR setup and use this test system to design, implement, and evaluate several approaches to reducing the conflict between depth cues. These approaches include adaptive focus, which updates the virtual image of the VR scene to match the user's gaze; accommodation-invariant mode, which disables the retinal blur cue entirely so that accommodation can be freely driven by vergence; and monovision, which provides different planes for each eye to passively provide more planes to which the user can accommodate. We then consider the different accessibility implications of these displays for older and younger users, and evaluate all three approaches with respect to how well they drive the accommodative system.
Learning domain-specific cameras: End-to-end optimization of optical sensing pipelines
A large part of today’s images are consumed by machines - for instance in assisted / autonomous driving, security, and multimedia. Yet, oftentimes, these systems have been optimized in a sequential manner, where the optical system is designed first, then the image processing pipeline is tuned for good reconstruction performance, and finally, semantic algorithms such as segmentation or classification networks are tuned on the processed image.This is not only laborious and expensive, it is also suboptimal. In this talk, I will show that we can do better by optimizing camera pipelines jointly with higher-level post-processing for specific applications. Specifically, I will talk about how end-to-end optimized image processing can help us classifying images at 3 lux, substantially outperforming approaches that solely rely on a classification network. Similarly, I will show how we can learn deep image priors that allow us to achieve state-of-the-art performance in a variety of linear inverse reconstruction problems, and allow us to train for any differentiable loss (such as a perceptual loss). Finally, I will discuss recent work on extending this paradigm to the optics of the camera. This line of work takes steps towards end-to-end optimizing whole optical sensing pipelines for applications that need only be specified by a dataset, a forward model and a loss function.

Contact

Ellen Fries
9325-4000
--email hidden
passcode not visible
logged in users only

Ellen Fries, 03/07/2018 09:53 -- Created document.