MPI-I-2014-4-002

Using hand gestures as input in human--computer interaction is of ever-increasing interest. Markerless tracking of hands and fingers is a promising enabler, but adoption has been hampered because of tracking problems, complex and dense capture setups, high computing requirements, equipment costs, and poor latency. In this paper, we present a method that addresses these issues. Our method tracks rapid and complex articulations of the hand using a single depth camera. It is fast (50~fps without GPU support) and supports varying close-range camera-to-scene arrangements, such as in desktop or egocentric settings, where the camera can even move. We frame pose estimation as an optimization problem in depth using a new objective function based on a collection of Gaussian functions, focusing particularly on robust tracking of finger articulations. We demonstrate the benefits of the method in several interaction applications ranging from manipulating objects in a 3D blocks world to egocentric interaction on the go. We also present extensive evaluation of our method on publicly available datasets which shows that our method achieves competitive accuracy.