Scalable Techniques for Organizing and Visualizing Community Photo Collections

The increasing popularity of digital photography and online photo-sharing sites such as Flickr is creating large photo collections of landmarks and popular destinations around the world, commonly known as Community Photo Collections. These image collections, however, first need to be structured before any useful information used about them. One of the simplest way to organize them would be constructing an Image match graph capturing every matching image with an edge between the images. The current advances in Computer Vision, not only allow the unsupervised discovery of matching images in large unstructured collections, but also, along with the advances made in Structure from Motion, in particular, it is now possible to automatically do a 3D scene reconstruction from these image collections. However, the improving the scalability, accuracy, quality etc. of these techniques is still an actively researched area. In this talk, I would talk about the following two contributions made in this field.
Exhaustive pairwise matching to build match graphs on large datasets presents serious practical challenges, and has mostly remained an unexplored domain. We make a step in this direction by demonstrating the feasibility of scalable indexing and fast retrieval of appearance and geometric information in images. We identify unification of database filtering and geometric verification steps, used in most of the Image retrieval systems, as a key step for doing this. Hence, we device a novel inverted indexing scheme, based on Bloom filters, to scalably index high order features extracted from pairs of nearby features. Unlike a conventional inverted index, we can adapt the size of the inverted index to maintain adequate sparsity of the posting lists. This ensures constant time query retrievals. We are thus able to implement an exhaustive pairwise matching scheme, with linear time complexity, using the `query each image in turn' technique.
Secondly, we present a scalable and incremental approach for creating interactive image-based walkthroughs from a dynamically growing collection of photographs of a scene. Prior approaches, such as Photo Tourism, perform a global scene reconstruction as they require the knowledge of all the camera poses. These are recovered via batch processing involving pairwise image matching and Structure from motion, on collections of photographs. Both steps can become computational bottlenecks for large image collections. Instead of computing a global reconstruction and all the camera poses, our system utilizes several partial reconstructions, each of which is computed from only a small subset of overlapping images. These subsets are efficiently determined using a Bag of Words-based matching technique. Our framework easily allows an incoming stream of new photographs to be incrementally inserted into an existing reconstruction. We demonstrate that an image-based rendering framework based on only partial scene reconstructions can be used to navigate large collections containing thousands of images without sacrificing the navigation experience. As our system is designed for incremental construction from a stream of photographs, it is well suited for processing the ever-growing photo collections.