Stereo-browsing from Calibrated Cameras

Abstract

Modern Structure-from-Motion (SfM) methods enable the registration of a set of cameras and the reconstruction of the corresponding sparse point cloud of the object/location depicted in the input images. Despite the quality of such techniques they often fail where the input point cloud has a very low density thus decreasing the final user experience. On the other side, modern image-based rendering (IBR) techniques try to avoid a full reconstruction of the geometry by empowering the user with interfaces for the smooth navigation of the acquired images. In such methods, images viewed from viewpoints in-between the actual cameras are generated in some way, for example by using a textured proxy or by warping properly the input images. Usual navigation interfaces, however, neglect to use the inherent nature of such set of cameras which, despite having a wide-baseline, are often well spatially organized as they usually maintain a good overlap between images, varying smoothly both the position and the orientation of the camera. Given such a set of registered cameras, we present a framework for the stereoscopic exploration of the object/location depicted using any type of stereoscopic devices. In the proposed system, the users can have a full tridimensional experience without the need of a complete 3D reconstruction. Our method starts by building a graph where each node is associated to a calibrated camera that represents a virtual eye. Two virtual eyes give a stereo pair. Along each edge of this graph we can instantiate a novel virtual camera using simple linear interpolation of the extrinsic parameters and we can generate its corresponding novel view by using known IBR techniques. This, in practice, extends the domain of the possible views from the discrete set of acquired cameras to a continuous domain given by our graph. Combining any couple of cameras that we can pick on this graph we obtain the set of all possible stereo pairs, that is the codomain of our graph. We give a formal definition of this space, that we called StereoSpace. Built on this, we designed our prototype system for the stereoscopic exploration of photo collections.