Classification of Illumination Methods for Mixed Reality

Abstract

A mixed reality (MR) represents an environment composed both by real and virtual objects. MR applications are more and more used, for instance in surgery, architecture, cultural heritage, entertainment, etc. For some of these it is important to merge the real and virtual elements using consistent illumination. In this paper, we propose a classification of illumination methods for MR applications that aim at generating a merged environment in which illumination and shadows are consistent. Three different illumination methods can be identified: common illumination, relighting and methods based on inverse illumination. In this report a classification of the illumination methods for MR is given based on their input requirements: the amount of geometry and radiance known from the real environment. This led us to define four categories of methods that vary depending on the type of geometric model used for representing the real scene, and the different radiance information available for each point of the real scene. Various methods are described within their category. The classification points out that in general the quality of the illumination interactions increases with the amount of input information available. On the other hand, the accessibility of the method decreases since its pre-processing time increases to gather the extra information. Recent developed techniques managed to compensate unknown data with clever techniques using an iterative algorithm, hardware illumination or recent progress in stereovision. We complete the review of illumination techniques for MR with a discussion on important properties such as the possibility of interactivity or the amount of complexity in the simulated illumination.