Inverse Global Illumination

Inverse Global Illumination:
Recovering Reflectance Models of Real Scenes from Photographs

Yizhou Yu Paul Debevec Jitendra Malik Tim Hawkins
Computer Science Division
University of California at Berkeley

We present a method for recovering the reflectance properties of all surfaces in a real scene from a sparse set of photographs, taking into account both direct and indirect illumination. The result is a lighting-independent model of the scene's geometry and reflectance properties, which can be rendered with arbitrary modifications to structure and lighting via traditional rendering methods. Our technique models reflectance with a low-parameter reflectance model, and allows diffuse albedo to vary arbitrarily over surfaces while assuming that non-diffuse characteristics remain constant across particular regions. The method's input is a geometric model of the scene and a set of calibrated high dynamic range photographs taken with known direct illumination. The algorithm hierarchically partitions the scene into a polygonal mesh, and uses image-based rendering to construct estimates of both the radiance and irradiance of each patch from the photographic data. The algorithm computes the expected location of specular highlights, and then analyzes the highlight areas in the images by running a novel iterative optimization procedure to recover the diffuse and specular reflectance parameters for each region. Lastly, these parameters are used in constructing high-resolution diffuse albedo maps for each surface.

The algorithm has been applied to both real and synthetic data, including a synthetic cubical room and a real meeting room. Re-renderings are produced using a global illumination system under both original and novel lighting, and with the addition of synthetic objects. Side-by-side comparisons show success at predicting the appearance of the scene under novel lighting conditions.

This algorithm is extremely useful in image-based modeling and rendering, such as image-based visualization of a real scene and integration between virtual world and reality. It can be used for rendering a real scene with virtual objects, under novel lighting conditions as well as moving specular highlights to the right places for novel viewpoints under original illumination.

Paper Postcript file as it appears in the SIGGRAPH'99 proceedings.
Paper PDF file as it appears in the SIGGRAPH'99 proceedings.
Paper PDF file as it appears in the SIGGRAPH'99 CDROM proceedings.

(Download QuickTime movies at the end)




Before testing on a real room, we did a test on a simulated cubical room with 6 tiny spherical light sources, and compared the recovered parameters with ground truth. The results are shown in the paper.




Some examples of the input radiance images of a real room to our algorithm.




The recovered geometry of the real room: an example of the input geometry to the algorithm. The white dots are recovered camera positions.




We set up 3 concentrated globular light bulbs in the room to help get better estimation of specular parameters.




On the left is one of the images for the whiteboard. There are a lot of specular highlights on it.
On the right is the recovered diffuse albedo map for the whiteboard. It is almost uniform besides the markings.




The diffuse albedo maps of three identical SIGGRAPH'99 posters. Although they were placed at different places inside the room, the recovered albedo maps look almost the same, which shows reflectance is lighting-independent.




On the left is an image with color bleeding effect on the white wall.
On the right is the diffuse albedo map of that part of the wall with color bleeding removed.





The first row shows three real photographs of the room.
The second row shows three corresponding synthetic images at the same viewpoints.




A synthetic panoramic view of the room under the original lighting




A synthetic panoramic view of the room under a novel lighting condition




A synthetic panoramic view of the room with virtual objects


Movie 1: visualization of the hierarchical triangular mesh and the average color of each triangle from photographs
Movie 2: synthetic rendering under the original lighting condition
Movie 3: synthetic rendering under a novel lighting condition
Movie 4: synthetic rendering with virtual objects