Virtual Photo Sets

Capturing reality for computer graphics applications

Siggraph Asia 2015 Course
Jonas Unger, Andrew Gardner, Per Larsson, and Francesco Banterle

Over the last decade, we have seen the development of new computer graphics applications in, for example, product visualization and augmented reality where it is a requirement that the rendered images exhibit the quality and accuracy needed to make them comparable to a photograph of the same scene. A common goal in these applications is to render virtual objects as placed into background photographs, or digitized models of real scenes. This demand of ever increasing fidelity and accuracy has spurred the development of highly efficient physically based rendering algorithms for accurate computation of light transport, techniques for capturing high fidelity lighting environments in real scenes, and methods for accurate measurement of material properties such as reflectance, color, and surface textures. These tools, methods, and algorithms now make it possible to build digital models of real scenes where every aspect including geometry, lighting, and material properties are modeled based on accurate measurements. This enables the appearance of virtual objects to be simulated so that they can be placed into real scenes and appear as if they where actually there. We call such a model a Virtual Photo Set (VPS).

This course covers algorithms and tools for measuring, calibrating, and registering the information required to build models of real scenes with the intent of photo-realistic image synthesis. We focus on the pipeline, i.e. how it all fits together, and practical artist-driven methods and algorithms for capturing and fusing all captured data into a VPS: a geometric 3D-model of the scene and the lighting (radiance distribution) in the scene, so that photo-realistic renderings of virtual objects can be produced.
Course notes can be downloaded here