Example Based 3D Reconstruction from Single 2D Images

T.Hassner and R.Basri

Beyond Patches Workshop at CVPR'06

Abstract

We present a novel solution to the problem of depth reconstruction from a single image. Single view 3D reconstruction is an ill-posed problem. We address this problem by using an example-based synthesis approach. Our method uses a database of objects from a single class (e.g. hands, human figures) containing example patches of feasible mappings from the appearance to the depth of each object. Given an image of a novel object, we combine the known depths of patches from similar objects to produce a plausible depth estimate. This is achieved by optimizing a global target function representing the likelihood of the candidate depth. We demonstrate how the variability of 3D shapes and their poses can be handled by updating the example database on-the-fly. In addition, we show how we can employ our method for the novel task of recovering an estimate for the occluded backside of the imaged objects. Finally, we present results on a variety of object classes and a range of imaging conditions.

 

Full paper : pdf (487kb). BibTeX.


Some Results

More results can be found in the paper.

Human figure reconstruction

Human figure - input image
Human figure - our depth estimate
Human figure - our back depth estimate
Input image
Our depth estimate (rendered in 3D)
Our depth estimate for the man's back
 
 

Fish reconstruction

Fish - input image
Fish - our depth estimate Fish - our depth estimate, textured
Input image
Our depth estimate (rendered in 3D)

Textured renderings of our depth estimate

 

 

Hand reconstruction


Hand - Input image
Hand - our depth estimate
Hand - our back depth estimate
Input image
Our depth estimate (rendered in 3D)
Our depth estimate for the back of the hand


Face reconstruction


Face - input image
Face - our depth estimate
Face - our depth estimate, textured
Cyclops - input image Cyclops - our depth estimate Cyclops - our depth estimate, textured
Input image
Our depth estimate (rendered in 3D)
Textured rendering of our depth estimate

 

 

To the Weizmann Institute of Science Home Page
Faculty of Mathematics and Computer Science
Computer Vision Group



Last update:  July. 30, 2006
For questions and remarks please contact:  Tal Hassner.
Confidential - Proprietary Information.



 
eXTReMe Tracker