Ronen Basri: Research

Recognition and alignment with point features

 

My PhD thesis dealt with the question of recognizing (specific) 3D objects in 2D images. The main question was can we represent an object using a small number of views instead of an explicit 3D representation. I then showed that indeed, under the orthographic projection, if we can match features in these views we can use only two views to represent a whole aspect where every novel view can be shown to be a linear combination of feature coordinates in the two model views. In addition, I derived a model to describe the observed location of the silhouette of smooth curved objects by using the 3D curvatures along the silhouette. This model was also shown to be linear and extractable from three views. In a later study we examined the use of various distance measures between 3D objects and 2D images and their relations. Years later we returned to the question of alignment hoping that advances in understanding lighting can make alignment method more practical. We then constructed a method for fast indexing (i.e. selecting the correct object and pose from a database) based on David Jacobs’s affine indexing scheme to which we added a component that checks consistency with lighting based on harmonic representations. Some results are shown in the figures below.

What lies ahead? Alignment methods were criticized (justly) for being restricted to handling views of the same object. At the same time, current categorization methods mostly are restricted to single view recognition. Incorporating alignment into categorization approaches can potentially lead to multiview recognition. And of course there is the need to develop effective ways to deal with the notorious correspondence problem.

Representing rigid objects under orthographic projection by a linear combination of views was presented in
                Shimon Ullman and Ronen Basri, “Recognition by linear combinations of models,” IEEE Transactions on Pattern Analysis                 and Machine Intelligence, 13(10): 992-1006, 1991.

The alignment of smooth curved objects is described in
                Ronen Basri and Shimon Ullman, “The alignment of objects with smooth surfaces,” Computer Vision, Graphics, and Image                 Processing: Image Understanding, 57(3): 331-345, 1993.

The discussion of distance measures to compare 3D objects with 2D views appeared in
                Ronen Basri and Daphna Weinshall, “Distance metric between 3D models and 2D images for recognition and                classification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(4): 465-470, 1996.

Our indexing method that solves simultaneously for lighting and pose appeared in
                Ira Kemelmacher and Ronen Basri, “Indexing with unknown illumination and pose,” IEEE Conf. on Computer Vision and                 Pattern Recognition (CVPR-05), San Diego: 909 – 916, 2005.

Another related papers deal with constructing invariant representations for objects that belong to constraints classes
                Ronen Basri and Yael Moses, “When is it possible to identify 3D objects from single images using class constraints?”                 International Journal of Computer Vision, 33(2): 40-61, 1999.

And yet another deals with the formulation of the so called ‘paraperspective projection’ and its relation to the affine transformation
                Ronen Basri, “Para perspective ≡ affine,” International Journal of Computer Vision, 19(2): 169-180, 1996.

Indexing under lighting and pose. Two input images (left images), voting maps (middle) - points in the image vote to the object in the database they are consistent with, each object represented by a different color, and the indexed objects rendered in the recovered pose and lighting (right). The 3D laser scans used for this work are available for download here.

Dino

Hippo

Camel

Pinokio

Bear

Elephant

Shark

Face