Ronen Basri: Research

Hierarchical image segmentation

 

Detecting the boundaries of objects and separating them from surrounding objects is a difficult problem, as is evident by camouflage images. Hierarchical methods offer ways to approach these problems that achieve state-of-the-art performance. Typical algorithms face the notorious ’chicken and egg problem.’ For example, segmentation relies on texture features, which must be computed on regions. But since segmentation is not known in advance the region used typically may cross the boundaries of a segment, leading to degradation of the measurements used for segmentation. Similarly, in edge detection to overcome noise it is desired to smooth the image only along (rather than across) the edges, but the direction of edges can only be inferred robustly after smoothing is taking place. Multiscale methods can avoid this problem by gradually and simultaneously merging regions and measuring features at different scales. Another advantage is that these methods can be made very efficient. During the past several years we developed a segmentation algorithm based on ideas from Algebraic Multigrid (AMG). An executable version is now available online for research purposes only. As this method had quite a number of parameters specifying the integration of the different cues (intensity contrast, texture) at the different scales we recently proposed a parameter-free method that uses probabilistic reasoning to integrate the cues. This method was tested on an extensive human annotated database which is available online. We also extended our basic approach to motion segmentation.

What lies ahead? There is of course still  room to improve segmentation quality. But the main challenge is to use segmentation output for recognition as an alternative (or in addition) to other features. As we cannot be sure to obtain complete shapes as segments, we hope that the hierarchical structure can provide parts that can be glued by higher level processes to complete objects based on prior knowledge.

Hierarchical image segmentation.

Image segmentation results.

The AMG inspired segmentation algorithm was presented in a sequence of papers starting from CVPR 2000. A summary of this algorithm has appeared in
                Eitan Sharon, Meirav Galun, Dahlia Sharon, Ronen Basri, and Achi Brandt, “Hierarchy and adaptivity in segmenting visual                 scenes,” Nature, 442(7104): 719-846, August 17, 2006.

The probabilistic variation appeared in
                Sharon Alpert, Meirav Galun, Ronen Basri, Achi Brandt, “Image segmentation by probabilistic bottom-up aggregation and                 cue integration,” IEEE Conf. on Computer Vision and Pattern Recognition (CVPR-07), 2007.

Motion segmentation can be found in
                Meirav Galun, Alexander Apartsin, and Ronen Basri, “Multiscale segmentation by combining motion and intensity cues,”                 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR-05), San Diego, 256-263, 2005.

We also applied segmentation in medical vision applications, see for example
                Ayelet Akselrod-Ballin, Meirav Galun, Moshe J. Gomori, M. Filippi, P. Valsasina, Ronen Basri and Achi Brandt,                  “Integrated segmentation and classification approach applied to multiple sclerosis analysis,” IEEE Conf. on Computer Vision                 and Pattern Recognition (CVPR-06), New York: 1122-1129, 2006.

An implementation of the SWA algorithm is available online here.

A human annotated database for evaluating segmentation algorithms (see example on right) is available online here.