Separating Transparent Layers

through

Layer Information Exchange

Bernard Sarel   and   Michal Irani
ECCV'2004.

Abstract
In this paper we present an approach for separating two transparent layers in images and video sequences. Given two initial unknown physical mixtures, I₁ and I₂, of real scene layers, L₁ and L₂, we seek a layer separation which minimizes the structural correlations across the two layers, at every image point. Such a separation is achieved by transferring local grayscale structure from one image to the other wherever it is highly correlated with the underlying local grayscale structure in the other image, and vice versa. This bi-directional transfer operation, which we call the “layer information exchange”, is performed on diminishing window sizes, from global image windows (i.e., the entire image), down to local image windows, thus detecting similar grayscale structures at varying scales across pixels. We show the applicability of this approach to various real-world scenarios, including image and video transparency separation. In particular, we show that this approach can be used for separating transparent layers in images obtained under different polarizations, as well as for separating complex non-rigid transparent motions in video sequences. These can be done without prior knowledge of the layer mixing model (simple additive, alpha-mated composition with an unknown alpha-map, or other), and under unknown complex temporal changes (e.g., unknown varying lighting conditions).

Paper in pdf

 

 

Some example results:

We show here some video sequence results. For image results, please look in the paper.

Example 1: Synthetically Mixed Sequence

In this example we have synthetically additively mixed two video sequences into one, to serve as the input for the “Layer Information Exchange” algorithm. The first video sequence has a static image of a woman that translates in the field of view. We have artificially added periodic illumination changes to this sequence (dimming and brightening). The second video sequence portrays the non-rigid motions of small water fountain.

 

The following MPEG file shows the input sequence and the two recovered video sequence layers. Note that the illumination changes are clearly recovered in the woman video sequence.

 

 

Example 2: Real Video Transparency

In this example we show a real video transparency sequence. The scene was taken at the entrance to one of the buildings here at the Weizmann Institute of Science (see sketch below).

 

 

A camera was placed pointing at a swinging entrance door to a building. The view of the camera includes the scene reflected from the swinging door (the panorama behind the camera), and the scene perceived through the swinging door (the building’s entrance hall). A man is walking behind the camera, thus his image is reflected from the door (along with the back panorama), while a statue and some greenery is seen through the door (in the entrance hall). The indoors scene is static, while the reflected outdoor scene has objects with non-rigid motion (the man), and changing illumination (due to the AGC function of the camera).