Separating Transparent Layers


Layer Information Exchange

Bernard Sarel   and   Michal Irani

In this paper we present an approach for separating two transparent layers in images and video sequences. Given two initial unknown physical mixtures, I1 and I2, of real scene layers, L1 and L2, we seek a layer separation which minimizes the structural correlations across the two layers, at every image point. Such a separation is achieved by transferring local grayscale structure from one image to the other wherever it is highly correlated with the underlying local grayscale structure in the other image, and vice versa. This bi-directional transfer operation, which we call the “layer information exchange”, is performed on diminishing window sizes, from global image windows (i.e., the entire image), down to local image windows, thus detecting similar grayscale structures at varying scales across pixels. We show the applicability of this approach to various real-world scenarios, including image and video transparency separation. In particular, we show that this approach can be used for separating transparent layers in images obtained under different polarizations, as well as for separating complex non-rigid transparent motions in video sequences. These can be done without prior knowledge of the layer mixing model (simple additive, alpha-mated composition with an unknown alpha-map, or other), and under unknown complex temporal changes (e.g., unknown varying lighting conditions).

Paper in pdf



Some example results:

We show here some video sequence results. For image results, please look in the paper.

Example 1: Synthetically Mixed Sequence

In this example we have synthetically additively mixed two video sequences into one, to serve as the input for the “Layer Information Exchange” algorithm. The first video sequence has a static image of a woman that translates in the field of view. We have artificially added periodic illumination changes to this sequence (dimming and brightening). The second video sequence portrays the non-rigid motions of small water fountain.


The following MPEG file shows the input sequence and the two recovered video sequence layers. Note that the illumination changes are clearly recovered in the woman video sequence.





Doris and Fountain (1M MPEG)



Example 2: Real Video Transparency

In this example we show a real video transparency sequence. The scene was taken at the entrance to one of the buildings here at the Weizmann Institute of Science (see sketch below).



A camera was placed pointing at a swinging entrance door to a building. The view of the camera includes the scene reflected from the swinging door (the panorama behind the camera), and the scene perceived through the swinging door (the building’s entrance hall). A man is walking behind the camera, thus his image is reflected from the door (along with the back panorama), while a statue and some greenery is seen through the door (in the entrance hall). The indoors scene is static, while the reflected outdoor scene has objects with non-rigid motion (the man), and changing illumination (due to the AGC function of the camera).





Statue Sequence - Input (810K avi)






Statue Sequence - First Recovered Layer (800K avi).




Statue Sequence - Second Recovered Layer (600K avi)

The original filmed input video sequence

The first recovered layer


Portrays the walking man and the back panorama reflected in the swinging door.

The second recovered layer


Portrays the indoor scene with the statue and greenery




eXTReMe Tracker