Video Segmentation by Non-Local Consensus Voting
Alon Faktor and Michal Irani
(BMVC 2014)
Paper [PDF]
Abstract
We address the problem of Foreground/Background segmentation of "unconstrained"
video. By "unconstrained" we mean that the moving objects and the background scene
may be highly non-rigid (e.g., waves in the sea); the camera may undergo a complex
motion with 3D parallax; moving objects may suffer from motion blur, large scale and illumination
changes, etc. Most existing segmentation methods fail on such unconstrained
videos, especially in the presence of highly non-rigid motion and low resolution. We
propose a computationally efficient algorithm which is able to produce accurate results
on a large variety of unconstrained videos. This is obtained by casting the video segmentation
problem as a voting scheme on the graph of similar ('re-occurring') regions in the
video sequence. We start from crude saliency votes at each pixel, and iteratively correct
those votes by 'consensus voting' of re-occurring regions across the video sequence. The
power of our consensus voting comes from the non-locality of the region re-occurrence,
both in space and in time - enabling propagation of diverse and rich information across
the entire video sequence. Qualitative and quantitative experiments indicate that our approach
outperforms current state-of-the-art methods.
Code can be downloaded here
Example Videos: (Press on Video to watch it!)