Space-Time Video Completion

Y. Wexler, E. Shechtman and M. Irani

NEW: This method is now part of the Content Aware Fill feature in Adobe Photoshop CS5.

A journal preprint from TPAMI March 2007, can be found here (pdf, 30MB).
A shorter version from CVPR 2004 is here (pdf, 3.5MB).


We present a method for space-time completion of large space-time holes in video sequences of complex dynamic scenes. The missing portions are filled-in by sampling spatio-temporal patches from the available parts of the video, while enforcing global spatio-temporal consistency between all patches in and around the hole. This is obtained by posing the task of video completion and synthesis as a global optimization problem with a well-defined objective function.The consistent completion of static scene parts simultaneously with dynamic behaviors leads to realistic looking video sequences. Space-time video completion is useful for a variety of tasks, including, but not limited to:

An implementation inspired by this work for filling images is available here.

Sample videos:

Note: these are new results -- improved since the video demo in CVPR'2004. See details in the journal version.
Added raw uncompressed data for some of the following examples. These videos can be used in publications provided the above paper is cited.
(These are uncompressed avis that for some reason don't always open. QuickTime player ususally works.)

Removal of a beach umbrella

Raw data may be found here (zip, 9MB).
Removal of the jogging lady

Raw data may be found here (zip, 5MB).
Reconstruction of three missing frames in a sequence
Removal of the waving lady

Raw data may be found here (zip, 32MB).
Retouching of old Chaplin movie with simulated large "coffee stain"
Full field-of-view of stabilized video.

The input was taken from this nice work

Last changed on Dec 8, 2010