The Weizmann Institute of Science
Faculty of Mathematics and Computer Science
Computer Vision Lab

Space-Time Super-Resolution from a Single Video

Oded Shahar, Alon Faktor, Michal Irani

This webpage presents the paper "Space-Time Super-Resolution from a Single Video" (CVPR 2011).
Paper [PDF] [bibtex]


Spatial Super Resolution (SR) aims to recover fine image details, smaller than a pixel size. Temporal SR aims to recover rapid dynamic events that occur faster than the video frame-rate, and are therefore invisible or seen incorrectly in the video sequence. Previous methods for Space-Time SR combined information from {multiple} video recordings of the same dynamic scene. In this paper we show how this can be done from a {single video recording}. Our approach is based on the observation that small space-time patches (`ST-patches', e.g., 5x5x3) of a single `natural video', recur many times inside the same video sequence at multiple spatio-temporal scales. We statistically explore the degree of these ST-patch recurrences inside `natural videos', and show that this is a very strong statistical phenomenon. Space-time SR is obtained by combining information from multiple ST-patches at sub-frame accuracy. We show how finding similar ST-patches can be done both efficiently (with a randomized-based search in space-time), and at sub-frame accuracy (despite severe motion aliasing). Our approach is particularly useful for temporal SR, resolving both severe motion aliasing and severe motion blur in complex `natural videos'.

Example Videos:

Fan Video:
Flag Video:
Treadmill Video:
Turbine Video:
Dirty Dancing Video: