Aligning Sequences and Actions by Maximizing Space-Time Correlations

Y. Ukrainitz and M. Irani

ECCV 2006 version here (pdf)


We introduced an algorithm for sequence alignment, based on maximizing local space-time correlations. Our algorithm aligns sequences of the same action performed at different times and places by different people, possibly at different speeds, and wearing different clothes. Moreover, the algorithm offers a unified approach to the problem of sequence alignment for a wide range of scenarios (e.g., sequence pairs taken with stationary or jointly moving cameras, with the same or different photometric properties, with or without moving objects). Our algorithm is applied directly to the dense space-time intensity information of the two sequences (or to filtered versions of them). This is done without prior segmentation of foreground moving objects, and without prior detection of corresponding features across the sequences. Examples of challenging sequences with complex actions are shown, including ballet dancing, actions in the presence of other complex scene dynamics (clutter), as well as multi-sensor sequence pairs.

Sample videos are:

Multi-Sensor Alignment: Moving Cameras

Multi-Sensor Alignment: Moving Objects

Action Alignment

Action Alignment

Action Alignment vs. Background Alignment

Robustness to Noise

The Locking Property

Last changed on February 25, 2006