Feature-Based Sequence-to-Sequence Matching
Yaron Caspi, Denis Simakov and Michal Irani

Paper: "Feature-Based Sequence-to-Sequence Matching"
Contact us:
  Yaron   Denis   Michal
Our affiliation:
  Computer Vision Group
  Faculty of Mathematics and Computer Science
  Weizmann Institute of Science
Read abstract
See examples:

Wide base-line. Hall sequence.

  1. The setup: scene and cameras

    2. Hall: setup

    Two cameras are looking at each other, capturing a hall between them with people walking, sitting down and rising.

  2. Input sequences

    3. Camera 1 Camera 2
    Hall: input from camera 1 Hall: input from camera 2
    Synchronized video:MPEG 730Kb

  3. Detect moving objects

    4. ... using background subtraction:
    Camera 1 Camera 2
    Hall: mask from camera 1 Hall: mask from camera 2
    Synchronized video: GIF 310Kb

  4. Extract interest points

    5. ... by taking the highest point on each blob, in each frame:
    Camera 1 Camera 2
    Hall: mask and point from camera 1 Hall: mask and point from camera 2

    Construct trajectories from these points

    Camera 1 Camera 2
    Hall: mask and trajectory from camera 1 Hall: mask and trajectory from camera 2
    Synchronized video: GIF 310Kb

  5. Use the trajectories as features for matching algorithm

    6. Camera 1 Camera 2
    Hall: trajectories from camera 1 Hall: trajectories from camera 2
    Each trajectory starts at a green square and stops at a red square.
    There are 11 trajectories for camera 1, and 28 trajectories for camera 2.

  6. Recover epipolar geometry (spatial alignment)

    7. Camera 1 Camera 2
    Hall: epipolar geometry, camera 1 Hall: epipolar geometry, camera 2
    Red points on the left correspond to the red lines on the right (their epipolar lines);
    blue points on the right correspond to the blue lines on the left (their epipolar lines).
    Points are manually put and their epipolar lines calculated using recovered fundamental matrix.
    White cross marks location of the opposite camera (the ground-truth epipole).

    ... and sub-frame time shift (temporal alignment)

To the top of the page Last modified: Oct 06, 2006