Chapter 6: Motion & Tracking

This chapter delves into the fundamental concepts and techniques for understanding and tracking motion within video sequences. We will explore essential algorithms for motion analysis and build practical applications for tracking objects.

Key Concepts

1. Background Subtraction

Background subtraction is a foundational technique used to isolate moving objects from a static background in a video. The core idea is to maintain a model of the background and compare each new frame against this model. Pixels that deviate significantly from the background model are identified as foreground pixels, representing potential moving objects.

Common Techniques:

• Frame Differencing: The simplest form of background subtraction. It involves subtracting the current frame from the previous frame. While easy to implement, it is highly sensitive to noise and can only detect moving pixels, not identify objects that have stopped moving.

2. Frame Differencing

Frame differencing is a straightforward method for detecting motion by comparing consecutive frames.

How it works:

1. Take two consecutive frames, Frame A and Frame B.
2. Calculate the absolute difference between corresponding pixels in Frame A and Frame B.
3. Threshold the resulting difference image. Pixels with a difference above the threshold are considered motion or changed pixels.

Limitations:

• Sensitivity to Noise: Any minor camera shake or sensor noise can be misinterpreted as motion.
• Object Stopping: If an object stops moving, it will no longer be detected by frame differencing after its initial movement.
• Background Changes: Non-static backgrounds (e.g., swaying trees, flickering lights) will lead to false positives.

Hands-on: Build a Basic Tracking System on Video

This section will guide you through building a basic motion tracking system. We will use common techniques to identify and follow moving objects in a video.

3. Kalman Filter

The Kalman Filter is a powerful recursive algorithm that estimates the state of a dynamic system from a series of incomplete and noisy measurements. In the context of tracking, it can predict the future position of an object based on its past trajectory and current measurements, even when those measurements are imperfect.

How it applies to tracking:

• State Estimation: The Kalman Filter estimates the object's position, velocity, and potentially acceleration.
• Prediction: It predicts the object's state in the next time step.
• Correction: It corrects the predicted state using the actual measurement (e.g., detected object position) from the current frame.
• Noise Reduction: It effectively filters out noise from the measurements, leading to smoother and more robust tracking.

4. Optical Flow

Optical flow is a technique that estimates the motion of objects, surfaces, and edges in a sequence of images. It represents the apparent velocity of pixels between consecutive frames.

Key Optical Flow Algorithms:

• Lucas-Kanade Method: A widely used algorithm for sparse optical flow. It assumes that the optical flow is constant within a small neighborhood of pixels. It is computationally efficient and works well for tracking a few specific feature points.

  • Pros: Computationally efficient, good for sparse tracking.
  • Cons: Sensitive to lighting changes, requires good feature points.

• Farneback Method: An algorithm for dense optical flow. It estimates the flow vector for every pixel in the image, providing a richer representation of motion across the entire frame. It is more robust to noise and lighting variations than sparse methods but is computationally more expensive.

  • Pros: Dense motion estimation, more robust.
  • Cons: Computationally intensive.

By understanding and implementing these techniques, you will gain a solid foundation in analyzing and tracking motion in video data.