Pattern Matching 4 - Video, Matching & Tracking¶
Objectives
- Analyze video sequences
- Match objects between frames for tracking
Video import¶
Download the images from the video sequence, and the supervised segmentation ground truth:
(Reference: http://groups.inf.ed.ac.uk/f4k/PAPERS/VIGTA2012.pdf)
import numpy as np
import matplotlib.pyplot as plt
%matplotlib notebook
from skimage.io import imread
import os
vid_path = '../../extern_data/Video/'
gt_path = '../../extern_data/GroundTruth/'
seq0 = [685,705]
seq1 = [1173,1191]
for no in range(*seq1):
fid = os.path.join(vid_path,'frame%04d.jpg'%no)
rgb = imread(fid)
fid = os.path.join(gt_path,'%d.png'%no)
supervision = imread(fid)
plt.figure()
plt.subplot(1,2,1)
plt.title('rgb image')
plt.imshow(rgb)
plt.subplot(1,2,2)
plt.imshow(supervision)
plt.title('ground truth')
break #stop after one frame for test purpose
Background image (Video segmentation)¶
When analyzing video sequences, we can use the movement of the frame to help our object segmentation. In this video, our goal is to segment the fish from the background. The background here is the part of the image that doesn't change from frame to frame: detecting the objects can therefore be done using background subtraction.
Implement the background subtraction algorithm:
- Initialize background image: $B_{0} = I_{0}$ where $I_t$ is the image at time $t$ and $B$ is the estimated background.
- Update background: $B_{t} = \alpha B_{t-1} + (1-\alpha) I_t$
What is the purpose of $\alpha$? How can you determine that your background estimation is "stable" and can be used?
- Subtract background to detect foreground: $F_t = I_t - B_t$
- Find a good post-processing to clean-up the foreground image. Label the individual objects (fish).
- Using the supervised part of the sequence, estimate the detection error of your algorithm.
Hint: useful metrics for segmentation performance include per-pixel accuracy, precision, recall, or F1-score (DICE). These measures can also be made per-object, which is slightly more difficult to compute but in many applications a better indicator of the actual performance of the algorithm.
# -- Your code here -- #
Object tracking¶
- Extract interesting features from the detected objects.
- Suggest a method to recognise and track objects from one frame to the next frame.
# -- Your code here -- #