I want to remove older lines from Python OpenCV "Sparse Optical Flow"

Question

Python OpenCV > "Sparse Optical Flow" algorithm typically works fine, but for my project, I need to remove the older line from Optical Flow, for example, is it possible to keep the optical lines only for last 5 seconds of each object?

Answer 1

Let's avoid this:

And get this:

I managed to do this by using a sort of mask memory system. Typically, the sparse optical flow method compounds the annotations it tracks onto a single mask variable. This leads to older lines cluttering the screen, as everything is stored on that single mask and never cleared. Rather than this, if you store a few of the most recent masks in a list and cycle through them, you can gradually clear older lines.

Try this:

import cv2 as cv
import numpy as np

# Parameters for Shi-Tomasi corner detection
feature_params = dict(maxCorners = 300, qualityLevel = 0.2, minDistance = 2, blockSize = 7)
# Parameters for Lucas-Kanade optical flow
lk_params = dict(winSize = (15,15), maxLevel = 2, criteria = (cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 0.03))
# The video feed is read in as a VideoCapture object
cap = cv.VideoCapture("PATH/TO/VIDEO/FILE/OR/CAMERA")
# Variable for color to draw optical flow track
color = (0, 255, 0)
# ret = a boolean return value from getting the frame, first_frame = the first frame in the entire video sequence
ret, first_frame = cap.read()
# Converts frame to grayscale because we only need the luminance channel for detecting edges - less computationally expensive
prev_gray = cv.cvtColor(first_frame, cv.COLOR_BGR2GRAY)
# Finds the strongest corners in the first frame by Shi-Tomasi method - we will track the optical flow for these corners
# https://docs.opencv.org/3.0-beta/modules/imgproc/doc/feature_detection.html#goodfeaturestotrack
prev = cv.goodFeaturesToTrack(prev_gray, mask = None, **feature_params)
# Creates an image filled with zero intensities with the same dimensions as the frame - for later drawing purposes
mask = np.zeros_like(first_frame)

iteration = 1  # The nth run of the while loop
frame_retention = 5  # The number of annotated frames showed (higher number means longer duration of motions shown on screen before cleared)

frame_memory = [mask]  # This is where the previous masked frames will be stored. The first frame is blank for future use, the memory length is frame_retention + 1
# Initialize blank frames in frame memory
for i in range(0, frame_retention-1):
    i = np.zeros_like(first_frame)
    frame_memory.append(i)

def create_output_mask(masklist):
    product = masklist[0]
    for i in range(0, len(masklist)-1):
        product = cv.add(product, masklist[i+1])
    masklist = masklist.pop(1)
    return product

while(cap.isOpened()):
    # ret = a boolean return value from getting the frame, frame = the current frame being projected in the video
    ret, frame = cap.read()
    # Converts each frame to grayscale - we previously only converted the first frame to grayscale
    gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
    # Calculates sparse optical flow by Lucas-Kanade method
    # https://docs.opencv.org/3.0-beta/modules/video/doc/motion_analysis_and_object_tracking.html#calcopticalflowpyrlk
    prev = cv.goodFeaturesToTrack(prev_gray, mask = None, **feature_params)
    next, status, error = cv.calcOpticalFlowPyrLK(prev_gray, gray, prev, None, **lk_params)
    # Selects good feature points for previous position
    good_old = prev[status == 1].astype(int)
    # Selects good feature points for next position
    good_new = next[status == 1].astype(int)
    # Draws the optical flow tracks
    for i, (new, old) in enumerate(zip(good_new, good_old)):
        # Returns a contiguous flattened array as (x, y) coordinates for new point
        a, b = new.ravel()
        # Returns a contiguous flattened array as (x, y) coordinates for old point
        c, d = old.ravel()
        # Draws line between new and old position with green color and 2 thickness
        mask = cv.line(mask, (a, b), (c, d), color, 2)
        # Draws filled circle (thickness of -1) at new position with green color and radius of 3
        frame = cv.circle(frame, (a, b), 3, color, -1)
    frame_memory.append(mask)
    overlay_mask = create_output_mask(frame_memory)
    # Overlays the optical flow tracks on the original frame
    output = cv.add(frame, overlay_mask)
    # Updates previous frame
    prev_gray = gray.copy()
    # Updates previous good feature points
    prev = good_new.reshape(-1, 1, 2)
    # Opens a new window and displays the output frame
    cv.imshow("sparse optical flow", output)
    # Resets mask to prevent carry-over from outside of what is stored in frame_memory
    mask = np.zeros_like(first_frame)
    # Frames are read by intervals of 10 milliseconds. The programs breaks out of the while loop when the user presses the 'q' key
    if cv.waitKey(10) & 0xFF == ord('q'):
        break
# The following frees up resources and closes all windows
cap.release()
cv.destroyAllWindows()

Modify the frame_retention variable as needed to manage the duration after which older lines clear.

Hope this helps!

Used code from https://nanonets.com/blog/optical-flow/ and https://github.com/chuanenlin/optical-flow/blob/master/sparse-solution.py as reference.