How to draw a bounding box for a video in Matlab

Question

I am interested in knowing how to get a bounding box for detecting blobs in a video. My code involves background subtraction and I am using simple functions:

clc
clear all
close all

m=0; n=0;

readerobj = mmreader('dt2.wmv');% dt2 is my sample fixed cam video
vidframes = read(readerobj);

thresh = 15;

bg = read(readerobj,1);
bg_bw = double(rgb2gray(bg));

fr_size = size(vidframes);
width = fr_size(2);
height = fr_size(1);

fg = zeros(height, width);
numFrames=get(readerobj,'NumberofFrames');

for k = 1 : numFrames
    mov(k).cdata = vidframes(:,:,:,k);
    mov(k).colormap = [];
end

movie(mov, 1, readerobj.FrameRate)

a=1; x=[0 0]; p=0; c=0; 

for i = 2:2:numFrames
    fr = read(readerobj,i); 
    fg = zeros(size(fr));
    fr_bw = rgb2gray(fr);       
    fr_diff = abs(double(fr_bw) - double(bg_bw));  
    for j=1:width                 
      for k=1:height
         if ((fr_diff(k,j) > thresh))
             fg(k,j) = 255; %fr_bw(k,j)
         else
             fg(k,j) = 0;
         end
         if (fr_bw(k,j) > bg_bw(k,j))          
           bg_bw(k,j) = bg_bw(k,j) + 1;           
         elseif (fr_bw(k,j) < bg_bw(k,j))
           bg_bw(k,j) = bg_bw(k,j) - 1;     
         end
     end    

    %median filter to remove noise
    L=medfilt2(fg,[5,5]);            

    %removing small parts less than threshold area
    final=bwareaopen(L,4000);               

    %filling the holes
    ifill=imfill(final,'holes');          

    %to know the number of connected objects
    [Ilabel num]=bwlabel(ifill);          

    if (num>=1)
       %region properties of the image
       Iprops=regionprops(Ilabel);            
       %extracting the bounding box properties
       Ibox=[Iprops.BoundingBox]; 
    end
% ????? What do I do next?
end

After this, I am looking for code which would continuously draw a bounding box around the blob.

Answer 1

If you take a look at the properties of the BoundingBox attribute, there are two fields:

• ul_corner - Denotes the upper left corner of the bounding box
• width - Denotes the width in each dimension

This is usually a 4 element vector if you are dealing with a single frame. As such, the first two elements give you the (x,y) co-ordinates of the top left corner of the image, while the next two elements give you the width spanned in each dimension.

Supposing that your BoundingBox attribute gave you this:

[1 2 11 14]

This means that the top left corner is located at (x,y) = (1,2) with a width of 11 and a height of 14. Remember, the x co-ordinate spans horizontally from left to right, while the y co-ordinate spans vertically from top to bottom.

When you finally obtain this BoundingBox attribute, as seen in your Ibox=[Iprops.BoundingBox]; statement, you can easily superimpose the bounding box onto your frame using the rectangle command. However, keep in mind that the regionprops function may return more than one region. I'm not sure what your overall goal is here, but calling Iprops.BoundingBox and encapsulating within brackets will return ALL of your bounding box properties as a single vector. I would suggest reshaping it so that it is an N x 4 matrix as so:

Ibox = reshape(Ibox, 4, length(Iprops))';

As such, the i'th row will give you the bounding box for the i'th object in your image.

Using the fr variable that you have for the frame, simply do this (assuming that we are delineating the first object in your image):

imshow(fr);
hold on;
rectangle('Position', Ibox(1,:), 'EdgeColor', 'r');

The rectangle command takes in 2 parameters (and optionally more if you want to modify its properties): You can specify if you want a regular rectangle, a rounded rectangle or an ellipse with the first parameter, and the second parameter you give it the dimensions of the rectangle. Since you want a bounding box, you choose Position as the first parameter, and the second parameter is a 4 element vector that denotes the top-left corner of the bounding box, followed by its width and height.

Conveniently, this is given by the BoundingBox attribute from the regionprops structure already, so you just need to substitute Ibox in. You can specify what kind of colour you want for the box. I chose red (r) to make it stand out.

Now if you want to continuously show this for each frame, you simply have to show each image, and call the rectangle function each time.

As an example, let's use a built-in MATLAB image and extract the relevant bounding boxes for each of the shapes.

BW = imread('text.png');
s  = regionprops(BW);

This is a built-in image that contains a bunch of text. The second MATLAB statement will try to extract all of the relevant bounding boxes for each of the characters. What s will return is an 88 element structure. Now, let's draw bounding boxes around the first three characters it detects:

b1 = s(1).BoundingBox;
b2 = s(2).BoundingBox;
b3 = s(3).BoundingBox;

imshow(BW);
hold on;
rectangle('Position', b1, 'EdgeColor', 'r');
rectangle('Position', b2, 'EdgeColor', 'g');
rectangle('Position', b3, 'EdgeColor', 'c');

... and this is what I get. I have delineated one bounding box as red, one as green, and the other as cyan.

If you want to automate this for a single frame, do something like this (assuming you do the reshaping I told you about) and place this where the question marks are in your code:

Ibox = reshape(Ibox, 4, length(Iprops))';
imshow(fr);
hold on;
for k = 1 : size(Ibox, 1)
   rectangle('Position', Ibox(k,:), 'EdgeColor', 'r');
end

Hope this helps!