Does Hu moments remain unchanged for a slightly changed detected shape-OpenCV?

Question

Below you can see a set of binary shapes that belong to the same object being detected in a video based on color as main feature representation.

Since i'm depending on object color in the detection, there might be other irrelevant object presented in the binary mask image because they simply have similar color distribution as the target object of interest (the object to be detected is red car or green car for example)

I tried to distinguish the object based the area and aspect ratio. However, it didn't work efficiently as there were other detected irrelevant objects which their area and aspect ratio fall with in the boundaries of the object to be detected as shown in the below image.

In the image above the target object is the top white spot.

And because it is hard to maintain color consistency due ti illumination effects, i'm trying to distinguish the object of interest using additional features such as Hu Moments, since Hu Moment are Scale, position and rotation invariant for a given shape

As you can see, the set of masks (see first image) captured from different frames of the video slightly differ from each other.

So would the Hu Moments despite this slight change in shape differ?, and if they remain the same how can i use them as additional property to make the detection more accurate?

I've calculated the 7 Hu-Moments for different frames and the result was:

HuMoments
[[ 2.08520054e-01]
[ 1.69631372e-02]
[ 7.21551810e-05]
[ 6.90080950e-06]
[ 1.42410779e-12]
[-1.68289072e-07]
[-1.53980330e-10]]
HuMoments
[[ 2.06895709e-01]
[ 1.60840647e-02]
[ 3.30558368e-06]
[ 1.11390263e-07]
[-6.73641826e-14]
[ 1.41245011e-08]
[-5.54413536e-15]]
HuMoments
[[ 2.13862698e-01]
[ 1.92304207e-02]
[ 1.59852416e-06]
[ 3.25416763e-07]
[ 2.14353017e-13]
[ 1.50649139e-08]
[-9.55954816e-14]]
HuMoments
[[ 2.14002398e-01]
[ 1.92008910e-02]
[ 4.97181458e-06]
[ 4.78948215e-07]
[ 3.76015109e-13]
[-3.15822288e-08]
[-6.36277450e-13]]
HuMoments
[[ 2.06781829e-01]
[ 1.61130389e-02]
[ 8.62252599e-06]
[ 2.40464402e-07]
[ 2.97886471e-13]
[ 3.07837011e-09]
[-1.76507042e-13]]
HuMoments
[[ 2.05995932e-01]
[ 1.60756861e-02]
[ 7.67493642e-07]
[ 9.51817986e-08]
[ 2.14701063e-14]
[-3.25280940e-09]
[-1.41721342e-14]]
HuMoments
[[ 2.05805316e-01]
[ 1.54879330e-02]
[ 1.22114797e-04]
[ 1.25265644e-05]
[ 8.26008946e-11]
[-2.83389042e-07]
[-4.82914455e-10]]
HuMoments
[[ 2.09471113e-01]
[ 1.72160248e-02]
[ 2.20128873e-05]
[ 1.49897869e-06]
[-2.30866693e-12]
[-1.57913774e-07]
[-8.29529391e-12]]
HuMoments
[[2.15147844e-01]
[1.93443753e-02]
[8.08244156e-06]
[3.97563367e-06]
[2.20658357e-11]
[5.52855740e-07]
[4.58034173e-12]]
HuMoments
[[2.15429849e-01]
[1.93910849e-02]
[3.17392350e-05]
[1.45996626e-05]
[3.13970937e-10]
[1.99024297e-06]
[1.38642988e-11]]
HuMoments
[[ 2.11382574e-01]
[ 1.77909581e-02]
[ 2.79886054e-06]
[ 4.41359415e-06]
[ 1.53146745e-11]
[ 5.02198776e-07]
[-2.46891717e-12]]
HuMoments
[[2.12028845e-01]
[1.81061901e-02]
[1.15214622e-05]
[4.73220650e-06]
[3.35997756e-11]
[6.36751845e-07]
[9.59204224e-12]]
HuMoments
[[ 2.10406753e-01]
[ 1.72785541e-02]
[ 5.29421210e-06]
[ 9.58072012e-08]
[-2.65557753e-14]
[-1.25685175e-08]
[ 6.28538516e-14]]
HuMoments
[[2.17360183e-01]
[2.02630904e-02]
[6.14574197e-05]
[9.08990258e-06]
[2.13670968e-10]
[1.27309750e-06]
[2.24332748e-11]]
HuMoments
[[ 2.14532303e-01]
[ 1.87193203e-02]
[ 4.37330273e-05]
[ 5.07043463e-06]
[ 5.19617629e-11]
[-8.75846866e-08]
[-5.47804462e-11]]

Thanks in advance

For tests below are images of the object masks at different timestamp with and without irrelevant objects and one of the original images

Answer 1

There are many shape features you could experiment with.

Hu moments are certainly worth trying out for your case. As with most features, they change by a small amount when the shape changes by a small amount, and change more strongly when the shape changes more strongly. However, this statement only holds for the continuous domain. In the discrete domain, we have a set of pixels that approximate the object being imaged, and the shape of this set changes with (subpixel) translation, with rotation and with scaling. The smaller the objects, the larger the relative differences introduced by these changes.

Hu moments are computed based on the central moments of up to 3rd order. The higher the order, the more the moments will vary with small changes. Higher order moments are more sensitive to noise.

The first two Hu moments only use second order central moments, and are therefore less noise sensitive than the other ones, and more suitable for very small objects. Second order moments describe the best-fit ellipse to the shape. The first two Hu moments therefore describe the shape as an ellipse: elongation and similarity to the ellipse.

Other features you should add to this set are area, and maybe things like convexity or solidity, and Podczeck shapes (especially the similarity to a square seems relevant).

You will have to set some thresholds around your ideal values, and you should determine these thresholds from a collection of example shapes.