How can I find a template image in another image? Kinect and AForge preferred

Question

I copied the AForge-Sample from here: http://www.aforgenet.com/framework/features/template_matching.html And hoped, it would work with 2 Bitmaps as sources as in the following code:

    Bitmap findTemplate (Bitmap sourceImage, Bitmap template)
    {

    // create template matching algorithm's instance
    // (set similarity threshold to x.y%, 1.0f = 100%)
    ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching( 0.4f );
    // find all matchings with specified above similarity
    TemplateMatch[] matchings = tm.ProcessImage( sourceImage, template ); **// "Unsupported pixel format of the source or template image." as error message**
    // highlight found matchings
    BitmapData data = sourceImage.LockBits(
        new Rectangle( 0, 0, sourceImage.Width, sourceImage.Height ),
        ImageLockMode.ReadWrite, sourceImage.PixelFormat );
    foreach ( TemplateMatch m in matchings )
    {
        AForge.Imaging.Drawing.Rectangle( data, m.Rectangle, System.Drawing.Color.White );
        // do something else with matching
    }
    sourceImage.UnlockBits( data );
    return sourceImage;
    }

But when calling TemplateMatch[] matchings = tm.P.... it gives the error mentioned above. The template is generated this way:

Bitmap templatebitmap=(Bitmap)AForge.Imaging.Image.FromFile("template.jpg");

the source is generated with the kinect-webcam, where the PlanarImage is formatted as Bitmap (method copied from somewhere, but it was working up to now)

 Bitmap PImageToBitmap(PlanarImage PImage)
        {
            Bitmap bmap = new Bitmap(
              PImage.Width,
              PImage.Height,
              System.Drawing.Imaging.PixelFormat.Format32bppRgb);
            BitmapData bmapdata = bmap.LockBits(
              new Rectangle(0, 0, PImage.Width,
                                   PImage.Height),
              ImageLockMode.WriteOnly,
              bmap.PixelFormat);
            IntPtr ptr = bmapdata.Scan0;
            Marshal.Copy(PImage.Bits,
                         0,
                         ptr,
                         PImage.Width *
                            PImage.BytesPerPixel *
                                 PImage.Height);
            bmap.UnlockBits(bmapdata);
            return bmap;
        }

So, is anbody able to help me, where my mistake might be? Or maybe anyone knows a better way to match a template with a Kinect? The overall job is to detect a known object with the kinect, in my case a rubberduck.

Thank you in advamce.

Answer 1

So, I just implemented it myself. But it is so slow - so if anyone has an idea to improve, feel free to criticize my code:

    public class Position
    {
        public int bestRow { get; set; }
        public int bestCol { get; set; }
        public double bestSAD { get; set; }
        public Position(int row, int col, double sad)
        {
            bestRow = row;
            bestCol = col;
            bestSAD = sad;
        }
    }

    Position element_position = new Position(0, 0, double.PositiveInfinity);
    Position ownSearch(Bitmap search, Bitmap template) {
        Position position = new Position(0,0,double.PositiveInfinity);
        double minSAD = double.PositiveInfinity;


        // loop through the search image
        for (int x = 0; x <= search.PhysicalDimension.Width - template.PhysicalDimension.Width; x++)
        {
            for (int y = 0; y <= search.PhysicalDimension.Height - template.PhysicalDimension.Height; y++)
            {
                position_label2.Content = "Running: X=" + x + "  Y=" + y;
                double SAD = 0.0;

                // loop through the template image
                for (int i = 0; i < template.PhysicalDimension.Width; i++)
                {
                    for (int j = 0; j < template.PhysicalDimension.Height; j++)
                    {
                        int r = Math.Abs(search.GetPixel(x + i, y + j).R - template.GetPixel(i, j).R);

                        int g = Math.Abs(search.GetPixel(x + i, y + j).G - template.GetPixel(i, j).G);

                        int b = Math.Abs(search.GetPixel(x + i, y + j).B - template.GetPixel(i, j).B);

                        int a = template.GetPixel(i, j).A;

                        SAD = SAD + ((r + g + b)*a/255 );

                    }
                }
                // save the best found position 
                if (minSAD > SAD)
                {
                    minSAD = SAD;
                    // give me VALUE_MAX
                    position.bestRow = x;
                    position.bestCol = y;
                    position.bestSAD = SAD;
                }
            }
        }
        return position;
    }

Answer 2

here's the solution using AForge but it is slow take around 5 seconds but it works As usuall u need to introduce AForge framework download and install it. specify using AForge namespace

and copy paste to make it work

System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(@"C:\SavedBMPs\1.jpg");
System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(@"C:\SavedBMPs\2.jpg");
// create template matching algorithm's instance
// (set similarity threshold to 92.5%)

ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
// find all matchings with specified above similarity

TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings

BitmapData data = sourceImage.LockBits(
    new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
    ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{

        Drawing.Rectangle(data, m.Rectangle, Color.White);

    MessageBox.Show(m.Rectangle.Location.ToString());
    // do something else with matching
}
sourceImage.UnlockBits(data);