CODEWITHSUNDEEP
javaalgorithmimage-processingcomputer-visionvideo-processing
Samuel Willems
Samuel Willems

Reputation: 135

What would be a good algorithm to find image shifts (Java)

I have a camera from which i receive a ByteBuffer for each frame. I extract a 640px x 480px 11 bit grayscale image from the ByteBuffer and save it in to a short[640][480]. I do this because I won't be needing it as an image and I thought this would be faster (please do correct me if I am wrong).

Now this is done about 30 times per second. With each frame the program will save any values that differ by more than 20 and are smaller than the existing value for the current pixel in to the value of that pixel. It effectively creates a background image in my short[640][480].

Now for the issue, the camera might move thus shifting the background. And the background I get from a not moving camera already changes a great deal (also with great margins) each frame. Really it is only just stable enough to extract big fore-ground objects. So I need an algorithm that can tell me how much the camera and thus the image have shifted, so I know which areas are new in the image but mostly which areas are still usable.

The only way I could think of was to scan the image for each possible shift and see which matches best, because like i said, it might just not match all that well but still be the best match. Is there any better way to go about this? Because this way I would have to scan the entire image roughly 1,2 million times per frame...

Additionally I do not use processing or openCV or any such libraries.

Edit: I forgot to mention a pretty important detail, the image is a depth map so lighting does not affect it.

Edit: Here's some code, I use the Open Kinect library to retrieve a depth map from the Kinect. I am not yet sure how to parse the information, this is the only way I've gotten it to work so far:

public static short[][] background = new short[640][480];

public void onFrameReceived(FrameMode format, ByteBuffer frame, int timestamp) {

    for(int n=0; n<frame.limit()/2; n++) {

        int index = n*2;
        short Gray = (0xff - frame.get(index) & 0xff) | ((3-frame.get(index+1) & 0x3) * 255);

        short x = n%640;
        short y = n/640;

        if(background[x][y] > Gray + 10 || background[x][y] == 0) {
            background[x][y] = Gray;
        }
    }
}

I get 2 bytes per frame from which I attempt to extract an 11 bit value that represents how far the object is from my kinect. I have no idea how else to go about this, but it works like this so I'll save that question for later.

Additional Information: frame.limit() is the ammount of bytes in the bytebuffer. frame.get gets a single byte from my bytebuffer. For some reason the kinect sends me the bytes in a backwards order...

Upvotes: 0

Views: 483

Answers (2)

MatthieuBizien
MatthieuBizien

Reputation: 1725

You should use an image library, it will be easier, more robust and more efficient than you own implementations. For detecting background shift, I would calculate the gradient of your image and compare it to the previous one. It may be interesting to blur the image. You can compare using a quadratic function for the error between the former gradient and the current one.

Upvotes: 1

Mike
Mike

Reputation: 346

This is how I would determine if the camera moved. Of course, some padding and variance will want to be added to "detectChange()", but since I am not familiar with your data results, I could not determine that:

//pick 100 points at random
private static Point[] keys = new Point[100];

//initially set to the values of background at the key points
private static short[] keyValues = new short[100];


private bool detectChange()
{
    boolean changed = false;
    int amtchanged = 0;
    for(int i = 0; i < 100; i++)
    {
        //point some variance here for leeway
        if(background[keys[i].x][keys[i].y] != keyValues[i])
            amtchanged++;
    }

    if(amtchanged > 75)
        changed = true;

    return changed
}

public void onFrameReceived(FrameMode format, ByteBuffer frame, int timestamp) {

    if(detectChange())
    {
        //find where they went to determine
        //the camera's pan
    }

    //the rest of your code.

    for(int i = 0; i < 100; i++)
    {
        //update the keys to the new data
        keyValues[i] = background[keys[i].x][keys[i].y];
    }
}

Upvotes: 1

Related Questions