CODEWITHSUNDEEP
kinectkinect-sdkpoint-cloudskinect-v2
ravi
ravi

Reputation: 6338

Shift in Point Cloud acquired using Kinect v2 API

I am acquiring Point Cloud using Kinect v2 API in Windows 10 64 Bit OS. Below is the code snippet-

depthFrame = multiSourceFrame.DepthFrameReference.AcquireFrame();
colorFrame = multiSourceFrame.ColorFrameReference.AcquireFrame();

if (depthFrame == null || colorFrame == null) return;

depthFrame.CopyFrameDataToArray(depthData);
coordinateMapper.MapDepthFrameToCameraSpace(depthData, cameraSpacePoints);
coordinateMapper.MapDepthFrameToColorSpace(depthData, colorSpacePoints);
colorFrame.CopyConvertedFrameDataToArray(pixels, ColorImageFormat.Rgba);

for (var index = 0; index < depthData.Length; index++)
{
    int u = (int)Math.Floor(colorSpacePoints[index].X);
    int v = (int)Math.Floor(colorSpacePoints[index].Y);

    if (u < 0 || u >= COLOR_FRAME_WIDTH || v < 0 || v >= COLOR_FRAME_HEIGHT) continue;

    int pixelsBaseIndex = v * COLOR_FRAME_WIDTH + u) * COLOR_BYTES_PER_PIXEL;

    float x = cameraSpacePoints[index].X;
    float y = cameraSpacePoints[index].Y;
    float z = cameraSpacePoints[index].Z;

    byte red   = pixels[pixelsBaseIndex + 0];
    byte green = pixels[pixelsBaseIndex + 1];
    byte blue  = pixels[pixelsBaseIndex + 2];
    byte alpha = pixels[pixelsBaseIndex + 3];

    PointXYZRGB point = new PointXYZRGB(); // Color point in 3D
    point.postion(x, y, z);
    point.color(red, green, blue, apha);
}

Please see below a screenshot of the point cloud-

enter image description here

Please look around the orange-colored ball in above picture. Upon close inspection, it is visible that there exists a shift in the point cloud.

I am wondering, why such shift exists and how to remove/minimize it? Any workaround, please.

Upvotes: 0

Views: 169

Answers (1)

Atif Anwer
Atif Anwer

Reputation: 441

The amount of shift in color overlay and depth map can be due to a number of reasons.

  • Frame acquisition of depth and color frames are not at the same instant (as that is how the _reader_MultiSourceFrameArrived function in kinect SDK works. The timestamps for both cameras are slightly different, hence the slight shift. This is more prominent if you are moving the object in view.
  • The coordinateMapper function in the sdk for mapping the color frame and depth frame uses the camera calibration parameters. The default camera calibration parameters are had coded in the sdk, however there are slight differences in each and every device. You could try to recalibrate the Kinect cameras and use the updated calibration parameters to get the correct overlay of the color and depth maps. Note however, that by siply rplacing the camera calibration parameters in the Kinect Fusion code and recompiling does not work, as the parameters are replaced from the closed-source Kinect fusion dll.So you'll have to write your own code to update each frame at runtime.

Hope this helps.

Upvotes: 1

Related Questions