CODEWITHSUNDEEP
c#kinect-sdkkinect-v2
ravi
ravi

Reputation: 6328

Get RGB values of each 3D point in Kinect v2

I am getting confused with the Kinect v2 CoordinateMapper API. I want to get RGB values of each 3D point aka CameraSpacePoint in Kinect v2.

Please see below the code snippet:

var depthFrame = multiSourceFrame.DepthFrameReference.AcquireFrame();
var colorFrame = multiSourceFrame.ColorFrameReference.AcquireFrame();

var depthWidth = depthFrame.FrameDescription.Width;
var depthHeight = depthFrame.FrameDescription.Height;

ushort[] depthData = new ushort[depthWidth * depthHeight];
CameraSpacePoint[] camerapoints = new CameraSpacePoint[depthData.Length];
ColorSpacePoint[] colorpoints = new ColorSpacePoint[depthData.Length];

depthFrame.CopyFrameDataToArray(depthData);
this.coordinateMapper.MapDepthFrameToCameraSpace(depthData, camerapoints);
this.coordinateMapper.MapDepthFrameToColorSpace(depthData, colorpoints);

The 3D points are stored in camerapoints variable. I want to the RGB values of each camerapoints. In other words, please see below pesudo code:

RGBPoint[] rgbpoints = new RGBPoint[depthData.Length];
RGBPoint rgbpoint = rgbpoints[0];
int red   = rgbpoint.r
int green = rgbpoint.g
int blue  = rgbpoint.b

As always, thank you very much. I really appreciate your kind response.

Upvotes: 0

Views: 257

Answers (1)

Tuwuh S
Tuwuh S

Reputation: 301

The colorpoints array contains the (x, y) index that corresponds to each pixels in the depthData array. From here, we can just get the color frame data (from colorFrame) at each specific (x, y) index indicated by colorpoints[index].

var colorWidth = colorFrame.FrameDescription.Width;
var colorHeight = colorFrame.FrameDescription.Height;

// Assuming BGRA format here
byte[] pixels = new byte[colorWidth * colorHeight * 4];
colorFrame.CopyFrameDataToArray(pixels);

byte[] bgraPoints = new byte[depthWidth * depthHeight * 4];

for (var index = 0; index < depthData.Length; index++)
{
    var u = colorpoints[index].X;
    var v = colorpoints[index].Y;
    if (u < 0 || u >= colorWidth || v < 0 || v >= colorHeight) continue;
    var pixelsBaseIndex = v * colorWidth + u;
    Array.Copy(pixels, 4 * pixelsBaseIndex, bgraPoints, 4 * index, 4);
}

var index = 0;
var red = bgraPoints[4 * index + 2];
var green = bgraPoints[4 * index + 1];
var blue = bgraPoints[4 * index];

Upvotes: 1

Related Questions