CODEWITHSUNDEEP
iosiphone3dlidartruedepth-camera
ubermensch
ubermensch

Reputation: 11

TrueDepth to real point cloud

I am trying to convert TrueDepth depth maps into point clouds. Is there any way to transform TrueDepth camera depth map into a real size point cloud using provided by Apple intrinsics? Other questions don't help. This is how I extract parameters:

        if let calibrationData = depthData.cameraCalibrationData {
            let intrinsicMatrix = calibrationData.intrinsicMatrix
            let extrinsicMatrix = calibrationData.extrinsicMatrix
            let referenceDimensions = calibrationData.intrinsicMatrixReferenceDimensions
            let intrinsicReferenceDimensionWidth = referenceDimensions.width
        }

where depthData: AVDepthData

My transformation code is here (with IPhone 12 intrinsics and reference dimensions divided by depth dimensions):

def depth_to_point_cloud(depth, intrinsics=[[2742.1404 / (4032 / 640), 0, 2015.808 / (4032 / 640)], [0, 2742.1404 / (4032 / 640), 1513.418 / (4032 / 640)], [0, 0, 1] ]):
    height, width = depth.shape
    x, y = np.meshgrid(np.arange(width), np.arange(height))
    xrw = (x - intrinsics[0][2]) * depth / intrinsics[0][0]
    yrw = (y - intrinsics[1][2]) * depth / intrinsics[1][1]
    xyzw = np.stack([xrw, yrw, depth], axis=2)
    return xyzw.reshape(-1, 3)

The result is very stretched and does not look like a point cloud with real distances. What am I missing? Any help, questions, suggestions, links, guides, literature would be appreciated!

Upvotes: 1

Views: 273

Answers (2)

Иван Романенко
Иван Романенко

Reputation: 11

this is the right way to get XYZ in object coordination system: this is the right way to get XYZ in object coordination system

the image I posted contains formulas for converting from an image coordinate system to an object coordinate system using trigonometric formulas. Each element is shown in the diagram for understanding

Upvotes: 1

John Seong
John Seong

Reputation: 105

It might be because it's supposed to be...

   float xrw = (pos.x - cameraIntrinsics[2][0]) * depth / cameraIntrinsics[0][0];
   float yrw = (pos.y - cameraIntrinsics[2][1]) * depth / cameraIntrinsics[1][1];

So [2] and [0] switched positions, as well as [2] and [1].

Upvotes: 0

Related Questions