ARKit 4, RealityKit - Converting local transform to world transform

In ARKit 4 with RealityKit, one could find, for example, left hand transform relatively to skeleton base (hip). With lefthand transform (relative to hip) and hip transform (relative to world), how to calculate the lefthand transform relative to world? Where is the API? It seems that when I figured out the math formula, I could use SIMD api. But I guess there should be a simple API do this kind of math? Thanks.

EDIT: Adding some code to make it clear ..

guard let bodyAnchor = anchor as? ARBodyAnchor else { continue }
let skeleton = bodyArchor.skeleton
let leftHandModelTransform = skeleton.modelTransform(for: .leftHand)

// at this point. I have access to:
// hip transform (bodyArchor.tranform) which is in the world coordinate system;
// lefthand transform which is relative to hip of the skeleton model. 
// what to do next? any api? or do I have to figureout math and then use simd apis ? 
//

EDIT: also, could someone recommend a book / web page describing how to use SIMD in different 3D computation? I found this post(Augmented Reality 911 — Transform Matrix 4x4) really useful. But it did not touch upon conversion between two vector spaces.

Upvotes: 3

Views: 2100

Answers (2)

This code is an answer to Jan from comments. Hope it help. bodyAnchor is the root anchor for the skeleton tracking.

    let skeleton = bodyAnchor.skeleton
    let handTransform = skeleton.modelTransform(for: (ARSkeleton.JointName(rawValue: "left_handIndexStart_joint"))

    let hipEntity = Entity()
    hipEntity.transform = Transform(matrix: bodyAnchor.transform)
    let worldEntity = Entity()
    let handGlobalTransform = worldEntity.convert(transform: Transform(matrix:handTransform!), from: hipEntity)

I have to admit that this is not an ideal solution as it created two entities just for math calculation. I believe (could not wrong), Apple created two different mechanism for location manipulation - one is Entity have location and then another is anchor. So if you are directly manipulating entity location, math is given there. However, I was trying to manipulate anchor location (and assign entity to it). That gave some trouble.

Upvotes: 1

Ok. After reviewing all the math about matrix, 3D transformation, and quaternion, I finally found a solution! It is actually buried in RealityKit API:

func convert(normal: SIMD3<Float>, from referenceEntity: Entity?) -> SIMD3<Float>

I had to create two entities to use this API. But at least it works ...

Upvotes: 3

Related Questions