c4b4d4
c4b4d4

Reputation: 1074

Controlling a virtual character's joints rotation with OpenNI + Kinect

I'm starting a project where I need to control a virtual character. The character is being rendered in multiple 3D engines, such as Three.JS and iOS SceneKit.

I'm getting the Quaternions of every joint of the skeleton with OpenNI, and it looks kind of like this:

The code that saves and pass the quaternion, looks like this:

float confidence = context.getJointOrientationSkeleton(userId, jointName, 
joint);
joints[jointIndex]=joint.m00;
joints[jointIndex+1]=joint.m01;
joints[jointIndex+2]=joint.m02;
joints[jointIndex+3]=joint.m10;
joints[jointIndex+4]=joint.m11;
joints[jointIndex+5]=joint.m12;
joints[jointIndex+6]=joint.m20;
joints[jointIndex+7]=joint.m21;
joints[jointIndex+8]=joint.m22;
jointIndex+=9;

This repeats for every joint of the skeleton.

The last row and last column is always [0 0 0 1] [0, 0, 0, 1], so I just append that on the client once I receive it and build a 4x4 matrix.

I want to be able to make the right rotations with this data, but the rotations I'm getting are definitely wrong.

This is how I'm processing the matrix: (pseudo-code)

row1 = [m00 m01 m02 0]
row2 = [m10 m11 m12 0]
row3 = [m20 m21 m22 0]
row4 = [0 0 0 1]

matrix4by4 = matrix4x4(rows:[row1,row2,row3,row4])

And then I got the quaternion with two methods, and both methods were showing bad rotations, I cannot find what's wrong or what I'm missing.

First method

There's an iOS function that gets a 3x3 or 4x4 matrix, and transforms it into quaternion:

boneRotation = simd_quatf.init(matrix4by4).vector //X,Y,Z,W

Second method

I found the following code on the web:

let tr = m00 + m11 + m22

var qw = 0
var qx = 0
var qy = 0
var qz = 0
if (tr > 0) {
    var S = sqrt(tr+1.0) * 2 // S=4*qw
    qw = 0.25 * S
    qx = (m21 - m12) / S
    qy = (m02 - m20) / S
    qz = (m10 - m01) / S
} else if ((m00 > m11) && (m00 > m22)) {
    var S = sqrt(1.0 + m00 - m11 - m22) * 2 // S=4*qx
    qw = (m21 - m12) / S
    qx = 0.25 * S
    qy = (m01 + m10) / S
    qz = (m02 + m20) / S
} else if (m11 > m22) {
    var S = sqrt(1.0 + m11 - m00 - m22) * 2 // S=4*qy
    qw = (m02 - m20) / S
    qx = (m01 + m10) / S
    qy = 0.25 * S
    qz = (m12 + m21) / S
} else {
    let S = sqrt(1.0 + m22 - m00 - m11) * 2 // S=4*qz
    var = (m10 - m01) / S
    qx = (m02 + m20) / S
    qy = (m12 + m21) / S
    qz = 0.25 * S
}

boneRotation = vector4(qx, qy, qw, qz)

I started testing only with shoulder and elbow rotation to help me visualize what could be wrong or missing, and made a video.

Here's how it's behaving: https://www.youtube.com/watch?v=xUtNiwH_AGk

What could I be missing? For example, the axis of rotation of the shoulder are like this:

X-Axis X-Axis

Y-Axis Y-Axis

Z-Axis Z-Axis

Thank you in advance :-)

YouTuve Video: https://www.youtube.com/watch?v=xUtNiwH_AGk YouTube Video

Upvotes: 1

Views: 190

Answers (0)

Related Questions