MS Kinect FaceTracker creating IFTResult

Question

I have a fairly simple application that contains the following:

context->mFaceTracker = FTCreateFaceTracker();
hr = context->mFaceTracker->Initialize( &mVideoCameraConfig, &mDepthCameraConfig, NULL, NULL );

which works fine and returns S_OK and mFaceTracker is (as far as I can tell) initialized properly. However, the next line is:

hr = context->mFaceTracker->CreateFTResult( &context->mFTResult );

which always returns FT_ERROR_UNINITIALIZED, doesn't allocate the pointer, and has me puzzled. I've tried many different strategies for getting this to work from changing how the threading for the device and detector works, to changing my FTcontext object from a class to a struct to match the samples, all with no success. The Kinect SDK samples all work fine, but trying to use them in my own application doesn't seem to, despite my closely mirroring how they initialize the device and the Face Tracker. I'm curious if anyone else has run into this or similar problems around initializing either IFTFaceTracker or the IFTResult. Also, I'm curious how else I can test the IFTFaceTracker for correct intialization, other than testing the HRESULT that Initialize() returns. Thanks in advance,

=== edit

I've had a few request for more code. It's built on Cinder and is using this block for Cinder: https://github.com/BanTheRewind/Cinder-KinectSdk

I can't post all of the code, but I've posted at least most of the relevant Kinect initialization code here:

void Kinect::start( const DeviceOptions &deviceOptions ) 
    {
        if ( !mCapture ) {

            // Copy device options
            mDeviceOptions  = deviceOptions;
            string deviceId = mDeviceOptions.getDeviceId();
            int32_t index   = mDeviceOptions.getDeviceIndex();

            // Clamp device index
            if ( index >= 0 ) {
                index = math<int32_t>::clamp( index, 0, math<int32_t>::max( getDeviceCount() - 1, 0 ) );
            }

            // Initialize device instance
            long hr = S_OK;
            if ( index >= 0 ) {
                hr = NuiCreateSensorByIndex( index, &mSensor );
                if ( FAILED( hr ) ) {
                    trace( "Unable to create device instance " + toString( index ) + ": " );
                    error( hr );
                    return;
                }
            } else if ( deviceId.length() > 0 ) {
                _bstr_t id = deviceId.c_str();
                hr = NuiCreateSensorById( id, &mSensor );
                if ( FAILED( hr ) ) {
                    trace( "Unable to create device instance " + deviceId + ":" );
                    error( hr );
                    return;
                }
            } else {
                trace( "Invalid device name or index." );
                return;
            }

            // Check device
            hr = mSensor != 0 ? mSensor->NuiStatus() : E_NUI_NOTCONNECTED;
            if ( hr == E_NUI_NOTCONNECTED ) {
                error( hr );
                return;
            }

            // Get device name and index
            if ( mSensor != 0 ) {
                mDeviceOptions.setDeviceIndex( mSensor->NuiInstanceIndex() );
                BSTR id = ::SysAllocString( mSensor->NuiDeviceConnectionId() ); 
                _bstr_t idStr( id );
                if ( idStr.length() > 0 ) {
                    std::string str( idStr );
                    mDeviceOptions.setDeviceId( str );
                }
                ::SysFreeString( id );
            } else {
                index = -1;
                deviceId = "";
            }
            flags |= NUI_INITIALIZE_FLAG_USES_COLOR;
            }
            hr = mSensor->NuiInitialize( flags );
            if ( FAILED( hr ) ) {
                trace( "Unable to initialize device " + mDeviceOptions.getDeviceId() + ":" );
                error( hr );
                return;
            }
                hr = mSensor->NuiSkeletonTrackingEnable( 0, flags );
                if ( FAILED( hr ) ) {
                    trace( "Unable to initialize skeleton tracking for device " + mDeviceOptions.getDeviceId() + ": " );
                    error( hr );
                    return;
                }
            mIsSkeletonDevice = true;

            mThread = CreateThread(NULL, 0, &Kinect::StaticThread, (PVOID)this, 0, 0);
        }
    }

    DWORD WINAPI Kinect::StaticThread(PVOID lpParam)
    {
        Kinect* device = static_cast<Kinect*>(lpParam);
        if (device)
        {
            return device->run();
        }
        return 0;
    }

void run() {

if(mSensor) {

if(mEnabledFaceTracking)
                {

                    if(mNeedFaceTracker) {
                        mFaceTracker = new FaceTracker( 
                        mDeviceOptions.getVideoSize().x,
                        mDeviceOptions.getVideoSize().y,
                        mDeviceOptions.getDepthSize().x,
                        mDeviceOptions.getDepthSize().y,
                        1.0,
                        1 );

                        mNeedFaceTracker = false;
                    }


                    // make sure we have both color && depth buffers to work with
                    if(newDepth || newVideo)
                    {
                        FT_SENSOR_DATA sensorData(mFTColorImage, mFTDepthImage);
                        FT_VECTOR3D hint[2]; // this is initialized elsewhere

                        mFaceTracker->checkFaces( (NUI_SKELETON_FRAME*) &skeletonFrame, mFTColorImage, mFTDepthImage, 1.0, 0);

                        if(mFaceTracker->getNumFaces() > 0) {

                            cout << " we have a face " << mFaceTracker->getNumFaces() << endl;

                            mNewFaceTrackData = true;
                            mFaceData.clear();
                            for( int i = 0; i < mFaceTracker->getNumFaces(); i++) {

                                Face newFace;
                                mFaceTracker->getProjectedShape(0, newFace.scale, newFace.rotation, newFace.transform, newFace.screenPositions);
                                mFaceData.push_back(newFace);
                            }
                        }
                    }
                }
Sleep( 8 );
}

}

Answer 1

It looks like you never call (or omitted including in the code sample) NuiImageStreamOpen(), such as this code snippet from the SingleFace sample, KinectSensor.cpp in the Init method:

hr = NuiImageStreamOpen(
    colorType,
    colorRes,
    0,
    2,
    m_hNextVideoFrameEvent,
    &m_pVideoStreamHandle );
if (FAILED(hr))
{
    return hr;
}

hr = NuiImageStreamOpen(
    depthType,
    depthRes,
    (bNearMode)? NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE : 0,
    2,
    m_hNextDepthFrameEvent,
    &m_pDepthStreamHandle );

Calling those before you call CreateFTResult() may fix the uninitialized error.

Additionally, you call CreateThread() and then call run(), but there is no while loop so that thread will exit almost immediately, certainly without enough time for the Kinect to start providing data to the FaceTracking.

It doesn't look like you have included the Thread or event loop that is checking the sensor for new data, updating mFTColorImage and mFTDepthImage, and setting the newDepth and newVideo flags. This could be in the same thread that you create above (provided you create a while loop, and ignoring performance or other classes needing the Kinect data), or could be a different thread as in the SingleFace Kinect SDK sample.