Gesture Recognition Algorithm - Kinect

Question

I'm developing an application for the Kinect for my final year university project, and I have a requirement to develop a number of gesture recognition algorithms. I'd appreciate some advice on this.

My initial algorithm is detecting the users hand moving closer towards the kinect, within a certain time frame. For now i'll say this is an arbitrary 500ms.

My idea is as follows:

• Record z-axis position every 100ms and store in List.
• Each time a new position is recorded, check the z-position for each of the previous 4 positions in the List.
• If the z position has varied by the required distance between any of those individually or collectively, fire off a gesture recognised event.
• If gesture recognised, clear List, and start again.

This is the first time that I have tried anything like this, and would like some advise on my initial naive implementation.

Thanks.

Answer 1

Are you going to use the official Kinect SDK or opensource drivers(libfreenect or OpenNI) ?

If you're using the Kinect SDK you can start by having a look at something like:

• Kinect SDK Dynamic Time Warping (DTW) Gesture Recognition

• Candescent NUI

(Candescent NUI focuses more on finger detection though)

If you're planning to use opensource drivers, try OpenNI and NITE. NITE comes with hand tracking and gestures(swipe, circle control, 2d sliders, etc.).

The idea is to at least have hand detection and carry on from there. If you've got that, you could implement something like an adaptation of the Unistroke Gesture Recognizer or look into other techniques like Motion Templates/MotionHistory, etc....adapting them to the new data you can play with now.

Goodluck!

Answer 2

If you're just trying to recognise the user swinging her hand towards you, your approach should work (despite being very susceptible to misfiring due to noisy data). What you're trying to do falls very nicely in the field of pattern recognition. For this, and very similar tasks, people very often use hidden Markov models with great success. You might want to check the Wikipedia article. I'm not a C# person, but as far as I know, Microsoft has very nice statistical inference libraries for C#, and they will definitely include HMM implementations.