CODEWITHSUNDEEP
c#gesture-recognitionhidden-markov-modelsaccord.net
user3144101
user3144101

Reputation: 43

HMM using Accord.Net for gesture recognition

I'm working on gesture recognition project. My data set consists of 4 different gestures, where each gesture set contains about 70 images. And I have extracted 4 features for each image. I'm trying to use Accord.Net for implementing HMM, and I understand that i'm going to need 4 HMMs, one for each gesture, but I'm not sure how can I construct the sequences of feature vectors for learning/training phase. Does anyone knows how to solve it?

This is the simple code for sequences:

double[][] sequences = new double[][] 
{
    new double[] { 0,1,2,3,4 }, // This is the first  sequence with label = 0 
    new double[] { 4,3,2,1,0 }, // This is the second sequence with label = 1
};

// Labels for the sequences 
int[] labels = { 0, 1 };

Upvotes: 2

Views: 1914

Answers (1)

Cesar
Cesar

Reputation: 2118

You are right that you will need one HMM for each gesture. However, the framework can already give you this construction if you use the HiddenMarkovClassifier class (it is a wrapper around multiple HMMs created after each class you are trying to detect).

If you have 4 features for each image, you will need to assume a probability distribution that will be able to model your multivariate features. One simple option would be to assume your features are independent from each other, and that each of them follows a Normal distribution.

As such, you can use the following sample code to create your models. It assumes your database has only two training sequences, but on reality you must have many more.

double[][][] sequences = new double[][][]
{
    new double[][] // This is the first  sequence with label = 0
    { 
        new double[] { 0, 1, 2, 1 }, // <-- this is the 4-features feature vector for
        new double[] { 1, 2, 5, 2 }, //       the first image of the first sequence
        new double[] { 2, 3, 2, 5 },
        new double[] { 3, 4, 1, 1 },
        new double[] { 4, 5, 2, 2 },
    }, 

    new double[][] // This is the second sequence with label = 1
    {
        new double[] { 4,  3, 4, 1 }, // <-- this is the 4-features feature vector for
        new double[] { 3,  2, 2, 2 }, //       the first image of the second sequence
        new double[] { 2,  1, 1, 1 },
        new double[] { 1,  0, 2, 2 },
        new double[] { 0, -1, 1, 2 },
    }
};

// Labels for the sequences
int[] labels = { 0, 1 };

The above code shows how the learning database can be set up. Now, once it has been set, you can create a hidden Markov classifier for 4 normal features (assuming independence between Normal distributions) as

// Create one base Normal distribution to be replicated accross the states
var initialDensity = new MultivariateNormalDistribution(4); // we have 4 features

// Creates a sequence classifier containing 2 hidden Markov Models with 2 states
// and an underlying multivariate mixture of Normal distributions as density.
var classifier = new HiddenMarkovClassifier<MultivariateNormalDistribution>(
    classes: 2, topology: new Forward(2), initial: initialDensity);

// Configure the learning algorithms to train the sequence classifier
var teacher = new HiddenMarkovClassifierLearning<MultivariateNormalDistribution>(
    classifier,

    // Train each model until the log-likelihood changes less than 0.0001
    modelIndex => new BaumWelchLearning<MultivariateNormalDistribution>(
        classifier.Models[modelIndex])
    {
        Tolerance = 0.0001,
        Iterations = 0,

        FittingOptions = new NormalOptions()
        {
            Diagonal = true,      // only diagonal covariance matrices
            Regularization = 1e-5 // avoid non-positive definite errors
        }

        // PS: Setting diagonal = true means the features will be
        // assumed independent of each other. This can also be
        // achieved by using an Independent<NormalDistribution>
        // instead of a diagonal multivariate Normal distribution
    }
);

Finally, we can train the model and test its outputs on the learned data:

// Train the sequence classifier using the algorithm
double logLikelihood = teacher.Run(sequences, labels);


// Calculate the probability that the given
//  sequences originated from the model
double likelihood, likelihood2;

// Try to classify the 1st sequence (output should be 0)
int c1 = classifier.Compute(sequences[0], out likelihood);

// Try to classify the 2nd sequence (output should be 1)
int c2 = classifier.Compute(sequences[1], out likelihood2);

Upvotes: 3

Related Questions