CODEWITHSUNDEEP
kinectspeech-recognitionspeech
user322416
user322416

Reputation: 113

I can't get kinect sdk to do speech recognition and track skeletal data at the sime time

I' ve a program in witch I enabled speech recognition with..

        RecognizerInfo ri = GetKinectRecognizer();

        speechRecognitionEngine = new SpeechRecognitionEngine(ri.Id);

        // Create a grammar from grammar definition XML file.
        using (var memoryStream = new MemoryStream(Encoding.ASCII.GetBytes(fileContent)))
        {
            var g = new Grammar(memoryStream);
            speechRecognitionEngine.LoadGrammar(g);
        }

        speechRecognitionEngine.SpeechRecognized += new EventHandler<SpeechRecognizedEventArgs>(speechEngine_SpeechRecognized);
        speechRecognitionEngine.SpeechRecognitionRejected += new EventHandler<SpeechRecognitionRejectedEventArgs>(speechEngine_SpeechRecognitionRejected);

speechRecognitionEngine.SetInputToAudioStream( sensor.AudioSource.Start(), new SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));

        speechRecognitionEngine.RecognizeAsync(RecognizeMode.Multiple);

..

all'is working fine and SpeechRecognized event get fired correctly..

The problem is, when I anable skeletal data tracking,

          sensor.SkeletonStream.Enable();
          sensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
          sensor.SkeletonFrameReady += sensor_SkeletonFrameReady;

speech recognition stops working ...

can I get your help?

Thank you so much!

Upvotes: 1

Views: 763

Answers (1)

Atul Verma
Atul Verma

Reputation: 2132

Audio is not processed if skeleton stream is enabled after starting audio capture Due to a bug, enabling or disabling the SkeletonStream will stop the AudioSource stream returned by the Kinect sensor. The following sequence of instructions will stop the audio stream: kinectSensor.Start(); kinectSensor.AudioSource.Start(); // --> this will create an audio stream kinectSensor.SkeletonStream.Enable(); // --> this will stop the audio stream as an undesired side effect

The workaround is to invert the order of the calls or to restart the AudioSource after changing SkeletonStream status.

        Workaround #1 (start audio after skeleton):
        kinectSensor.Start();
        kinectSensor.SkeletonStream.Enable();
        kinectSensor.AudioSource.Start();

        Workaround #2 (restart audio after skeleton):
        kinectSensor.Start();
        kinectSensor.AudioSource.Start(); // --> this will create an audio stream
        kinectSensor.SkeletonStream.Enable(); // --> this will stop the audio stream as an undesired side effect
        kinectSensor.AudioSource.Start(); // --> this will create another audio stream

Source - http://msdn.microsoft.com/en-us/library/jj663798.aspx

Upvotes: 1

Related Questions