CODEWITHSUNDEEP
c#unity-game-engineinputspeechmrtk
Rossi
Rossi

Reputation: 11

Recognize the custom speech keywords (Unity, MRTK, Hololens 1)

I'm trying to get the hololens (1) to recognize a custom defined speech keyword. Using Unity and MRTK. It recognizes the pre-defined keywords (ex: "Select") but my custom keyword (let's call it "Keyword") isn't recognized.

I have both a tooltip pop up to give feedback if it recognized a keyword and a audio feedback I put in there for debugging purposes (aka: if ANY keyword is recognized, play a chime). It works for "Select" but not for "Keyword."

I want this keyword to be globally recognized.

  1. Microphone capability is selected.
  2. Keyword is defined in the profiles section. (Input>Speech>Add a New Speech Command)
  3. Input Action Handler, Speech Input Handler, and a custom code "SpeechManager.cs" ALL do the same thing. "Select" is recognized (pop up and chime played) but "Keyword" is not.
  4. "Focus Required" is not selected.

  5. I have tried several different keywords to make sure I didn't pick a difficult to recognize one.

    using UnityEngine;
using Microsoft.MixedReality.Toolkit.Input;
using Microsoft.MixedReality.Toolkit;
public class SpeechManager : MonoBehaviour, IMixedRealitySpeechHandler
{

void Start()
{
    //Debugging "play sound" declarations deleted as not important for this example script.
    CoreServices.InputSystem?.RegisterHandler<IMixedRealitySpeechHandler>(this);
}

void IMixedRealitySpeechHandler.OnSpeechKeywordRecognized(SpeechEventData eventData)
{
    PlaySound(1);  //DEBUG AUDIO FEEDBACK

    if (eventData.Command.Keyword == "keyword")
    {
        PlaySound(2);  //DEBUG AUDIO FEEDBACK
    }
    else if (eventData.Command.Keyword == "select")
    {
        PlaySound(3);  //DEBUG AUDIO FEEDBACK
    }
}
}

What SHOULD happen:

"Select" spoken: PlaySound(1) and PlaySound(3) triggered.

"Keyword" spoken: PlaySound(1) and PlaySound(2) triggered.

What ACTUALLY happens:

"Select": PlaySound(1) triggered*

"Keyword" nothing happens <- Why? WHY???

'* Yes, PlaySound(3) is not triggered, but that is a whole separate issue that I'm not too concerned about because both Speech Input Handler and Input Action Handler both recognize "Select." One issue at a time!

Upvotes: 0

Views: 2051

Answers (3)

Rob Chambers
Rob Chambers

Reputation: 111

You could also try the newer speech platform. It supports unity, and it's available here: http://aka.ms/speech/sdk. Some find it easier to use.

You can checkout a Unity quick start here

Upvotes: 0

Rossi
Rossi

Reputation: 11

Perazim, thank you for your input. Mine looks exactly like yours. So why wasn't it working? I figured it out. Another user on another problem solved their problem by deleting the App folder (or whatever you named your build folder). So i thought, let's give it a try. It worked!!! Tip to save you massive amount of time: if you think it should work and it doesn't, delete App!!!

Upvotes: 1

Perazim
Perazim

Reputation: 1549

I can share how I set up my scene and my words are getting recognized. I attach the SpeechInputHandler e.g. on my toolbar and reference my keywords to the toolbar buttons. Is Focus Required is also set to false.

1. Define your keyword under MRT>Input>Speech:
enter image description here
2. Setup the Speech Input Handler:
enter image description here

Upvotes: 0

Related Questions