CODEWITHSUNDEEP
unity-game-engineibm-watsonwatson
199user911
199user911

Reputation: 425

Watson keyword spotting unity

I have downloaded the Watson unity SDK and set it up like show in the picture and it works. My question is how do I add keyword spotting? I have read this question For Watson's Speech-To-Text Unity SDK, how can you specify keywords? But I cant for example locate the SendStart function.

enter image description here

Upvotes: 1

Views: 326

Answers (1)

taj
taj

Reputation: 1138

The Speech to Text service does not find keywords. To find keywords you would need to take the final text output and send it to the Alchemy Language service. Natural Language Understanding service is still being abstracted into the Watson Unity SDK but will eventually replace Alchemy Language.

private AlchemyAPI m_AlchemyAPI = new AlchemyAPI();

private void FindKeywords(string speechToTextFinalResponse)
{
    if (!m_AlchemyAPI.ExtractKeywords(OnExtractKeywords, speechToTextFinalResponse))
        Log.Debug("ExampleAlchemyLanguage", "Failed to get keywords.");
}

void OnExtractKeywords(KeywordData keywordData, string data)
{
    Log.Debug("ExampleAlchemyLanguage", "GetKeywordsResult: {0}", JsonUtility.ToJson(resp));
}

EDIT 1

Natural Language Understanding has been abstracted in tot he Watson Unity SDK.

NaturalLanguageUnderstanding m_NaturalLanguageUnderstanding = new NaturalLanguageUnderstanding();
private static fsSerializer sm_Serializer = new fsSerializer();

private void FindKeywords(string speechToTextFinalResponse)
{
    Parameters parameters = new Parameters()
    {
    text = speechToTextFinalResponse,
    return_analyzed_text = true,
    language = "en",
    features = new Features()
    {
        entities = new EntitiesOptions()
        {
            limit = 50,
            sentiment = true,
            emotion = true,
        },
        keywords = new KeywordsOptions()
        {
            limit = 50,
            sentiment = true,
            emotion = true
        }
    }

    if (!m_NaturalLanguageUnderstanding.Analyze(OnAnalyze, parameters))
        Log.Debug("ExampleNaturalLanguageUnderstanding", "Failed to analyze.");
}

private void OnAnalyze(AnalysisResults resp, string customData)
{
    fsData data = null;
    sm_Serializer.TrySerialize(resp, out data).AssertSuccess();
    Log.Debug("ExampleNaturalLanguageUnderstanding", "AnalysisResults: {0}", data.ToString());
}

EDIT 2 Sorry, I didn't realize Speech To Text had the ability to do keyword spotting. Thanks to Nathan for pointing that out to me! I added this functionality into a future release of Speech to Text in the Unity SDK. It will look like this for the Watson Unity SDK 1.0.0:

void Start()
{
    //  Create credential and instantiate service
    Credentials credentials = new Credentials(_username, _password, _url);
    _speechToText = new SpeechToText(credentials);

    //  Add keywords
    List<string> keywords = new List<string>();
    keywords.Add("speech");
    _speechToText.KeywordsThreshold = 0.5f;
    _speechToText.Keywords = keywords.ToArray();
    _speechToText.Recognize(_audioClip, HandleOnRecognize);
}


private void HandleOnRecognize(SpeechRecognitionEvent result)
{
    if (result != null && result.results.Length > 0)
    {
        foreach (var res in result.results)
        {
            foreach (var alt in res.alternatives)
            {
                string text = alt.transcript;
                Log.Debug("ExampleSpeechToText", string.Format("{0} ({1}, {2:0.00})\n", text, res.final ? "Final" : "Interim", alt.confidence));

                if (res.final)
                    _recognizeTested = true;
            }

            if (res.keywords_result != null && res.keywords_result.keyword != null)
            {
                foreach (var keyword in res.keywords_result.keyword)
                {
                    Log.Debug("ExampleSpeechToText", "keyword: {0}, confidence: {1}, start time: {2}, end time: {3}", keyword.normalized_text, keyword.confidence, keyword.start_time, keyword.end_time);
                }
            }
        }
    }
}

Currently you can find the refactor branch here. This release is a breaking change and has all of the higher level (widgets, config, etc) functionality removed.

Upvotes: 2

Related Questions