How to use Google Speech to Text Feature with Microsoft Bot Framework Chatbot ?

Question

I want to use, Google Speech to Text feature with my Microsoft Bot Framework based Chatbot, Regular implementation of Microsoft BorFramework based bot is as :

const params = BotChat.queryParams(location.search);
    const user = {
        id: params['userid'] || 'userid',
        name: params['username'] || 'username'
    };
    const bot = {
        id: params['botid'] || 'botid',
        name: params['botname'] || 'botname'
    };
    window.botchatDebug = params['debug'] && params['debug'] === 'true';

    const speechOptions = {
        speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: '0000SPEECHKEY00000000' }),
        speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
            gender: CognitiveServices.SynthesisGender.Female,
            subscriptionKey: '0000SPEECHKEY00000000',
            voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
        })
    };
    BotChat.App({
        bot: bot,
        locale: params['locale'],
        resize: 'detect',
        // sendTyping: true,    // defaults to false. set to true to send 'typing' activities to bot (and other users) when user is typing
        speechOptions: speechOptions,
        user: user,
        directLine: {
            domain: params['domain'],
            secret: params['s'],
            token: params['t'],
            webSocket: params['webSocket'] && params['webSocket'] === 'true' // defaults to true
        }
    }, document.getElementById('BotChatGoesHere'));

Answer 1

You have several possibilities for speech recognition in the webchat:

• use Microsoft Cognitive Services
• use Browser speech capabilities
• use a custom speech recognition

Here you are in the 3rd option. You have to provide your own custom speech recognition that implements ISpeechRecognizer.

This interface is detailed here:

export interface ISpeechRecognizer {
    locale: string;
    isStreamingToService: boolean;
    referenceGrammarId: string; // unique identifier to send to the speech implementation to bias SR to this scenario

    onIntermediateResult: Func<string, void>;
    onFinalResult: Func<string, void>;
    onAudioStreamingToService: Action;
    onRecognitionFailed: Action;

    warmup(): void;
    setGrammars(grammars?: string[]): void;
    startRecognizing(): Promise<void>;
    stopRecognizing(): Promise<void>;
    speechIsAvailable(): boolean;
}

Source: https://github.com/Microsoft/BotFramework-WebChat/blob/master/src/SpeechModule.ts

You will also find in the link above the implementations of BrowserSpeechRecognizer which is the 2nd option