CODEWITHSUNDEEP
c#speech-recognitionazure-speechazure-ai
David_Strachan
David_Strachan

Reputation: 35

Azure AI Speech Service - No punctuation on Recognized return

I am using the Azure AI speech recognition, which I think is configured correctly to punctuate en-GB. Currently, the recognized return is unpunctuated and comes out as straight text

 _speechConfig = SpeechConfig.FromSubscription(Token, "uksouth");
 _speechConfig.OutputFormat = OutputFormat.Detailed;
 _speechConfig.SpeechRecognitionLanguage = "en-GB";
 _speechConfig.EnableDictation();
 _speechStream = AudioInputStream.CreatePushStream(AudioStreamFormat.GetWaveFormatPCM(16000, 16, 1));
 _audioConfig = AudioConfig.FromStreamInput(_speechStream);

 _speechRecognizer = new SpeechRecognizer(_speechConfig, _audioConfig);

 _speechRecognizer.Recognized += OnRecognized;
 _speechRecognizer.Recognizing += RecognizingAudio;
 await _speechRecognizer.StartContinuousRecognitionAsync();

The above configuration returns:

2024-03-22 18:14:34,962 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:34,964 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"04a63ad5a8a14de7aba4c3256dddb620","Text":"today i went","Offset":11500000,"Duration":10800000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:35,257 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:35,257 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"c53ea1614bff479f96df4847c306ea49","Text":"today i went to","Offset":11500000,"Duration":14000000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:35,555 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:35,556 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"f0b1bdbfea924ba2b100364d59522ce9","Text":"today i went to the","Offset":11500000,"Duration":16400000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:35,851 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:35,851 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"6081cf3af7cd4b8aa113f5b64d2b8d62","Text":"today i went to the shop","Offset":11500000,"Duration":16000000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:36,261 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:36,262 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"2923456f66544624b4adced94f6e4ca2","Text":"today i went to the shop i","Offset":11500000,"Duration":19600000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:36,461 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:36,461 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"787311418c484d57810cc19e3e1a6bbf","Text":"today i went to the shop i bought","Offset":11500000,"Duration":26000000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:36,858 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:36,858 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"da59bcd910314a0e9232bead20475b29","Text":"today i went to the shop i bought apples","Offset":11500000,"Duration":28400000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:37,206 [INFO] Namspace.Client.Windows.Transcription.TranscriptionTimer - Time elapsed: 00:00:05.0181346
2024-03-22 18:14:37,758 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:37,758 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"d27d1a08570d4454a4a010f6f7919352","Text":"today i went to the shop i bought apples pears","Offset":11500000,"Duration":35600000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:38,156 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:38,157 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"6d78a51f69cf4d90b8772f6b836a9264","Text":"today i went to the shop i bought apples pears ban","Offset":11500000,"Duration":42000000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:38,457 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:38,457 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"f97850bf361c402b84a37e64dc70133a","Text":"today i went to the shop i bought apples pears bananas","Offset":11500000,"Duration":44800000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:38,758 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:38,758 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"e34e6710aabe4e5d9178ae5e3cbedc6f","Text":"today i went to the shop i bought apples pears bananas and","Offset":11500000,"Duration":48000000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:39,059 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:39,060 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"c458e883f2a0465087fafe0555bae6b7","Text":"today i went to the shop i bought apples pears bananas and oranges","Offset":11500000,"Duration":53200000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:40,359 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:40,360 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"9c521b18be3f4dc3b12079be51307478","Text":"today i went to the shop i bought apples pears bananas and oranges do you","Offset":11500000,"Duration":62000000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:40,764 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:40,765 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"b6ff1bed23c5457caeca5da1108df74b","Text":"today i went to the shop i bought apples pears bananas and oranges do you think","Offset":11500000,"Duration":65600000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:40,960 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:40,960 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"2010d0d4c8224c4a84510354b81c4aee","Text":"today i went to the shop i bought apples pears bananas and oranges do you think i missed","Offset":11500000,"Duration":71200000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:41,362 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizingSpeech
2024-03-22 18:14:41,362 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"1cefaace3a2e4448993f4a5838349992","Text":"today i went to the shop i bought apples pears bananas and oranges do you think i missed anything","Offset":11500000,"Duration":75200000,"PrimaryLanguage":{"Language":"en-GB"},"Channel":0}
2024-03-22 18:14:41,941 [INFO] Namspace.Clients.AzureTranscriptionClient - RecognizedSpeech
2024-03-22 18:14:41,941 [INFO] Namspace.Clients.AzureTranscriptionClient - {"Id":"49510f5baedf4aaca02a3620bbc7cb38","RecognitionStatus":"Success","Offset":11500000,"Duration":75200000,"Channel":0,"DisplayText":"Today I went to the shop I bought apples pears bananas and oranges do you think I missed anything","NBest":[{"Confidence":0.8908741,"Lexical":"today i went to the shop i bought apples pears bananas and oranges do you think i missed anything","ITN":"today i went to the shop i bought apples pears bananas and oranges do you think i missed anything","MaskedITN":"today i went to the shop i bought apples pears bananas and oranges do you think i missed anything","Display":"Today I went to the shop I bought apples pears bananas and oranges do you think I missed anything","Words":[{"Word":"today","Offset":11500000,"Duration":5200000,"Confidence":0.797704},{"Word":"i","Offset":16700000,"Duration":400000,"Confidence":0.9929609},{"Word":"went","Offset":17100000,"Duration":3200000,"Confidence":0.9909313},{"Word":"to","Offset":20300000,"Duration":800000,"Confidence":0.85922694},{"Word":"the","Offset":21100000,"Duration":1200000,"Confidence":0.9740201},{"Word":"shop","Offset":22300000,"Duration":5200000,"Confidence":0.99293035},{"Word":"i","Offset":29100000,"Duration":400000,"Confidence":0.7901372},{"Word":"bought","Offset":29500000,"Duration":3200000,"Confidence":0.4641154},{"Word":"apples","Offset":32700000,"Duration":7200000,"Confidence":0.5474731},{"Word":"pears","Offset":39900000,"Duration":7200000,"Confidence":0.68669873},{"Word":"bananas","Offset":47500000,"Duration":8800000,"Confidence":0.9829177},{"Word":"and","Offset":56700000,"Duration":2800000,"Confidence":0.9852306},{"Word":"oranges","Offset":59500000,"Duration":7200000,"Confidence":0.9747303},{"Word":"do","Offset":69900000,"Duration":2000000,"Confidence":0.9733427},{"Word":"you","Offset":71900000,"Duration":1200000,"Confidence":0.96302515},{"Word":"think","Offset":73100000,"Duration":3200000,"Confidence":0.99290127},{"Word":"i","Offset":76300000,"Duration":400000,"Confidence":0.989389},{"Word":"missed","Offset":76700000,"Duration":3600000,"Confidence":0.9898689},{"Word":"anything","Offset":80300000,"Duration":6400000,"Confidence":0.97900313}]},{"Confidence":0.88955027,"Lexical":"today i went to the shop i brought apples pears bananas and oranges do you think i missed anything","ITN":"today i went to the shop i brought apples pears bananas and oranges do you think i missed anything","MaskedITN":"today i went to the shop i brought apples pears bananas and oranges do you think i missed anything","Display":"today i went to the shop i brought apples pears bananas and oranges do you think i missed anything","Words":[{"Word":"today","Offset":11500000,"Duration":5200000,"Confidence":0.797704},{"Word":"i","Offset":16700000,"Duration":400000,"Confidence":0.9929609},{"Word":"went","Offset":17100000,"Duration":3200000,"Confidence":0.9909313},{"Word":"to","Offset":20300000,"Duration":800000,"Confidence":0.85922694},{"Word":"the","Offset":21100000,"Duration":1200000,"Confidence":0.9740201},{"Word":"shop","Offset":22300000,"Duration":5200000,"Confidence":0.99293035},{"Word":"i","Offset":28700000,"Duration":400000,"Confidence":0.7901372},{"Word":"brought","Offset":29100000,"Duration":2800000,"Confidence":0.022906482},{"Word":"apples","Offset":31900000,"Duration":8000000,"Confidence":0.97443783},{"Word":"pears","Offset":39900000,"Duration":7200000,"Confidence":0.6758339},{"Word":"bananas","Offset":47500000,"Duration":8800000,"Confidence":0.98287463},{"Word":"and","Offset":56700000,"Duration":2800000,"Confidence":0.9852306},{"Word":"oranges","Offset":59500000,"Duration":7200000,"Confidence":0.9747303},{"Word":"do","Offset":69900000,"Duration":2000000,"Confidence":0.9733427},{"Word":"you","Offset":71900000,"Duration":1200000,"Confidence":0.96302515},{"Word":"think","Offset":73100000,"Duration":3200000,"Confidence":0.99290127},{"Word":"i","Offset":76300000,"Duration":400000,"Confidence":0.989389},{"Word":"missed","Offset":76700000,"Duration":3600000,"Confidence":0.9898689},{"Word":"anything","Offset":80300000,"Duration":6400000,"Confidence":0.97900313}]},{"Confidence":0.8071691,"Lexical":"the day i went to the shop i bought apples pears bananas and oranges do you think i missed anything","ITN":"the day i went to the shop i bought apples pears bananas and oranges do you think i missed anything","MaskedITN":"the day i went to the shop i bought apples pears bananas and oranges do you think i missed anything","Display":"the day i went to the shop i bought apples pears bananas and oranges do you think i missed anything","Words":[{"Word":"the","Offset":11500000,"Duration":1600000,"Confidence":0.0040036854},{"Word":"day","Offset":13100000,"Duration":3600000,"Confidence":0.0051958677},{"Word":"i","Offset":16700000,"Duration":400000,"Confidence":0.992046},{"Word":"went","Offset":17100000,"Duration":3200000,"Confidence":0.99208415},{"Word":"to","Offset":20300000,"Duration":800000,"Confidence":0.86416095},{"Word":"the","Offset":21100000,"Duration":1200000,"Confidence":0.97456455},{"Word":"shop","Offset":22300000,"Duration":5200000,"Confidence":0.99249375},{"Word":"i","Offset":29100000,"Duration":400000,"Confidence":0.7901372},{"Word":"bought","Offset":29500000,"Duration":3200000,"Confidence":0.4641154},{"Word":"apples","Offset":32700000,"Duration":7200000,"Confidence":0.5474731},{"Word":"pears","Offset":39900000,"Duration":7200000,"Confidence":0.68669873},{"Word":"bananas","Offset":47500000,"Duration":8800000,"Confidence":0.9829177},{"Word":"and","Offset":56700000,"Duration":2800000,"Confidence":0.9852306},{"Word":"oranges","Offset":59500000,"Duration":7200000,"Confidence":0.9747303},{"Word":"do","Offset":69900000,"Duration":2000000,"Confidence":0.9733427},{"Word":"you","Offset":71900000,"Duration":1200000,"Confidence":0.96302515},{"Word":"think","Offset":73100000,"Duration":3200000,"Confidence":0.99290127},{"Word":"i","Offset":76300000,"Duration":400000,"Confidence":0.989389},{"Word":"missed","Offset":76700000,"Duration":3600000,"Confidence":0.9898689},{"Word":"anything","Offset":80300000,"Duration":6400000,"Confidence":0.97900313}]},{"Confidence":0.85787326,"Lexical":"today i went to the shop and i bought apples pears bananas and oranges do you think i missed anything","ITN":"today i went to the shop and i bought apples pears bananas and oranges do you think i missed anything","MaskedITN":"today i went to the shop and i bought apples pears bananas and oranges do you think i missed anything","Display":"today i went to the shop and i bought apples pears bananas and oranges do you think i missed anything","Words":[{"Word":"today","Offset":11500000,"Duration":5200000,"Confidence":0.797704},{"Word":"i","Offset":16700000,"Duration":400000,"Confidence":0.9929609},{"Word":"went","Offset":17100000,"Duration":3200000,"Confidence":0.9909313},{"Word":"to","Offset":20300000,"Duration":800000,"Confidence":0.85922694},{"Word":"the","Offset":21100000,"Duration":1200000,"Confidence":0.9740201},{"Word":"shop","Offset":22300000,"Duration":5200000,"Confidence":0.99293035},{"Word":"and","Offset":28300000,"Duration":1600000,"Confidence":0.0032146487},{"Word":"i","Offset":29900000,"Duration":400000,"Confidence":0.64129204},{"Word":"bought","Offset":30300000,"Duration":2400000,"Confidence":0.4008528},{"Word":"apples","Offset":32700000,"Duration":7200000,"Confidence":0.98722744},{"Word":"pears","Offset":39900000,"Duration":7200000,"Confidence":0.6866962},{"Word":"bananas","Offset":47500000,"Duration":8800000,"Confidence":0.9829177},{"Word":"and","Offset":56700000,"Duration":2800000,"Confidence":0.9852306},{"Word":"oranges","Offset":59500000,"Duration":7200000,"Confidence":0.9747303},{"Word":"do","Offset":69900000,"Duration":2000000,"Confidence":0.9733427},{"Word":"you","Offset":71900000,"Duration":1200000,"Confidence":0.96302515},{"Word":"think","Offset":73100000,"Duration":3200000,"Confidence":0.99290127},{"Word":"i","Offset":76300000,"Duration":400000,"Confidence":0.989389},{"Word":"missed","Offset":76700000,"Duration":3600000,"Confidence":0.9898689},{"Word":"anything","Offset":80300000,"Duration":6400000,"Confidence":0.97900313}]},{"Confidence":0.8059115,"Lexical":"the day i went to the shop i brought apples pears bananas and oranges do you think i missed anything","ITN":"the day i went to the shop i brought apples pears bananas and oranges do you think i missed anything","MaskedITN":"the day i went to the shop i brought apples pears bananas and oranges do you think i missed anything","Display":"the day i went to the shop i brought apples pears bananas and oranges do you think i missed anything","Words":[{"Word":"the","Offset":11500000,"Duration":1600000,"Confidence":0.0040036854},{"Word":"day","Offset":13100000,"Duration":3600000,"Confidence":0.0051958677},{"Word":"i","Offset":16700000,"Duration":400000,"Confidence":0.992046},{"Word":"went","Offset":17100000,"Duration":3200000,"Confidence":0.99208415},{"Word":"to","Offset":20300000,"Duration":800000,"Confidence":0.86416095},{"Word":"the","Offset":21100000,"Duration":1200000,"Confidence":0.97456455},{"Word":"shop","Offset":22300000,"Duration":5200000,"Confidence":0.99249375},{"Word":"i","Offset":28700000,"Duration":400000,"Confidence":0.7901372},{"Word":"brought","Offset":29100000,"Duration":2800000,"Confidence":0.022906482},{"Word":"apples","Offset":31900000,"Duration":8000000,"Confidence":0.97443783},{"Word":"pears","Offset":39900000,"Duration":7200000,"Confidence":0.6758339},{"Word":"bananas","Offset":47500000,"Duration":8800000,"Confidence":0.98287463},{"Word":"and","Offset":56700000,"Duration":2800000,"Confidence":0.9852306},{"Word":"oranges","Offset":59500000,"Duration":7200000,"Confidence":0.9747303},{"Word":"do","Offset":69900000,"Duration":2000000,"Confidence":0.9733427},{"Word":"you","Offset":71900000,"Duration":1200000,"Confidence":0.96302515},{"Word":"think","Offset":73100000,"Duration":3200000,"Confidence":0.99290127},{"Word":"i","Offset":76300000,"Duration":400000,"Confidence":0.989389},{"Word":"missed","Offset":76700000,"Duration":3600000,"Confidence":0.9898689},{"Word":"anything","Offset":80300000,"Duration":6400000,"Confidence":0.97900313}]}]}
2024-03-22 18:14:41,943 [DEBUG] Namspace.Client.Windows.Transcription.TranscriptionFacade - Received transcription response

I found https://github.com/microsoft/cognitive-services-speech-sdk-js/issues/686 and used the HTML file to validate the settings. The HTML file provided here does indeed return Recognized as expected.

Expected: "Today I went to the shop. I bought apples, pears, bananas, and oranges. Do you think I missed anything?" Output from C#: "Today I went to the shop I bought apples pears bananas and oranges do you think I missed anything"

Can anyone spot the probably obvious missing piece?

Upvotes: 0

Views: 156

Answers (1)

David_Strachan
David_Strachan

Reputation: 35

I raised this question also: https://learn.microsoft.com/en-gb/answers/questions/1632691/azure-ai-speech-service-no-punctuation-on-recogniz

The issue turns out that having enabledDictation on was at fault, and the reason this was not picked up in Test suites was the audio stream was being terminated and a raise condition was in play.

Still not sure about the differences of HTML and C# examples giving different results, but I can move on.

Upvotes: 0

Related Questions