CODEWITHSUNDEEP
c#unity-game-enginehololensmrtk
mattycorbett
mattycorbett

Reputation: 23

Determining if the user is speaking on a HL2

I've been using the MRTK MicStream APIs to try to merely determine if the HoloLens 2 user is speaking. I've tried for a couple of weeks. All attempts have failed for reasons documented here and here. I am officially giving up on the MicStream API, and also tried the AudioFrame method from the MediaCapture APIs which also fail. (MediaCapture is how the MicStream DLL also attempts to access raw audio data which you can see here, on line 351 - 357 so this isnt surprising).

My question is: how else can I determine only if a user is speaking? I do not need dictation, a recording, or to use speech commands. I only want to know if the two user microphones on channel 1 and 2 are active above the normal room amplitude (in real time). Does anyone know any other ways outside these methods?

Upvotes: 0

Views: 130

Answers (1)

mattycorbett
mattycorbett

Reputation: 23

For what its worth - I finally fixed it. The MicStream DLL's MediaCapture instance was conflicting with one I had already instantiated for photo captures. In short, you cant use MicStream with another MediaCapture instance. I tried to set the settings for SharingMode on the first MediaCapture (in my script for capturing photos, but this didnt work. I had to completely stop using the MicStream .dll and streamline the audio capture under one MediaCapture instantiated with StreamingCaptureMode.AudioAndVideo. This fixed the problem.

Upvotes: 1

Related Questions