CODEWITHSUNDEEP
webrtc
Raghav Thakur
Raghav Thakur

Reputation: 321

Multi party video chat- Detect the peer speaker

I have implemented the multiparty video chat (audio and video enabled) and it is working fine. How can I figure out which peer is speaking and highlight a green icon besides that user.

The onStreamAdded gets called only when a new stream is added but how do I track who is speaking currently.

Regards Raghav

Upvotes: 3

Views: 4726

Answers (2)

Ragesh Punathil
Ragesh Punathil

Reputation: 85

Hi you can use below logic to show active user on page.

Typescript:- 
        class AudioListenerBase {
          private audio_progress: number = 0;
          private audioContext = new AudioContext();
          private analyser: AnalyserNode;
          private microphone: MediaStreamAudioSourceNode;
          private javascriptNode: ScriptProcessorNode;

          public remotesElement: any;

      constructor(
        private zone: NgZone,
        private cd: ChangeDetectorRef,
        private stream: any,  
        private audioProgressCallBack: any
      ) {
        this.analyser = this.audioContext.createAnalyser();
        this.microphone = this.audioContext.createMediaStreamSource(stream);
        this.javascriptNode = this.audioContext.createScriptProcessor(2048, 1, 1);

        this.analyser.smoothingTimeConstant = 0.8;
        this.analyser.fftSize = 1024;

        this.microphone.connect(this.analyser);
        this.analyser.connect(this.javascriptNode);
        this.javascriptNode.connect(this.audioContext.destination);
        this.javascriptNode.onaudioprocess = (() => {
          var array = new Uint8Array(this.analyser.frequencyBinCount);
          this.analyser.getByteFrequencyData(array);
          var values = 0;

          var length = array.length;
          for (var i = 0; i < length; i++) {
            values += (array[i]);
          }
          var average = (values / length) * 10;


          if (this.audio_progress - average > 5 || average - this.audio_progress > 5)
            this.zone.run(() => {
              this.audio_progress = average;
              this.cd.detectChanges();
              audioProgressCallBack(this.audio_progress, this.remotesElement)

            });

        });

        return this;

      }
    }


usage on component :-

       this.myAudioListener = new AudioListenerBase(this.zone, this.changeDetectorRef, stream, (val, remotesElement) => {
                    this.audio_progress = val;
                  });

On component Html:

     <div> <p>{{item.username}}</p> <p style="font-size:10px">{{item.audio_progress>20?'speaking..':''}}</p></div>

Upvotes: 1

Michael Gorham
Michael Gorham

Reputation: 1284

Take a look at the Otalk hark github project.

Hark is a tiny browser/commonJS module that listens to an audio stream, and emits events indicating whether the user is speaking or not. Hark uses the webaudio API to FFT (get the power of) the audio in the audio stream. If the power is above a threshold, it's determined to be speech.

https://github.com/otalk/hark

Upvotes: 3

Related Questions