CODEWITHSUNDEEP
htmlgoogle-chromefirefoxaudiowebrtc
wowpatrick
wowpatrick

Reputation: 5180

WebRTC - disable all audio processing

I'm currently trying to get a clean as possible audio channel via webrtc. Via the getUserMedia mediaconstraints object, I've set the following options:

constraints: {
        audio: {
            mandatory: {
                echoCancellation: false,
                googEchoCancellation: false,
                googAutoGainControl: false,
                googAutoGainControl2: false,
                googNoiseSuppression: false,
                googHighpassFilter: false,
                googTypingNoiseDetection: false,
                //googAudioMirroring: false // For some reason setting googAudioMirroring causes a navigator.getUserMedia error:  NavigatorUserMediaError
            }
        },
        video: false
    },

This greatly improves the audio quality, but there still seems to be audio processing present which causes the mutilation of the audio in the form of high frequency noise with some of the test samples.

There is a Chrome flag --use-file-for-fake-audio-capture as described at http://peter.sh/experiments/chromium-command-line-switches/#use-file-for-fake-audio-capture which allows input via file for testing. As mentioned in the description of the flag, all audio processing has to be disabled or the audio will be distorted - so there seems to be additional options for this purpose.

I also tried the --disable-audio-track-processing --audio-buffer-size=16 --enable-exclusive-audio Chrome flags, but still there seems to be some audio processing.

Is there any way to disable the still present audio processing (preferably via JS API)?

Upvotes: 23

Views: 11668

Answers (3)

Dmitriy G
Dmitriy G

Reputation: 359

Thanks to https://stackoverflow.com/a/58898246/9964552 answer. I have successfully mungled remote sdp offer by removing CN codecs and fixing fmtp string. That has removed comfort noise from audio and disabled audio processing

const removeComfortNoiseFromSdp = (sdp: sdpTransform.SessionDescription): sdpTransform.SessionDescription => {
  const audioMedias = sdp.media.filter(m => m.type === 'audio')
  const audioMedia = audioMedias[0]
  if (!audioMedia) {
    return sdp // no audio media
  }
  const whitelistedRtps = audioMedia.rtp.filter(rtp => {
    const codec = `${rtp.codec}`.toLowerCase()
    return codec !== 'cn'
  })
  const whitelistedPayloads = whitelistedRtps.map(rtp => rtp.payload)
  audioMedia.rtp = whitelistedRtps
  audioMedia.payloads = whitelistedPayloads.join(' ')
  audioMedia.fmtp = audioMedia.fmtp.map(fmtp => {
    return {
      ...fmtp,
      config: `${fmtp.config}`.replace('useinbandfec=1;', 'useinbandfec=1; stereo=1; maxaveragebitrate=510000'),
    }
  })
  return sdp
}

Upvotes: 0

Kim T
Kim T

Reputation: 6436

This is the updated way to disable audio processing and get a clean signal:

navigator.mediaDevices.getUserMedia({
  audio: {
    autoGainControl: false,
    channelCount: 2,
    echoCancellation: false,
    latency: 0,
    noiseSuppression: false,
    sampleRate: 48000,
    sampleSize: 16,
    volume: 1.0
  }
});

If you are streaming audio via WebRTC, it defaults to radio or phone quality audio optimized for voice. So make sure your SDP has stereo and maxaveragebitrate params:

a=fmtp:111 minptime=10;useinbandfec=1; stereo=1; maxaveragebitrate=510000

Upvotes: 6

xdumaine
xdumaine

Reputation: 10329

I'd wager that the variable bitrate (default) behavior of the opus codec is causing some compression or adjustment. You could manually mangle the SDP offer to use CBR (constant bitrate) instead of VBR (variable bit rate). When you get the SDP offer from the browser, change the line:

a=fmtp:111 minptime=10; useinbandfec=1

to:

a=fmtp:111 minptime=10; cbr=1

Note that I'm both adding cbr=1 and removing useinbandfec=1. I'm not positive that dropping useinbandfec is necessary, but it seems that in-band FEC (forwarding error correction) causes compression adjustment which you'd want to avoid as well.

Upvotes: 4

Related Questions