CODEWITHSUNDEEP
iosswiftobjective-cavfoundationavcapturesession
mrousavy
mrousavy

Reputation: 1107

iOS Camera: `AVCaptureAudioDataOutput` activate audio session on the fly, avoid background music stutter

I have created a Camera using AVFoundation which is able to record video and audio using AVCaptureVideoDataOutput and AVCaptureAudioDataOutput. I create my capture session, attach all inputs and the video- and audio-data outputs, and the Camera then sits idle. The user is now able to start a video recording.

Problem

The problem with this is that immediately after I start the capture session (captureSession.startRunning()), the background music stutters. I assume this is because once the capture session starts running, the AVCaptureAudioDataOutput internally activates the AVAudioSession (AVAudioSession.setActive(...)), which I don't want it doing. I want it sitting idle (and not providing any audio output buffers) until I explicitly activate the Audio Session (once the user starts recording).

This is really annoying, since the Camera is the start-screen in our app and everytime the user opens or closes the app his music stutters.

I know that this is somehow possible because Snapchat works that way - you open the App and background audio smoothly continues to play. Once you start recording, there is a small stutter on the background music, but the Camera smoothly operates and starts recording once the short stutter is over.

My code:

func configureSession() {

    captureSession.beginConfiguration()

    // Video, Photo and Audio Inputs
    ...

    // Video Output
    ...

    // Audio Output
    audioOutput = AVCaptureAudioDataOutput()
    guard captureSession.canAddOutput(audioOutput!) else {
      throw CameraError.parameter(.unsupportedOutput(outputDescriptor: "audio-output"))
    }
    audioOutput!.setSampleBufferDelegate(self, queue: audioQueue)
    captureSession.addOutput(audioOutput!)

    try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord,
                                                    options: [.mixWithOthers,
                                                              .allowBluetoothA2DP,
                                                              .defaultToSpeaker,
                                                              .allowAirPlay])

    captureSession.commitConfiguration()
}

What I tried

Delay configuring the AVAudioSession.sharedInstance()

I tried to first configure the AVAudioSession.sharedInstance with the category AVAudioSession.Category.playback, and switch to .playAndRecord once I want to start recording audio.

This didn't work and the AVCaptureSessionRuntimeError event gets invoked immediately after starting the Camera with the Error code -10851, which means kAudioUnitErr_InvalidPropertyValue. I think this means that the AVCaptureAudioDataOutput is not allowed to record from the Audio Session, but I don't event want to do that right now - it should just be idle.

Delay adding the AVCaptureAudioDataOutput output

I tried to not add the audio output (AVCaptureAudioDataOutput) in the beginning, and only add it "on-demand" once the user starts recording, and while that worked fine for the background music (no stutter when starting, only short stutter once the user starts recording, exactly how I want it), it made the Preview freeze for a short amount of time (because the Capture Session is being reconfigured via beginConfiguration + audio output adding + commitConfiguration)

Does anyone know how it's possible to achieve what I'm trying to do here - or how Snapchat does it? Any help appreciated, thanks!

Upvotes: 8

Views: 1395

Answers (2)

mrousavy
mrousavy

Reputation: 1107

Finally figured it out. I simply created a separate AVCaptureSession specifically for the audio input/output which I synchronize with the main capture session's masterClock. I can then start/stop the secondary capture session on the fly (shortly before start recording)

Upvotes: 1

Gordon Childs
Gordon Childs

Reputation: 36072

The .mixWithOthers looks like the right thing to do to avoid interruptions, but default AVCaptureSession will configure the AVAudioSession for you, so set

captureSession.automaticallyConfiguresApplicationAudioSession = false

to stop it replacing your settings.

The property's documentation tells us that AVCaptureSession does more than just set the .playAndRecord category, a fact which may be important for your use case:

the receiver ensures the application's audio session is set to the PlayAndRecord category, and picks an appropriate microphone and polar pattern to match the video camera being used

Upvotes: 0

Related Questions