Vëx
Vëx

Reputation: 1376

Capturing microphone input on macOS using ScreenCaptureKit

I'm having troubles capturing microphone input using ScreenCaptureKit on macOS. (note, macOS 15.0+)
Video capture is going quite fine, but capturing anything arriving from built-in mic, simply get's ignored...

Here's some sample code that I'm using:

import AVFAudio
import ScreenCaptureKit

// ... removed other initialization things for clarity
streamConfiguration.captureMicrophone = true
streamConfiguration.microphoneCaptureDeviceID = nil  //by docs, this should use built-in microphone

// on macOS 15+ this saves output directly to a file
recordingConfiguration.outputURL = "some_file"  //just holds file location on disk

// streamOutput config
streamOutput = CaptureEngineStreamOutput(continuation: continuation)
streamOutput?.capturedFrameHandler = { continuation.yield($0) }
streamOutput?.pcmBufferHandler = { self.powerMeter.process(buffer: $0) }

// initialize main stream with given parameters
stream = SCStream(filter: contentFilter, configuration: streamConfiguration, delegate: streamOutput)

//.. and file recording output
addRecordingOutput(configuration: recordingConfiguration)

// check if we have strong reference
guard let streamOutput = streamOutput else {
    logger.error("Unable to stream capture references. Aborting.")
    return
}
            
do {
    // Add audio and video stream outputs...
    try stream.addStreamOutput(streamOutput, type: .screen, sampleHandlerQueue: videoSamplesQueue)
    try stream.addStreamOutput(streamOutput, type: .audio, sampleHandlerQueue: audioSamplesQueue)
    try stream.addStreamOutput(streamOutput, type: .microphone, sampleHandlerQueue: microphoneSamplesQueue)
                
    stream.startCapture()
    logger.info("Started capturing...")
} catch {
    logger.error("Unable to start capturing.")
}

contentFilter, streamConfiguration and recordingConfiguration are pretty much helper classes that are needed to configure SCStream, nothing marginal on that end.

What I've tried so far

  1. Apple's official source on this topic brought no results. Seems like they completely ignored this topic and even if you run their sample app you won't be able to record any microphone input...

  2. Using any of AVAudio... framework(s) isn't available on Mac but is solely present for iOS systems. So using that is simply out of the game.

  3. Searching SO brought only this question: Synchronizing ScreenCaptureKit with microphone audio but seems like no-one still replied for months.

  4. Googling for anything even closely related brings no luck as well.

Both Info.plist privacy settings and Resource Access (Audio Input) have been set.

If anyone has suggestions or perhaps short sample code, it would be greatly appreciated.

Upvotes: 2

Views: 162

Answers (1)

Gordon Childs
Gordon Childs

Reputation: 36159

Make sure you set captureMicrophone to true in SCStreamConfiguration

and don't forget the microphone privacy description in your Info.plist, or you'll capture no buffers. Also make sure you have Audio Input enabled in App Sandbox or Hardened Runtime or both if applicable.

Update:
I looked at the Apple sample code and the authors forgot to

  1. enable audio input for the sandbox, and
  2. add a microphone privacy description

If you complete these two steps the audio meter starts working. The code still fails to produce a recording (for me, and that sounds like a whole other question), but audio capture from the microphone does appear to be working.

audio input configuration for sandbox in Xcode microphone privacy usage string configuration in Xcode

Upvotes: 0

Related Questions