Start microphone using App Intents "required condition is false: IsFormatSampleRateAndChannelCountValid(format)"

Question

I want to include Speech Recognition in my app, for this I used the Speech Framework from Apple in combination with App Intents Framework. The app intent "Listen" starts the speech recognition. Now I have the problem that I keep getting the following error message:

*** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio',      reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)'
 terminating with uncaught exception of type NSException

on the following line:

 self.audioEngine.inputNode.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) { (buffer, when) in
            self.recognitionRequest?.append(buffer)
    }

This problem doesn't occur when I execute the same code with a simple button click, but ONYL with the App Intent "Listen". I am aware, that there is problem with the microphone that Siri uses while listening to the App intent. But how can I solve this issue? I did a lot of research and also tried it with async functions, but it didn't help

My Code:

import Speech
import UIKit

class TestVoice: UIControl, SFSpeechRecognizerDelegate {    
let speechRecognizer        = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))
   var recognitionRequest      : SFSpeechAudioBufferRecognitionRequest?
   var recognitionTask         : SFSpeechRecognitionTask?
   let audioEngine             = AVAudioEngine()


func stopRecording() {
    self.audioEngine.stop()
    self.recognitionRequest?.endAudio()
}

func setupSpeech() {
    
       self.speechRecognizer?.delegate = self
       SFSpeechRecognizer.requestAuthorization { (authStatus) in

           switch authStatus {
               case .authorized:
                   print("yes")
               case .denied:
                   print("died")
               case .restricted:
                   print("died")
               case .notDetermined:
                   print("none")
           }
           OperationQueue.main.addOperation() {
           }
       }
   }


func startRecording() -> Bool {
        setupSpeech()
        clearSessionData()
        createAudioSession()
        recognitionRequest = bufferRecRequest()
        recognitionRequest?.shouldReportPartialResults = true
        self.recognitionTask = speechRecognizer?.recognitionTask(with: recognitionRequest!, resultHandler: { (result, error) in
            
            var finished = false
            
            if let result = result {
                //do something
                finished = result.isFinal
            }
            
            if error != nil || finished {
                self.audioEngine.stop()
                self.audioEngine.inputNode.removeTap(onBus: 0)
                self.recognitionRequest = nil
                self.recognitionTask = nil
            }
        })
        
    
    let recordingFormat = self.audioEngine.inputNode.outputFormat(forBus: 0)
    self.audioEngine.inputNode.installTap(onBus: 0, bufferSize: 2048, format: recordingFormat) { (buffer, when) in
            self.recognitionRequest?.append(buffer)
    }
        
        self.audioEngine.prepare()
        
        do {
            try self.audioEngine.start()
        } catch {
            print("audioEngine couldn't start because of an error.")
            delegate?.showFeedbackError(title: "Sorry", message: "Your microphone is used somewhere else")
            return false
        }
    return true
    }

func clearSessionData(){
    if recognitionTask != nil {
        recognitionTask?.cancel()
        recognitionTask = nil
    }
}

func bufferRecRequest()->SFSpeechAudioBufferRecognitionRequest{
    self.recognitionRequest = SFSpeechAudioBufferRecognitionRequest()
    guard let recognitionRequest = recognitionRequest else {
        fatalError("Unable to create an SFSpeechAudioBufferRecognitionRequest object")
    }
    return recognitionRequest
}

func createAudioSession()-> Bool{
    
    do {
        try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, options: .mixWithOthers)
        
    } catch {
        print("audioSession properties weren't set because of an error.")
        delegate?.showFeedbackError(title: "Sorry", message: "Mic is busy")
        return false
    }
    return true
}
}

The App Intent

 import AppIntents
 import UIKit


 struct ListenIntent: AppIntent {
     static var openAppWhenRun: Bool = true

    @available(iOS 16, *)
     static let title: LocalizedStringResource = "Listen"
     static var description =
        IntentDescription("Listens to the User")
     let speechRecognizer = TestVoice()

     func perform() throws -> some IntentResult & ProvidesDialog {
         speechRecognizer.startRecording()
         return .result(dialog: "Done")
        
}}

Answer 1

What I've discovered is that triggering the App Intent via Siri can cause the mic to be still blocked by Siri. As a result, the inputNode is not ready for recording and you can't install tap on it (=> crash). Solution: After you set the AVAudioSession (category, activate...), check the input format whether it's ready:

 let inputFormat = audioEngine.inputNode.inputFormat(forBus: 0)
 if inputFormat == 0 || inputFormat == 0 {
   // don't continue with the setup, the mic is not ready
   // throw error or return here and try it later
 }

This is also mentioned in Apple Docs:

When the engine renders to and from an audio device, the AVAudioSession category and the availability of hardware determines whether an app performs input (for example, input hardware isn’t available in tvOS). Check the input node’s input format (specifically, the hardware format) for a nonzero sample rate and channel count to see if input is in an enabled state. Trying to perform input through the input node when it isn’t available or in an enabled state causes the engine to throw an error (when possible) or an exception.