iCoder86
iCoder86

Reputation: 1895

Record Other App's Speaker output in iOS App

I am working on an application where I need to record and translate the audio coming out of the speaker while using other application i.e. YouTube or Facebook.

I have tried implementing AVFoundation and Speech from apple to record and translate the audio recorded in background.

Here is my code.

import UIKit
import AVFoundation
import Speech

class ViewController: UIViewController, SFSpeechRecognizerDelegate {

    private let speechRecognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))!
    private var recognitionRequest: SFSpeechAudioBufferRecognitionRequest?
    private var recognitionTask: SFSpeechRecognitionTask?
    private let audioEngine = AVAudioEngine()

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view.
    
        speechRecognizer.delegate = self
    
        // Asynchronously make the authorization request.
        SFSpeechRecognizer.requestAuthorization { authStatus in

            // Divert to the app's main thread so that the UI
            // can be updated.
            OperationQueue.main.addOperation {
                print(authStatus)
            }
        }
    
        self.setupSpeechRecogniser()
        self.startListning()
    }

    func setupSpeechRecogniser() {
    
        recognitionTask?.cancel()
        self.recognitionTask = nil
    
        recognitionRequest = SFSpeechAudioBufferRecognitionRequest()
        guard let recognitionRequest = recognitionRequest else { fatalError("Unable to create a SFSpeechAudioBufferRecognitionRequest object") }
        recognitionRequest.shouldReportPartialResults = true
        recognitionRequest.requiresOnDeviceRecognition = true
    
        // Keep speech recognition data on device
        if #available(iOS 13, *) {
            recognitionRequest.requiresOnDeviceRecognition = false
        }
    
        // Create a recognition task for the speech recognition session.
        // Keep a reference to the task so that it can be canceled.
        recognitionTask = speechRecognizer.recognitionTask(with: recognitionRequest) { result, error in
            var isFinal = false

            if let result = result {
                // Update the text view with the results.
                //self.latestResult = result.bestTranscription.formattedString
                print(result.bestTranscription.segments.last?.substring ?? "")
                isFinal = result.isFinal
            }

            if error != nil || isFinal {
                // Stop recognizing speech if there is a problem.
                self.audioEngine.stop()
                self.recognitionRequest = nil
                self.recognitionTask = nil
            }
        }
    }

    func startListning() {
        let inputNode = audioEngine.inputNode
        let bus = 0
        inputNode.installTap(onBus: bus, bufferSize: 2048, format: inputNode.outputFormat(forBus: bus)) {
            (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
            //print("--> \(buffer)")
            self.recognitionRequest?.append(buffer)
        }
        audioEngine.prepare()
        try! audioEngine.start()
    }
}

Can anyone suggests me best ways to do so.

There is one application did exactly the same which is MiniSpeech on AppleStore.

Link To The App

Upvotes: 0

Views: 345

Answers (0)

Related Questions