Jason Yee
Jason Yee

Reputation: 41

Write incoming audio raw data to AVAudioPCMBuffer of AVAudioFile and play AVAudioFile

I'm receiving packets of audio raw data within a callback function with the following properties:

In other words, each packet of incoming audio data is an array of 640 pointers to audio raw data values. Each audio raw data value has bit depth of 2 bytes (16 bits), PCM encoded.

I create an AVAudioFile prior to receiving incoming audio raw data. Once recording starts, I save the packets of audio raw data to an AVAudioPCMBuffer, convert the processing format of AVAudioPCMBuffer to the output format of the main mixer node of AVAudioEngine using AVAudioConverter, and write the converted AVAudioPCMBuffer to the AVAudioFile. This conversion of AVAudioPCMBuffer format is required since the output format of the main mixer node of AVAudioEngine is 2 ch, 48000 Hz, Float32, non-interleaved. Finally, once the audio recording stops, I play the AVAudioFile using AVAudioEngine.

Problem: When the AVAudioFile is played, I hear only white noise when I speak into the mic. However, the duration of the white noise is the same as the length of time I speak into the mic, which seems to indicate I'm close to the solution but not quite there yet.

My code in Swift 5 is as follows:

1. Create AVAudioFile

func createAudioFile() {
        let fileMgr = FileManager.default
        let dirPaths = fileMgr.urls(for: .cachesDirectory, in: .userDomainMask)
        var recordSettings: [String : Any] = [:]
        recordSettings[AVFormatIDKey] = kAudioFormatLinearPCM
        recordSettings[AVAudioFileTypeKey] = kAudioFileCAFType
        recordSettings[AVSampleRateKey] = 44100
        recordSettings[AVNumberOfChannelsKey] = 2
        self.soundFileUrl = dirPaths[0].appendingPathComponent("recording.pcm")
        do {
            audioFile = try AVAudioFile(forWriting: soundFileUrl!, settings: recordSettings, commonFormat: .pcmFormatFloat32, interleaved: false)
        } catch let error as NSError {
            print("error:", error.localizedDescription)
        }
    }

2. Process incoming audio raw data within callback function

func onAudioRawDataReceived(_ rawData: AudioRawData) {
        // when audio recording starts
        if self.saveAudio == true {
            do {
                let channels = 1
                let format = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: Double(rawData.sampleRate), channels: AVAudioChannelCount(channels), interleaved: false)!
                let audioFileBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(rawData.bufferLen / channels))
                let int16ChannelData = audioFileBuffer?.int16ChannelData!
                for i in 0..<rawData.bufferLen {
                    int16ChannelData![0][i] = Int16(rawData.buffer.pointee)
                    rawData.buffer += 1
                }
                audioFileBuffer!.frameLength = AVAudioFrameCount(rawData.bufferLen / channels)
                let outputFormat = audioEngine.mainMixerNode.outputFormat(forBus: 0)
                converter = AVAudioConverter(from: format, to: outputFormat)!
                let outputAudioFileBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat, frameCapacity: AVAudioFrameCount(rawData.bufferLen / channels))
                converter.convert(to: outputAudioFileBuffer!, error: nil) { inNumPackets, outStatus in
                    outStatus.pointee = .haveData
                    return audioFileBuffer
                }
                try audioFile!.write(from: outputAudioFileBuffer!)
                print("success")
            }
            catch let error as NSError {
                print("Error:", error.localizedDescription)
            }
            isRecordingAudio = true
        }
        else {
        //when audio recording stops
            if isRecordingAudio == true {
                playPCM(soundFileUrl!.absoluteString)
                isRecordingAudio = false
            }
        }
    }

3. Play audio file once recording stops

func playPCM(_ filefullpathstr: String) {
        do {
            let audioFile = try AVAudioFile(forReading: URL(string: filefullpathstr)!)
            let audioFormat = audioFile.processingFormat
            let audioFrameCount = UInt32(audioFile.length)
            let audioFileBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: audioFrameCount)
            try audioFile.read(into: audioFileBuffer!, frameCount: audioFrameCount)
            let mainMixer = audioEngine.mainMixerNode
            audioEngine.attach(audioFilePlayer)
            audioEngine.connect(audioFilePlayer, to:mainMixer, format: audioFileBuffer?.format)
            audioEngine.prepare()
            try audioEngine.start()
            audioFilePlayer.play()
            audioFilePlayer.scheduleBuffer(audioFileBuffer!, at: nil, options: [], completionHandler: {
                print("scheduled buffer")
            })
        } catch let error as NSError {
            print("Error:", error.localizedDescription)
        }
    }

I'm very new to audio processing on iOS using Swift and I'd greatly appreciate any advice or tips on solving this problem.

Many thanks for your help in advance.

Upvotes: 4

Views: 1380

Answers (0)

Related Questions