CODEWITHSUNDEEP
iosswiftcore-audioplaybackaudiounit
Muhammad Faizan
Muhammad Faizan

Reputation: 373

IOS AudioUnit playback crackle issue (swift)

I am trying to play bytes coming from the UDP from an android device in IOS. I am using TPCircularBuffer to play the bytes. My code is below:

let success = initCircularBuffer(&circularBuffer, 1024)
        if success {
            print("Circular buffer init was successful")
        } else {
            print("Circular buffer init not successful")
        }



func udpReceive() {
        receivingQueue.async {
            repeat {
                do {
                    let datagram = try self.tcpClient?.receive()
                    let byteData = datagram?["data"] as? Data
                    let dataLength = datagram?["length"] as? Int
                    self.dataLength = dataLength!

                    let _ = TPCircularBufferProduceBytes(&self.circularBuffer, byteData!.bytes, UInt32(dataLength! * MemoryLayout<UInt8>.stride * 2))

                } catch {
                    fatalError(error.localizedDescription)
                }
            } while true
        }


    }

    func consumeBuffer() -> UnsafeMutableRawPointer? {
        self.availableBytes = 0
        let tail = TPCircularBufferTail(&self.circularBuffer, &self.availableBytes)
        return tail
    }

We are recording with 16K sample rate and sending to IOS via UDP from Android side and from there we are using AudioUnit to play our bytes but the problem is the crackling and clipping sound in our voice.

Playback Callback code:

func performPlayback(
        _ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>,
        inTimeStamp: UnsafePointer<AudioTimeStamp>,
        inBufNumber: UInt32,
        inNumberFrames: UInt32,
        ioData: UnsafeMutablePointer<AudioBufferList>
    ) -> OSStatus {

        var buffer = ioData[0].mBuffers             

        let bufferTail = consumeBuffer()

        memcpy(buffer.mData, bufferTail, min(self.dataLength, Int(availableBytes)))
        buffer.mDataByteSize = UInt32(min(self.dataLength, Int(availableBytes)))


        TPCircularBufferConsume(&self.circularBuffer, UInt32(min(self.dataLength, Int(availableBytes))))

        return noErr
    }

UDP is sending us 1280 bytes per sample. What we think the problem is the BUFFER SIZE that is not being set correctly. Can anyone guide me how to set proper buffer size. It would be great help indeed. I know the work of @Gruntcakes as an voip engineer https://stackoverflow.com/a/57136561/12020007. I have also studied the work of @hotpaw2 and was looking at https://stackoverflow.com/a/58545845/12020007 to check if there is some threading issue. Any kind of help would be appreciated.

Upvotes: 1

Views: 707

Answers (2)

shayan
shayan

Reputation: 25

The playback callback Code is:

private let AudioController_RecordingCallback: AURenderCallback = {(
    inRefCon,
    ioActionFlags/*: UnsafeMutablePointer<AudioUnitRenderActionFlags>*/,
    inTimeStamp/*: UnsafePointer<AudioTimeStamp>*/,
    inBufNumber/*: UInt32*/,
    inNumberFrames/*: UInt32*/,
    ioData/*: UnsafeMutablePointer<AudioBufferList>*/)
    -> OSStatus

First we need to understand the callback based on our properties like our sample rate and number of channel, these things deduce the Frames the IOS callback will play, If our frames are less or more then the inNumberOfFrames the sound will contain crackle and clipping. The solution to the problem is the TPCircularBuffer that will memcpy exactly the inNumberOfFrames to the ioData to play the sound correctly. Make sure that you only copy the number of frames that the callBack is producing at that time as it can vary from time to time.

The problem with your code is that you are trying to play 1280bytes whereas the call back is expecting less than these bytes.

For TPCircularBuffer refer to https://github.com/michaeltyson/TPCircularBuffer

Upvotes: 1

hotpaw2
hotpaw2

Reputation: 70703

An Audio Unit callback should return only the requested number of frames (samples), as indicated by the inNumberFrames parameter. Your code appears to be copying some different number of samples into the AudioBufferList, which won't work, as an iOS Audio Unit will only send the requested number of frames to the audio output.

You can suggest a preferred buffer duration in your Audio Session configuration, but this is only a suggestion. iOS is free to ignore this suggestion, and use an inNumberFrames that is better matched to the device's audio hardware, system activity, and current power state.

Don't forget to pre-fill the circular buffer enough to account for the the maximum expected jitter in network (UDP) transit time. Perhaps measure network packet-to-packet latency jitter, and compute its statistics, min, max, std.dev. etc.

If your UDP buffers are not a power-of-2 in size, or contain samples that are not at the iOS hardware sample rate, then you will also have to account for fractional buffer and resampling jitter in your safety buffering overhead.

Upvotes: 5

Related Questions