Can't reverse AVAsset audio properly. The only result is white noise

Question

I'm trying to reverse an AVAsset audio and save it to a file. To make things clear, I've made simple application with the issue https://github.com/ksenia-lyagusha/AudioReverse.git

The application takes mp4 video file from bundle, exports it to Temporary folder in the sandbox as single m4a file, then tries to read it from there, reverse and save result file back. Temporary m4a file is OK.

The only result of my reverse part is Audio file in the Sandbox with white noise.

There is the part of code below, that is in charge of reversing AVAsset. It is based on related questions

• How to reverse an audio file?
• iOS audio manipulation - play local .caf file backwards

However, it doesn't work for me.

OSStatus theErr = noErr;
UInt64 fileDataSize = 0;
AudioFileID inputAudioFile;
AudioStreamBasicDescription theFileFormat;
UInt32 thePropertySize = sizeof(theFileFormat);

theErr = AudioFileOpenURL((__bridge CFURLRef)[NSURL URLWithString:inputPath], kAudioFileReadPermission, 0, &inputAudioFile);

thePropertySize = sizeof(fileDataSize);
theErr = AudioFileGetProperty(inputAudioFile, kAudioFilePropertyAudioDataByteCount, &thePropertySize, &fileDataSize);

UInt32 ps = sizeof(AudioStreamBasicDescription) ;
AudioFileGetProperty(inputAudioFile, kAudioFilePropertyDataFormat, &ps, &theFileFormat);

UInt64 dataSize = fileDataSize;
void *theData = malloc(dataSize);

// set up output file
AudioFileID outputAudioFile;

AudioStreamBasicDescription myPCMFormat;

myPCMFormat.mSampleRate       = 44100;
myPCMFormat.mFormatID         = kAudioFormatLinearPCM;
// kAudioFormatFlagsCanonical is deprecated
myPCMFormat.mFormatFlags      = kAudioFormatFlagIsFloat | kAudioFormatFlagIsNonInterleaved;
myPCMFormat.mChannelsPerFrame = 1;
myPCMFormat.mFramesPerPacket  = 1;
myPCMFormat.mBitsPerChannel   = 32;
myPCMFormat.mBytesPerPacket   = (myPCMFormat.mBitsPerChannel / 8) * myPCMFormat.mChannelsPerFrame;
myPCMFormat.mBytesPerFrame    = myPCMFormat.mBytesPerPacket;



NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:@"ReverseAudio.caf"];
NSURL *outputURL = [NSURL fileURLWithPath:exportPath];

theErr = AudioFileCreateWithURL((__bridge CFURLRef)outputURL,
                       kAudioFileCAFType,
                       &myPCMFormat,
                       kAudioFileFlags_EraseFile,
                       &outputAudioFile);

//Read data into buffer
//if readPoint  = dataSize, then bytesToRead = 0 in while loop and
//it is endless
SInt64 readPoint  = dataSize-1;
UInt64 writePoint = 0;

while(readPoint > 0)
{
    UInt32 bytesToRead = 2;
    AudioFileReadBytes(inputAudioFile, false, readPoint, &bytesToRead, theData);
    // bytesToRead is now the amount of data actually read

    UInt32 bytesToWrite = bytesToRead;
    AudioFileWriteBytes(outputAudioFile, false, writePoint, &bytesToWrite, theData);
    // bytesToWrite is now the amount of data actually written

    writePoint += bytesToWrite;
    readPoint -= bytesToRead;
}

free(theData);
AudioFileClose(inputAudioFile);
AudioFileClose(outputAudioFile);

If I change file type in AudioFileCreateWithURL from kAudioFileCAFType to another the result file is not created in the Sandbox at all.

Thanks for any help.

Answer 1

You get white noise because your in and out file formats are incompatible. You have different sample rates and channels and probably other differences. To make this work you need to have a common (PCM) format mediating between reads and writes. This is a reasonable job for the new(ish) AVAudio frameworks. We read from file to PCM, shuffle the buffers, then write from PCM to file. This approach is not optimised for large files, as all data is read into the buffers in one go, but is enough to get you started.

You can call this method from your getAudioFromVideo completion block. Error handling ignored for clarity.

- (void)readAudioFromURL:(NSURL*)inURL reverseToURL:(NSURL*)outURL {

//prepare the in and outfiles

  AVAudioFile* inFile = 
     [[AVAudioFile alloc] initForReading:inURL error:nil];

  AVAudioFormat* format = inFile.processingFormat;
  AVAudioFrameCount frameCount =(UInt32)inFile.length;
  NSDictionary* outSettings = @{
             AVNumberOfChannelsKey:@(format.channelCount)
            ,AVSampleRateKey:@(format.sampleRate)};

  AVAudioFile* outFile = 
    [[AVAudioFile alloc] initForWriting:outURL
                               settings:outSettings
                                  error:nil];

//prepare the forward and reverse buffers
  self.forwaredBuffer = 
    [[AVAudioPCMBuffer alloc] initWithPCMFormat:format 
                                  frameCapacity:frameCount];
  self.reverseBuffer = 
    [[AVAudioPCMBuffer alloc] initWithPCMFormat:format 
                                  frameCapacity:frameCount];

//read file into forwardBuffer
    [inFile readIntoBuffer:self.forwaredBuffer error:&error];

//set frameLength of reverseBuffer to forwardBuffer framelength
    AVAudioFrameCount frameLength = self.forwaredBuffer.frameLength; 
    self.reverseBuffer.frameLength = frameLength;

//iterate over channels 

     //stride is 1 or 2 depending on interleave format
     NSInteger stride = self.forwaredBuffer.stride;  

    for (AVAudioChannelCount channelIdx = 0;                    
          channelIdx < self.forwaredBuffer.format.channelCount;
          channelIdx++) {
      float* forwaredChannelData = 
          self.forwaredBuffer.floatChannelData[channelIdx];
      float* reverseChannelData = 
          self.reverseBuffer.floatChannelData[channelIdx];
      int32_t reverseIdx = 0;

     //iterate over samples, allocate to reverseBuffer in reverse order   
      for (AVAudioFrameCount frameIdx = frameLength; 
                    frameIdx >0; 
                    frameIdx--) {
           float sample = forwaredChannelData[frameIdx*stride];
           reverseChannelData[reverseIdx*stride] = sample;
           reverseIdx++;
            }
        }

//write reverseBuffer to outFile
        [outFile writeFromBuffer:self.reverseBuffer error:nil];
    }

Answer 2

I wasn't able to find the problem in your code, however I suggest you reversing AVAsset using AVAssetWriter. Following code is based on iOS reverse audio through AVAssetWritet. I've added additional method there to make it work. Finally I've got reversed file.

static NSMutableArray *samples;

static OSStatus sampler(CMSampleBufferRef sampleBuffer, CMItemCount index, void *refcon)
{
    [samples addObject:(__bridge id _Nonnull)(sampleBuffer)];
    return noErr;
}

- (void)reversePlayAudio:(NSURL *)inputURL
{
    AVAsset *asset = [AVAsset assetWithURL:inputURL];

    AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:asset error:nil];

    AVAssetTrack* audioTrack = [[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];

    NSMutableDictionary* audioReadSettings = [NSMutableDictionary dictionary];
    [audioReadSettings setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM]
                         forKey:AVFormatIDKey];

    AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:audioReadSettings];
    [reader addOutput:readerOutput];
    [reader startReading];

    NSDictionary *outputSettings = @{AVFormatIDKey     : @(kAudioFormatMPEG4AAC),
                                 AVSampleRateKey       : @(44100.0),
                                 AVNumberOfChannelsKey : @(1),
                                 AVEncoderBitRateKey   : @(128000),
                                 AVChannelLayoutKey    : [NSData data]};

    AVAssetWriterInput *writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio
                                                                     outputSettings:outputSettings];

    NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:@"reverseAudio.m4a"];

    NSURL *exportURL = [NSURL fileURLWithPath:exportPath];
    NSError *writerError = nil;
    AVAssetWriter *writer = [[AVAssetWriter alloc] initWithURL:exportURL
                                                      fileType:AVFileTypeAppleM4A
                                                         error:&writerError];
    [writerInput setExpectsMediaDataInRealTime:NO];
    writer.shouldOptimizeForNetworkUse = NO;
    [writer addInput:writerInput];
    [writer startWriting];
    [writer startSessionAtSourceTime:kCMTimeZero];

    CMSampleBufferRef sample;// = [readerOutput copyNextSampleBuffer];
    samples = [[NSMutableArray alloc] init];

    while (sample != NULL) {
        sample = [readerOutput copyNextSampleBuffer];

        if (sample == NULL)
            continue;

        CMSampleBufferCallForEachSample(sample, &sampler, NULL);

        CFRelease(sample);
    }

    NSArray* reversedSamples = [[samples reverseObjectEnumerator] allObjects];

    for (id reversedSample in reversedSamples) {
        if (writerInput.readyForMoreMediaData)  {
            [writerInput appendSampleBuffer:(__bridge CMSampleBufferRef)(reversedSample)];
        }
        else {
            [NSThread sleepForTimeInterval:0.05];
        }
    }

    [samples removeAllObjects];
    [writerInput markAsFinished];
    dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
    dispatch_async(queue, ^{

        [writer finishWritingWithCompletionHandler:^{

            //  writing is finished
            // reversed audio file in TemporaryDirectory in the Sandbox
        }];
    });
}

Known issues of the code.

1. There might be some problems with the memory, if the audio is long.
2. The audio file's duration is longer than original's. (As a quick fix you might cut it down as usual AVAsset).