AVFoundation - Reverse an AVAsset and output video file

Question

I've seen this question asked a few times, but none of them seem to have any working answers.

The requirement is to reverse and output a video file (not just play it in reverse) keeping the same compression, format, and frame rate as the source video.

Ideally, the solution would be able to do this all in memory or buffer and avoid generating the frames into image files (for ex: using AVAssetImageGenerator) and then recompiling it (resource intensive, unreliable timing results, changes in frame/image quality from original, etc.).

--

My contribution: This is still not working, but the best I've tried so far:

• Read in the sample frames into an array of CMSampleBufferRef[] using AVAssetReader.
• Write it back in reverse order using AVAssetWriter.
• Problem: Seems like timing for each frame is saved in the CMSampleBufferRef so even appending them backwards will not work.
• Next, I tried swapping the timing information of each frame with reverse/mirror frame.
• Problem: This causes an unknown error with AVAssetWriter.

• Next Step: I'm going to look into AVAssetWriterInputPixelBufferAdaptor

  - (AVAsset *)assetByReversingAsset:(AVAsset *)asset {
      NSURL *tmpFileURL = [NSURL URLWithString:@"/tmp/test.mp4"];    
      NSError *error;
  
      // initialize the AVAssetReader that will read the input asset track
      AVAssetReader *reader = [[AVAssetReader alloc] initWithAsset:asset error:&error];
      AVAssetTrack *videoTrack = [[asset tracksWithMediaType:AVMediaTypeVideo] lastObject];
  
      AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:videoTrack outputSettings:nil];
      [reader addOutput:readerOutput];
      [reader startReading];
  
      // Read in the samples into an array
      NSMutableArray *samples = [[NSMutableArray alloc] init];
  
      while(1) {
          CMSampleBufferRef sample = [readerOutput copyNextSampleBuffer];
  
          if (sample == NULL) {
              break;
          }
  
          [samples addObject:(__bridge id)sample];
          CFRelease(sample);
      }
  
      // initialize the the writer that will save to our temporary file.
      CMFormatDescriptionRef formatDescription = CFBridgingRetain([videoTrack.formatDescriptions lastObject]);
      AVAssetWriterInput *writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:nil sourceFormatHint:formatDescription];
      CFRelease(formatDescription);
  
      AVAssetWriter *writer = [[AVAssetWriter alloc] initWithURL:tmpFileURL
                                                        fileType:AVFileTypeMPEG4
                                                           error:&error];
      [writerInput setExpectsMediaDataInRealTime:NO];
      [writer addInput:writerInput];
      [writer startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp((__bridge CMSampleBufferRef)samples[0])];
      [writer startWriting];
  
  
      // Traverse the sample frames in reverse order
      for(NSInteger i = samples.count-1; i >= 0; i--) {
          CMSampleBufferRef sample = (__bridge CMSampleBufferRef)samples[i];
  
          // Since the timing information is built into the CMSampleBufferRef 
          // We will need to make a copy of it with new timing info. Will copy
          // the timing data from the mirror frame at samples[samples.count - i -1]
  
          CMItemCount numSampleTimingEntries;
          CMSampleBufferGetSampleTimingInfoArray((__bridge CMSampleBufferRef)samples[samples.count - i -1], 0, nil, &numSampleTimingEntries);
          CMSampleTimingInfo *timingInfo = malloc(sizeof(CMSampleTimingInfo) * numSampleTimingEntries);
          CMSampleBufferGetSampleTimingInfoArray((__bridge CMSampleBufferRef)sample, numSampleTimingEntries, timingInfo, &numSampleTimingEntries);
  
          CMSampleBufferRef sampleWithCorrectTiming;
          CMSampleBufferCreateCopyWithNewTiming(
                                                kCFAllocatorDefault,
                                                sample,
                                                numSampleTimingEntries,
                                                timingInfo,
                                                &sampleWithCorrectTiming);
  
          if (writerInput.readyForMoreMediaData)  {
              [writerInput appendSampleBuffer:sampleWithCorrectTiming];
          }
  
          CFRelease(sampleWithCorrectTiming);
          free(timingInfo);
      }
  
      [writer finishWriting];
  
      return [AVAsset assetWithURL:tmpFileURL];
  }
  

Answer 1

Swift 5 version of Original Answer:

extension AVAsset {
    func getReversedAsset(outputURL: URL) -> AVAsset? {
        do {
            let reader = try AVAssetReader(asset: self)

            guard let videoTrack = tracks(withMediaType: AVMediaType.video).last else {
                return .none
            }

            let readerOutputSettings = [
                "\(kCVPixelBufferPixelFormatTypeKey)": Int(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange)
            ]

            let readerOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: readerOutputSettings)

            reader.add(readerOutput)
            reader.startReading()

            // Read in frames (CMSampleBuffer is a frame)
            var samples = [CMSampleBuffer]()
            while let sample = readerOutput.copyNextSampleBuffer() {
                samples.append(sample)
            }

            // Write to AVAsset
            let writer = try AVAssetWriter(outputURL: outputURL, fileType: AVFileType.mp4)

            let writerOutputSettings = [
                AVVideoCodecKey: AVVideoCodecType.h264,
                AVVideoWidthKey: videoTrack.naturalSize.width,
                AVVideoHeightKey: videoTrack.naturalSize.height,
                AVVideoCompressionPropertiesKey: [AVVideoAverageBitRateKey: videoTrack.estimatedDataRate]
            ] as [String : Any]
            
            let sourceFormatHint = videoTrack.formatDescriptions.last as! CMFormatDescription
            let writerInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: writerOutputSettings, sourceFormatHint: sourceFormatHint)
            writerInput.expectsMediaDataInRealTime = false

            let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: writerInput, sourcePixelBufferAttributes: .none)
            writer.add(writerInput)
            writer.startWriting()
            writer.startSession(atSourceTime: CMSampleBufferGetPresentationTimeStamp(samples[0]))

            for (index, sample) in samples.enumerated() {
                let presentationTime = CMSampleBufferGetPresentationTimeStamp(sample)

                if let imageBufferRef = CMSampleBufferGetImageBuffer(samples[samples.count - index - 1]) {
                    pixelBufferAdaptor.append(imageBufferRef, withPresentationTime: presentationTime)
                }

                while !writerInput.isReadyForMoreMediaData {
                    Thread.sleep(forTimeInterval: 0.1)
                }
            }

            writer.finishWriting { }
            return AVAsset(url: outputURL)
        }
        catch let error as NSError {
            print("\(error)")
            return .none
        }
    }
}

Answer 2

Worked on this over the last few days and was able to get it working.

Source code here: http://www.andyhin.com/post/5/reverse-video-avfoundation

Uses AVAssetReader to read out the samples/frames, extracts the image/pixel buffer, and then appends it with the presentation time of the mirror frame.