CODEWITHSUNDEEP
iosavfoundationavassetavassetreader
jlw
jlw

Reputation: 3216

AVAssetReaderOutput copyNextSampleBuffer hangs when encoding video w/ audio on device

After implementing the solution to encoding video (with audio) in this question, Video Encoding using AVAssetWriter - CRASHES, I found that the code works correctly in the iPhone Simulator. Unfortunately, certain videos fail to encode their audio while running on an actual iPhone 5 (and other devices).

For example, videos generated from the WWDC 2011 sample code RosyWriter (https://developer.apple.com/library/IOS/samplecode/RosyWriter/Introduction/Intro.html) do not completely encode because the function -[AVAssetReaderOutput copyNextSampleBuffer] never returns.

The video buffers come in correctly, but as soon as it tries to copy the first audio CMSampleBufferRef, the call hangs. When I try this on videos that come from other sources, like those recorded in the native iOS Camera app, the audio imports correctly.

This thread, https://groups.google.com/forum/#!topic/coreaudio-api/F4cqCu99nUI, makes note of the copyNextSampleBuffer function hanging when being used with AudioQueues, and suggests keeping the operations on a single thread. I've tried keeping everything on a separate thread, on the main thread, but had no luck.

Did anyone else experience this and have a possible solution?

EDIT: It appears that videos generated from RosyWriter have their tracks reversed relative to videos from the native Camera app, i.e. audio stream as stream 0, and video stream as stream 1.

Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 60 kb/s
Metadata:
  creation_time   : 2013-10-28 16:13:05
  handler_name    : Core Media Data Handler
Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080, 8716 kb/s, 28.99 fps, 29.97 tbr, 600 tbn, 1200 tbc
Metadata:
  rotate          : 90
  creation_time   : 2013-10-28 16:13:05
  handler_name    : Core Media Data Handler

Not sure if this makes a difference to the AVAssetReader.

Upvotes: 10

Views: 4373

Answers (4)

Dan G
Dan G

Reputation: 232

I was still experiencing this issue on iOS 9.3.2, and the thing that resolved it was making sure that the AVAssetReaderAudioMixOutput* was initially set with options rather than nil when calling -[AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks].

Example:

NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                    [NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey,
                                    [NSNumber numberWithFloat:44100.0], AVSampleRateKey,
                                    [NSNumber numberWithInt:16], AVLinearPCMBitDepthKey,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsFloatKey,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsBigEndianKey,
                                    nil];

// create an AVAssetReaderOutput for the audio tracks
NSArray* audioTracks = [asset tracksWithMediaType:AVMediaTypeAudio];
AVAssetReaderAudioMixOutput* _audioReaderOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:audioTracks audioSettings:outputSettings];

This prevented later calls to -[AVAssetReaderOutput copyNextSampleBuffer] from hanging when they otherwise were doing so.

Upvotes: 4

etayluz
etayluz

Reputation: 16416

This is a a hardware specific issue. I tested it on the simulator, iPhone5, and iPhone 6. The problem occurs 100% with certain video files, but ONLY on the iPhone 5. I can't speak for the iPhone 5S though. iOS8 does NOT fix this issue. The fix may be having to limit your iPhone 5 users from certain features. Another work around is to have the user select a video from their camera roll, thus forcing Apple's special compression on it - and baptizing that video into a format that the iPhone 5 hardware can work with.

Upvotes: -1

jlw
jlw

Reputation: 3216

This was an AVFoundation bug in iOS7. It is now fixed in iOS8.

Upvotes: -1

John Lemberger
John Lemberger

Reputation: 2699

Check the time range on the audio tracks using something like this:

NSLog(@"audioTrack timeRange: %lld, %lld", audioTrack.timeRange.start.value, audioTrack.timeRange.duration.value);

Empty time ranges (0, 0) can cause copyNextSampleBuffer() to hang.

Upvotes: 0

Related Questions