Out-Of-Memory crash on iOS, but memory consumption is at 20MB

Question

I am experiencing out-of-memory crashed with my app on iOS on both iPhone 6 and 5s. I am sure that memory is the problem, because I receive memory warnings and also because of the crash log.

In the app I do video processing and process around 400 images in 10 seconds for one post-processing step. Each frame's resources get properly released in an autorelease pool. The crashes come after a random amount of repetition.

I allocate a lot of memory during those 10 seconds (a few hundred MB) in total, but the memory gets released after each processed image, due to the custom autorelease pool.

However, when I try to profile the root of the problem, I can not even detect high memory consumption, let alone search for the source of the problem. I have analyzed the used physical memory in the activity monitor in Instruments and neither my app nor mediaserverd consume more than 30 MB at any given moment.

Why do I get memory crashes, when my app is only using 30MB of memory at peak? How can I measure and debug the memory consumption correctly?

Any pointers a greatly appreciated!

Here is a screenshot of the memory report at runtime:

Here is the crash log:

Incident Identifier: 2B5E9345-964D-4F6E-87D2-5517F0C22531
CrashReporter Key:   2033c0b344ea7cbf4f153cd862d1e7b68969b42e
Hardware Model:      iPhone6,2
OS Version:          iPhone OS 8.1.1 (12B435)
Kernel Version:      Darwin Kernel Version 14.0.0: Mon Nov  3 22:23:57 PST 2014; root:xnu-2783.3.22~1/RELEASE_ARM64_S5L8960X
Date:                2015-01-26 12:04:35 +0100
Time since snapshot: 672 ms

Free pages:                              2515
Active pages:                            23951
Inactive pages:                          11816
Speculative pages:                       643
Throttled pages:                         0
Purgeable pages:                         0
Wired pages:                             133367
File-backed pages:                       10813
Anonymous pages:                         25597
Compressions:                            8823352
Decompressions:                          752051
Compressor Size:                         83913
Uncompressed Pages in Compressor:        155021
Page Size:                               16384
Largest process:   RedGlam

Processes
Name       |            <UUID>                |     CPU Time|     rpages|       purgeable| recent_max| lifetime_max| fds |  [reason]         | (state)

timed <6fa98ab7f5de312b9bfed47e04e3a43e>         0.075         240                0          +5           701   50   [vm-pageshortage]   (daemon) (idle)
calaccessd <0a7ad7bbfb523bfdbae43aa6f21279f6>         0.134         462                0         +47          1279   50   [vm-pageshortage]   (daemon) (idle)
MobileGestaltHel <19968f31a89230a6b66e51ad668f0921>         0.028         163                0           -           522   50   [vm-pageshortage]   (daemon) (idle)
awdd <58036e1703903ee798a8803de204c300>         0.123         405                0           -          1076   50   [vm-pageshortage]   (daemon) (idle)
WirelessRadioMan <c4181e6d863133e8aa0c95e77a7bb206>         0.051         293                0           -           787   50   [vm-pageshortage]   (daemon) (idle)
notification_pro <b143453e80393938a7ba23a0181dc52c>         0.158         148                0           -           451   50   [vm-pageshortage]   (daemon)
com.apple.dt.ins <c2263ead004b339f9fe48bbdf95286c9>        30.091         255                0           -           787   50   [vm-pageshortage]   (daemon)
debugserver <f00a5e2ac8f73e7e893c711d88c344a6>        24.790         208                0           -           681   50   [vm-pageshortage]   (daemon)
MobileMail <4b48abd990e93dbea47db1cbf328da9e>         6.030        1518                0           -          4299   50                       (resume) (continuous)
lsd <f554bd07b90a3cfc9d9ef9f8e234833c>         1.443         333                0           -           859   50                       (daemon)
tccd <f2878273872231afa1a6e0af2dcb73a6>         0.619         278                0           -           861   50                       (daemon)
MYAPP <417ba797cf3e39df82d79baf41050692>       389.444      105870                0           -         13874   50                       (audio) (frontmost) (resume)
ptpd <a06176d3eefe3e3c8549bb4f6d340658>         1.913         817                0           -          2043   50                       (daemon)
BTServer <2d0fc0974c073a0aafc9954110080950>        16.246         572                0           -          1859   50                       (daemon)
wifid <43e56e539a6a3114bf4cd7646c8dd90e>       149.958         561                0           -          1564   50                       (daemon)
discoveryd <68f73878299336d7872b0ae9ce3f7f08>       179.311         585                0           -          1220  100                       (daemon)
locationd <5b826e2c09c23eaaa2acc2472269cb30>      4461.461        2209                0           -          5025   50                       (daemon)
lockdownd <3a0b3375ad6e391da37a1f79f46843b0>        39.278         364                0           -          1345   50                       (daemon)
syslogd <05f6b5e5512938a892bac5af23ab1c08>       127.902         240                0           -           818   50                       (daemon)
powerd <2b4ae8758a5b3b709a97c452ec08923b>        56.978         310                0           -           579   50                       (daemon)
imagent <d5e037ad2173362d8a6077788b2d7074>        13.323         529                0           -          1354   50                       (daemon)
identityservices <9d4b00e3c6003685ac8697c59f4e4d38>        16.729         687                0           -          1794   50                       (daemon)
vmd <c3b3270d187f3dcaa843ba73f01ff8cb>         0.482         595                0           -          2598   50                       (daemon)
cfprefsd <4325eab208063b998046460a4c2ee484>        32.394         449                0           -           742   50                       (daemon)
iaptransportd <4bf77076d69630e389ba64229c526723>        20.807         364                0           -           971   50                       (daemon)
apsd <bb925404cb1137b09b85671a8d2c7656>        62.713         738                0           -          1892   50                       (daemon)
networkd <fa2acedf0b0035269d66a72e28c3a95a>       307.451         716                0           -          1585   50                       (daemon)
sharingd <1ed17c64831f32ea9cbb47e48c4d222c>        29.442         944                0           -          2298   50                       (daemon)
dataaccessd <33bcaea3bc473f128685f4df14a115eb>        13.535         729                0           -          2230   50                       (daemon)
SCHelper <779250b8a48638958a5922f6ae1066fa>        10.160         134                0           -           324   50                       (daemon)
searchd <e5c5e5675c0935eaab5feb15ebc0b934>         5.392         888                0           -          2986   50                       (daemon)
mediaserverd <a0354e528bc431958df0d50830bead36>      1080.882       91747                0           -         21944  200                       (daemon)
syslog_relay <087c67d0371b324fb6df48442016ec90>         6.746         114                0           -           237   50                       (daemon)
SpringBoard <96f929dd23123d8bbc9ba2a0bb48bde1>      1108.457        8031                0           -         17537   50
backboardd <e263837653b434f1880f9d37b3926998>      7219.545        7865                0           -          4676   50                       (daemon)
UserEventAgent <f5a211b9c88e3fa481f2bd1ee1f5a921>      1084.316        1000                0           -          2811  100                       (daemon)
fseventsd <16c9b62bb28c388ca10d54dbff18c4f8>        20.426         496                0           -           843   50                       (daemon)
configd <ed40fcde35ae337ab3b70073199564b1>       117.075         537                0           -          1463   50                       (daemon)
fairplayd.H2 <cae337642f6d396b82ac54e72bc0e0a4>         6.550         151                0           -          1433   50                       (daemon)
wirelessproxd <ab1fa7e43a7c3f9393533404c2cc80b8>         1.814         264                0           -           986   50                       (daemon)
assertiond <10ec04add18f3ecd8a8efbb1cc4e2bd6>        18.292         325                0           -          1045   50                       (daemon)
aggregated <281958649a3130aab6ecb1aa47f0a6c1>      2273.629        1367                0           -          2822   50                       (daemon)
distnoted <cb5e76091dc53ceeaf65290f8e197a89>         4.937         207                0           -           335   50                       (daemon)
discoveryd_helpe <492c39ae2d643adca0ed971675c77406>         0.120         156                0           -           752   50                       (daemon)
filecoordination <5ec159db1afe3317878b8ab794e2d7d1>         1.259         308                0           -           941   50                       (daemon)
DTMobileIS <2fdc94aa5069338e815fbe3a13e3d95c>       551.592        2538                0           -         36844   50                       (daemon)
gputoolsd <97e54c888a9a3978a85fd965a71e7669>         9.503        1031                0           -          2910   50                       (daemon)
CommCenter <33412ab229c738c8860c70803fed173b>       787.039        1465                0           -          4737   50                       (daemon)
notifyd <5fa8fd5e44c83f64be1475b882b16c82>       142.939         384                0           -           441   50                       (daemon)
ReportCrash <e946799f25f833fd9b37a6a1c7b1993c>         0.039         166                0           -           551   50                       (daemon)

**End**

Here is the code for one of the effects I am using (renderAnimation2DOverFace does not allocate any memory):

- (CIImage *) render:(CIImage*)targetImage imageContext:(CIContext*) imageContext
      facialFeatures:(NSArray*)facialFeatures currentFrameInd:(int)frameInd
{
    if ( !facialFeatures ) return targetImage;

    @autoreleasepool {
        UIImage *editingImage = [[UIImage alloc] initWithCIImage:targetImage];
        UIGraphicsBeginImageContextWithOptions(editingImage.size, NO, 1.0);

        [editingImage drawInRect:CGRectMake( 0, 0, editingImage.size.width, editingImage.size.height)];

        for ( ArtechFacialFeature *facialFeature in facialFeatures ) {
            [self renderAnimation2DOverFace:facialFeature.bounds
                                            currentFrameInd:frameInd];
        }

        UIImage *resultImage = UIGraphicsGetImageFromCurrentImageContext();
        UIGraphicsEndImageContext();

        targetImage = [targetImage initWithCGImage:resultImage.CGImage options:nil];

        editingImage = nil;
        resultImage = nil;
    }

    return targetImage;
}

This is another video effect routine:

- (CIImage *) render:(CIImage*)targetImage imageContext:(CIContext*) imageContext
    facialFeatures:(NSArray*)facialFeatures currentFrameInd:(int)frameInd
{
    if ( !facialFeatures ) return targetImage;

    @autoreleasepool {
        CGImageRef inputImage = [imageContext createCGImage:targetImage fromRect:[targetImage extent]];

        GPUImagePicture *sourcePicture = [[GPUImagePicture alloc] initWithCGImage:inputImage];
        GPUImageOutput *currentOutput = [self createFilterChain:sourcePicture facialFeatures:facialFeatures
            imageExtent:targetImage.extent];

        [currentOutput useNextFrameForImageCapture];
        [sourcePicture processImage];

        UIImage *currentFilteredVideoFrame = [currentOutput imageFromCurrentFramebuffer];

        targetImage = [targetImage initWithCGImage:currentFilteredVideoFrame.CGImage];

        currentFilteredVideoFrame = nil;

        [sourcePicture removeAllTargets];
        sourcePicture = nil;

        [currentOutput removeOutputFramebuffer];
        currentOutput = nil;

        CFRelease( inputImage );

        [[GPUImageContext sharedFramebufferCache] purgeAllUnassignedFramebuffers];
    }

    return targetImage;
}

Answer 1

The problem was that there was a CIImage which was de-referenced outside of the autoreleasepool and outside of the render methods, which I have posted in the question.

It is the targetImage parameter that is being passed as an argument. So now I added another autorelease pool outside of the method call and that fixed all my memory problems.

It still is a mystery to me though why those unreleased allocations of the CIImage did not manifest in the amount of memory used in the memory monitor and other tools.

Also, I do not understand, why the crashed would only happen occasionally.

Answer 2

When in a loop doing processing that uses autoreleased memory you need to put an autorelease pool inside the loop. Note that many Cocoa and Foundation methods use auto released memory.

The problem is that each operation is retaining and auto-releasing memory but the memory is not actually released until the current autoreleasepool is drained, usually in the run loop and that is not being called because of the tight loop.

Example:

for (...) {
    @autoreleasepool {
        perform work
    }
}

You can put an autoreleasepool around smaller parts of the operations.

To debug reduce the loop count so you don't crash and then use Heapshot:

Use instruments to check for leaks and memory loss due to retained but not leaked memory. The latter is unused memory that is still pointed to. Use Mark Generation (Heapshot) in the Allocations instrument on Instruments.

For HowTo use Heapshot to find memory creap, see: bbum blog

Basically the method is to run Instruments allocate tool, take a heapshot, run an iteration of your code and take another heapshot repeating 3 or 4 times. This will indicate memory that is allocated and not released during the iterations.

To figure out the results disclose to see the individual allocations.

If you need to see where retains, releases and autoreleases occur for an object use instruments:

Run in instruments, in Allocations set "Record reference counts" on (For Xcode 5 and lower you have to stop recording to set the option). Cause the app to run, stop recording, drill down and you will be able to see where all retains, releases and autoreleases occurred.