CODEWITHSUNDEEP
iphoneiosjsonipadstreaming
Wojczitsu
Wojczitsu

Reputation: 260

iOS Huge JSON (30MB) handling

I need to download and parse 40mb of json. Right now I'm using AFJSONRequestOperation, which on older devices causes memory warning and crash. How it should be done? I think the only way to do it correctly, is to stream json but I've got no idea how to do it or which library will be best. Please provide examples. Thanks a lot!

Upvotes: 3

Views: 2213

Answers (2)

Wojczitsu
Wojczitsu

Reputation: 260

For anybody who has the same problem, here's how I solved it: 1. Download JSON file to local storage using AFHTTPRequestOperation's output stream. 2. Parse little chunks of NSData using YAJLParser.

Result: I was testing it on 50mb json on iPad (1), without any memory warnings (memory around 10mb).

Example:

            NSError *error = nil;
         NSData *data = [NSData dataWithContentsOfFile:path
                                               options:NSDataReadingMappedAlways | NSDataReadingUncached
                                                 error:&error];
         YAJLParser *parser = [[YAJLParser alloc] initWithParserOptions:YAJLParserOptionsAllowComments];
         parser.delegate = self;
         [parser parse:data];
         parser.delegate = nil;
         parser = nil;

YAJLParser delegate:

// first declare in header file NSMutableArray *stack and NSString *mapKey
- (void)parserDidStartDictionary:(YAJLParser *)parser
{
    NSString *dictName = mapKey;
    if (mapKey == nil)
    {
        dictName = (stack.count == 0) ? @"" : [stack lastObject];
    }
    [stack addObject:(dictName)];
}

- (void)parserDidEndDictionary:(YAJLParser *)parser
{
    mapKey = nil;
    [stack removeLastObject];
}

- (void)parserDidStartArray:(YAJLParser *)parser
{
    NSString *arrayName = mapKey;
    if (mapKey == nil)
    {
        arrayName = stack.count == 0 ? @"" : [stack lastObject];
    }
    [stack addObject:(arrayName)];

    if([mapKey isEqualToString:@"something"])
    {
        // do something
    }
}

- (void)parserDidEndArray:(YAJLParser *)parser
{
    if([mapKey isEqualToString:@"some1"])
    {
        // do something
    }

    mapKey = nil;
    [stack removeLastObject];
}

- (void)parser:(YAJLParser *)parser didMapKey:(NSString *)key
{
    mapKey = key;
}

- (void)parser:(YAJLParser *)parser didAdd:(id)value
{
    if([mapKey isEqualToString:@"id"])
    {
        // do something    
    }
}

Upvotes: 4

Hai Hw
Hai Hw

Reputation: 1447

Write your data to a file, then use NSData's dataWithContentsOfFile:options:error: and specify the NSDataReadingMappedAlways and NSDataReadingUncached flags. This will tell the system to use mmap() to reduce the memory footprint, and not to burden the file system cache with blocks of memory (that makes it slower, but much less of a burden to iOS).

You can find the answer here iPad - Parsing an extremely huge json - File (between 50 and 100 mb)

Upvotes: 2

Related Questions