Why (ftruncate+mmap+memcpy) is faster than (write)?

Question

I found a different way to write data, which is faster than normal unix write function.

Firstly, ftruncate the file to the length we need, then mmap this block of file, finally, using memcpy to flush the file content. I will give the example code below.

As I known, mmap can load the file into the process address space, accelerating by ignoring the page cache. BUT, I don't have any idea why it can fast up the writing speed.

Whether I write a wrong test case or it can be a kind of opti trick?

Here is the test code. Some of its written in ObjC, but no matter. WCTTicker is just a statistics class using gettimeofday.

//find a dir to test
NSString* document = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES)[0];
NSString* dir = [document stringByAppendingPathComponent:@"testDir"];

//remove all existing test
if ([[NSFileManager defaultManager] fileExistsAtPath:dir]) {
    if (![[NSFileManager defaultManager] removeItemAtPath:dir error:nil]) {
        NSLog(@"fail to remove dir");
        return;
    }
}
//create dir to test
if (![[NSFileManager defaultManager] createDirectoryAtPath:dir withIntermediateDirectories:YES attributes:nil error:nil]) {
    NSLog(@"fail to create dir");
}

//pre-alloc memory
const int length = 10000000;
const int count = 100;
char* mem = (char*)malloc(length);
memset(mem, 'T', length);

{
    //start testing mmap
    // ftruncate && mmap(private) &&memcpy
    NSString* mmapFileFormat = [dir stringByAppendingPathComponent:@"privateMmapFile%d"];
    [WCTTicker tick];
    for (int i = 0; i < count; i++) {
        NSString* path = [[NSString alloc] initWithFormat:mmapFileFormat, i];
        int fd = open(path.UTF8String, O_CREAT | O_RDWR, S_IRWXG | S_IRWXU | S_IRWXO);
        if (fd<0) {
            NSLog(@"fail to open");
        }
        int rc = ftruncate(fd, length);
        if (rc<0) {
            NSLog(@"fail to truncate");
        }
        char* map = (char*)mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, fd, 0);
        if (!map) {
            NSLog(@"fail to mmap");
        }
        memcpy(map, mem, length);
        close(fd);
    }
    [WCTTicker stop];
}

{
    //start testing write
    // normal write
    NSString* writeFileFormat = [dir stringByAppendingPathComponent:@"writeFile%d"];
    [WCTTicker tick];
    for (int i = 0; i < count; i++) {
        NSString* path = [[NSString alloc] initWithFormat:writeFileFormat, i];
        int fd = open(path.UTF8String, O_CREAT | O_RDWR, S_IRWXG | S_IRWXU | S_IRWXO);
        if (fd<0) {
            NSLog(@"fail to open");
        }
        int written = (int)write(fd, mem, length);
        if (written!=length) {
            NSLog(@"fail to write");
        }
        close(fd);
    }
    [WCTTicker stop];
}

{
    //start testing mmap
    // ftruncate && mmap(shared) &&memcpy
    NSString* mmapFileFormat = [dir stringByAppendingPathComponent:@"sharedMmapFile%d"];
    [WCTTicker tick];
    for (int i = 0; i < count; i++) {
        NSString* path = [[NSString alloc] initWithFormat:mmapFileFormat, i];
        int fd = open(path.UTF8String, O_CREAT | O_RDWR, S_IRWXG | S_IRWXU | S_IRWXO);
        if (fd<0) {
            NSLog(@"fail to open");
        }
        int rc = ftruncate(fd, length);
        if (rc<0) {
            NSLog(@"fail to truncate");
        }
        char* map = (char*)mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_SHARED, fd, 0);
        if (!map) {
            NSLog(@"fail to mmap");
        }
        memcpy(map, mem, length);
        close(fd);
    }
    [WCTTicker stop];
}

Here is the test result:

2016-07-05 11:44:08.425 TestCaseiOS[4092:1070240] 
0: 1467690246.689788, info: (null)
1: 1467690248.419790, cost 1.730002, info: (null)
2016-07-05 11:44:14.126 TestCaseiOS[4092:1070240] 
0: 1467690248.427097, info: (null)
1: 1467690254.126590, cost 5.699493, info: (null)
2016-07-05 11:44:14.814 TestCaseiOS[4092:1070240] 
0: 1467690254.126812, info: (null)
1: 1467690254.813698, cost 0.686886, info: (null)

Answer 1

Even with the munmap (or msync) added, I think this should be faster at least for big data transfers because write() results in a copy operation while mmap and access to the map do not.

Answer 2

You have mmap() without corresponding munmap()

From mmap manual page (linux)

MAP_SHARED Share this mapping. Updates to the mapping are visible to other processes that map this file, and are carried through to the underlying file. The file may not actually be updated until msync(2) or munmap() is called.

Perhaps you should run your tests again so that there is a call to munmap:

char* map = (char*)mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_SHARED, fd, 0);
if (!map) {
   NSLog(@"fail to mmap");
}
memcpy(map, mem, length);
munmap(map, length);
close(fd);