CODEWITHSUNDEEP
cfileiobinary
papanoel87
papanoel87

Reputation: 151

Write large binary file in C

I'm using 64bit mingw to compile c code on windows x64. I'm using fwrite to create binary files from memory array. I want to write ~20Gb calling this function but it just write until 1.4~1.5gb and then it stops writting (without crashing, just hangs there.... doing nothing). Is there any solution? Right now I'm writing 20 files and then I merge them. Opening the file as 'ab' works but I cant read the file properly if I use that mode.

Sample (pseudo)code:

    short* dst= malloc(20GB);
    *calculations to fill dst* 
    file=fopen("myfile",'wb');
    fwrite(dst, sizeof(short), 20GB/sizeof(short), file);
    fclose(file)

That program never ends and file size is never grater than 1.5GB

Upvotes: 3

Views: 1591

Answers (3)

schily
schily

Reputation: 317

Mingw is a 32 bit environment, there AFAIK does not exist a 64 bit variant.

It may be that fwrite() from mingw is unable to deal with more than 2 GB or 4GB unless mingw is large file aware.

If you can find something similar to truss(1), run your progran under this debugging tool. With the information you provided, it is not possible to give a better advise.

Upvotes: -1

Mike Dunlavey
Mike Dunlavey

Reputation: 40709

Write it in smaller chunks. For heaven's sake, don't try to malloc 20gb.

Upvotes: 6

wallyk
wallyk

Reputation: 57804

Depending on the environment (operating system, memory model, file system), it might not be possible to create a file greater than 2 GB. This is especially true with MSDOS file systems and of course could be true on any file system if there is insufficient disk space or allocation quota.

If you show your code, we could see if there is any intrinsic flaw in the algorithm and suggest alternatives.

Upvotes: 1

Related Questions