CODEWITHSUNDEEP
cmultithreadingfwriteread-write
yuan
yuan

Reputation: 327

What is the correct approach to write multiple small pieces to a temp file in c, in multithreads?

I am simulating multithreads file downloading. My strategy is in each thread would receive small file pieces( each file piece has piece_length and piece_size and start_writing_pos )

And then each thread writes to the same buffer. How do I realize it ? Do I have to worry about collisions ?

//=================== follow up ============//

so I write a small demo as follows: 

#include <stdio.h>

int main(){


 char* tempfilePath = "./testing";
 FILE *fp;
 fp = fopen(tempfilePath,"w+");//w+: for reading and writing 
 fseek( fp, 9, SEEK_SET);//starting in 10-th bytes
 fwrite("----------",sizeof(char), 10, fp);
 fclose(fp);
}

And before execution I let content in "./testing" to be "XXXXXXXXXXXXXXXXXXX", after I do the above I get "^@^@^@^@^@^@^@^@^@----------" I wonder where is the problem then ....

Upvotes: 0

Views: 144

Answers (2)

Andrew Henle
Andrew Henle

Reputation: 1

Unless you want to use a mutex, you can't use fwrite(). FILE *-based IO using fopen(), fwrite(), and all related functions simply isn't reentrant - the FILE uses a SINGLE buffer., a SINGLE offset, etc.

You can't even use open() and lseek()/write() - multiple threads will interfere with each other, modifying the one offset an open file descriptor has.

Use open() to open the file, and use pwrite() to write data to exact offsets.

pwrite() man page:

pwrite() writes up to count bytes from the buffer starting at buf to the file descriptor fd at offset offset. The file offset is not changed.

Upvotes: 0

SzG
SzG

Reputation: 12619

Do what most torrent clients do. Create a file with the final size having an extension .part. Then allocate non-overlapping parts of the file to each thread, who shall have their own file-descriptors. Thus collisions are avoided. Rename to final name when finished.

Upvotes: 3

Related Questions