CODEWITHSUNDEEP
mpi
frankliuao
frankliuao

Reputation: 420

Write to a file in parallel using MPI or in serial?

I have a general question about MPI usage.

When you want to write data to files, for example, 100 files are gonna be created.

Do you prefer to use each rank (core) to write one file, or you prefer to write them serially, one by one? Which is faster?

Upvotes: 1

Views: 314

Answers (1)

C.B.
C.B.

Reputation: 8326

If you are having each rank write to one file (serially), there will exist a race condition, so all writing must be done atomically. This will give much slower overall write time than having each rank write to its own file.

However you would then need some sort of reduce where the reduce joins two separate files.You would have to weight the communication / joining overhead against the wait time for writing serially (unless you will leave the files in parts).

Upvotes: 1

Related Questions