CODEWITHSUNDEEP
mpi
syko
syko

Reputation: 3637

MPI one-sided file I/O

I have some questions on performing File I/Os using MPI.

A set of files are distributed across different processes. I want the processes to read the files in the other processes.

For example, in one-sided communication, each process sets a window visible to other processors. I need the exactly same functionality. (Create 'windows' for all files and share them so that any process can read any file from any offset)

Is it possible in MPI? I read lots of documentations about MPI, but couldn't find the exact one.

Upvotes: 1

Views: 178

Answers (1)

Gilles
Gilles

Reputation: 9519

The simple answer is that you can't do that automatically with MPI.

You can convince yourself by seeing that MPI_File_open() is a collective call taking an intra-communicator as first argument and returning a file handler to the opened file as last argument. In this communicator, all processes open the file and therefore, all processes must see the file. So unless a process sees a file, it cannot get a MPI_file handler to access it.

Now, that doesn't mean there's no solution. A possibility could be to do by hand exactly what you described, namely:

  1. Each MPI process opens individually the file they see and are responsible of; then
  2. Each of theses processes reads this local file into a buffer;
  3. Theses individual buffers are all exposed, using either a global MPI_Win memory windows, or several individual ones, ready for one-sided read accesses; and finally
  4. All read accesses to any data that were previously stored in these individual local files, are now done through MPI_Get() calls using the memory window(s).

The true limitation of this approach is that it requires to fully read all of the individual files, therefore, you need to have sufficient memory per node for storing each of them. I'm well aware that this is a very very big caveat that could just make the solution completely impractical. However, if the memory is sufficient, this is an easy approach.

Another even simpler solution would be to store the files into a shared file system, or having them all copied on all local file systems. I imagine this isn't an option since the question wouldn't have been asked otherwise...

Finally, in last resort, a possibility I see would be to dedicate a MPI process (or an OpenMP thread of a MPI process) per node to serve each files. This process would just act as a "file server", answering "read" request coming from the other MPI processes, and serving them by reading the requested data from the file, and sending it back via MPI. It's a bit lengthy to write, but it should work.

Upvotes: 2

Related Questions