CODEWITHSUNDEEP
c++windowsfilememoryio
intrigued_66
intrigued_66

Reputation: 17278

Faster way to read file than boost::file_mapping?

I'm writing a latency-sensitive app which reads a text file upon initialisation. I have profiled and re-written all my algorithms such that 85% of my execution time is from the lines:

boost::interprocess::file_mapping file(Path, read_only);
boost::interprocess::mapped_region data(file, read_only);

I am writing this on windows- is there any faster way to map a file into memory? Portability is not a concern.

Upvotes: 1

Views: 1306

Answers (3)

Bukes
Bukes

Reputation: 3718

If the files are small, just open and read them into memory using standard Win32 CreateFile()/ReadFile() APIs.

If you're consuming each file sequentially (or could arrange your code in such a way that you do), you should specify FILE_FLAG_SEQUENTIAL_SCAN. This is a hint for the file/caching subsystem to read-ahead aggressively. For small files, the file might be read into cache before your first call to ReadFile() is issued.

Edit: As requested, Here's a snippet that illustrates reading the contents of a file into a vector of bytes using the Win32 API:

void ReadFileIntoBuffer( const std::wstring& fileName, std::vector< uint8_t >& output )
{
    HANDLE hFile( INVALID_HANDLE_VALUE );
    try 
    {
        // Open the file.
        hFile = CreateFile( filename.c_str(), 
                                 GENERIC_READ,
                                 FILE_SHARE_READ,
                                 NULL,
                                 OPEN_EXISTING, 
                                 FILE_FLAG_SEQUENTIAL_SCAN,
                                 NULL );
        if( INVALID_HANDLE_VALUE != hFile )
            throw std::runtime_error( "Failed to open file." );

        // Fetch size
        LARGE_INTEGER fileSize;
        if( !GetFileSizeEx( hFile, &fileSize ) );
            throw std::runtime_error( "GetFileSizeEx() failed." );

        // Resize output buffer.
        output.resize( fileSize.LowPart );

        // Read the file contents.
        ULONG bytesRead;
        if( !ReadFile( hFile, &output[0], fileSize.LowPart, &bytesRead, NULL ) )
            throw std::runtime_error( "ReadFile() failed." );

        // Recover resources.
        CloseHandle( hFile );
    }
    catch( std::exception& ) 
    {
        // Dump the error.
        std::cout << e.what() << " GetLastError() = " << GetLastError() << std::endl;

        // Recover resources.
        if( INVALID_HANDLE_VALUE != hFile )
            CloseHandle( hFile );

        throw;
    }
}

Upvotes: 1

Pavel Radzivilovsky
Pavel Radzivilovsky

Reputation: 19104

I would suggest dropping the idea of file mapping.

FM is a complicated construct and adds some overhead. Plain cached read also involves non-trivial interaction with the physical device. You could do unbuffered reads. Probably the next thing to ask is what kind of IO you actually want - how big is the file? is it sequential? Is it on the network? Do you have a choice of hardware, or is it on the customers' machine?

Upvotes: 1

pbhd
pbhd

Reputation: 4467

You could just use the native functions of Win32, but I think you won't save alot, because boost will not add alot of overhead:

OFSTRUCT ofStruct;
ofStruct.cBytes=sizeof (OFSTRUCT);
HANDLE file=(HANDLE)OpenFile(fileName, &ofStruct, OF_READ); 
if (file==INVALID_HANDLE_VALUE) 
  handle errors
else {
  HANDLE map=CreateFileMapping(file, NULL, PAGE_READONLY, 0, 0, 0);
  if (map==INVALID_HANDLE_VALUE) 
    handle errors
  else {
    const char *p=(const char *)MapViewOfFile(map, FILE_MAP_READ, 0, 0, 0));
    if (p) {
      // enjoy using p to read access file contents.
    }
    // close all that handles now...
  }

Upvotes: 3

Related Questions