CODEWITHSUNDEEP
c++performanceiostoragedisk
Alfatau
Alfatau

Reputation: 243

C++ write options

I'm trying to open a file for writing using std::ofstream and I want to set it in write-through mode (i.e. like using the "FILE_FLAG_WRITE_THROUGH " provided by CreateFile Win API).

Is there some STL way to achieve it? I don't want to write code based on WinAPI. My target is to disable OS caching and perform writes using different block sizes in order to obtain data related to storage performances. I can't use a standard benchmark tool because the target is to understand how to optimize my write-layer settings for the particular storage I have to rely on.

@Update This is an MWE where I would like to see different save times when changing the value of blk_size:

#include <cstdlib>
#include <fstream>
#include <iostream>
#include <vector>
#include <ctime>

std::vector<unsigned char>
GenerateRandomData(long numBytes) 
{
    std::vector<unsigned char> res(numBytes);
    std::srand(std::time(0));

    for (int i = 0; i < numBytes; ++i)
        res[i] = static_cast<unsigned char>(std::rand() % 256);

    return res;
}

int main(int, char)
{
    // generate random data
    const long dataLength = 1 * 1024 * 1024 * 1024; // 3 GB
    std::vector<unsigned char> outbuf = GenerateRandomData(dataLength);

    // define I/O block size (
    const auto blk_size = 128 * 1024; // 128K
    char blk[blk_size];

    // configure output stream
    std::ofstream ofs;
    ofs.rdbuf()->pubsetbuf(blk, blk_size);
    ofs.setf(std::ios_base::unitbuf);

    // open file to write
    ofs.open("output.dat", std::ofstream::binary | std::ofstream::trunc);

    // write all data performing 512K I/O Operations
    auto ptr_idx = 0;
    auto ptr = reinterpret_cast<char*>(outbuf.data());
    const auto outbuf_size = outbuf.size();

    std::clock_t sw = clock();    

    ofs.write((const char *)&ptr[ptr_idx], outbuf_size);

    ofs.flush();
    ofs.close();

    sw = ( clock() - sw );

    double writtenBytes = static_cast<double>(outbuf.size());
    double writtenMBytes = writtenBytes / (1024 * 1024);
    double testSeconds = static_cast<double>(sw) / static_cast<double>(CLOCKS_PER_SEC);
    double avgSpeed = writtenMBytes / testSeconds;

    std::cout << "Benchmark: written " << writtenMBytes << " MB in " << testSeconds << " sec. => " << avgSpeed << "MB/s" << std::endl;  

    std::getchar();

    return 0;
}

Thank you in advance

Upvotes: 4

Views: 447

Answers (1)

shrike
shrike

Reputation: 4511

Use: std::unitbuf

std::ofstream outfile ("file.txt");
outfile << std::unitbuf <<  "Hello\n";  // immediately flushed

Upvotes: 1

Related Questions