CODEWITHSUNDEEP
c++
Elnur
Elnur

Reputation: 119

How to improve reading speed from a file?

I am reading from a txt file that contains roughly 1.7 million lines where each line consists of 3 integers. The problem that I am facing is that it takes me approximately 30 sec to loop through and store integers in a vector.

Here is the code that I wrote:

std::ifstream finVertices("Phantom Data/FA_vertices.txt", std::ios::in);
    if (!finVertices)
    {
        std::cerr << "Can not open verticies.txt" << std::endl;
    }

    std::cout << "Loading verticies" << std::endl;

    std::string verticesLineBuffer;
    while (std::getline(finVertices, verticesLineBuffer))
    {
        std::istringstream voxelStringCoordinates(verticesLineBuffer);
        GLfloat x, y, z;
        voxelStringCoordinates >> x >> y >> z;
        vertices.push_back(glm::vec3(y, z, x));
    }

    finVertices.close();

Example of txt file content:

297 13 164
297 13 165
297 14 164
297 14 165
298 13 164
298 13 165

Question: How can I improve reading process from a txt file?

EDIT: Thank you for help. With your help I managed to solve the problem. Here is the code:

std::ifstream is(fileName, std::ifstream::binary);

    if (is) {
        is.seekg(0, is.end);
        int length = is.tellg();
        is.seekg(0, is.beg);

        char* buffer = new char[length];

        is.read(buffer, length);

        is.close();

        for (unsigned int i = 0; i < is.gcount(); i++)
        {
            // here can get access to each indiviual character
        }

Upvotes: 3

Views: 520

Answers (1)

Jesper Juhl
Jesper Juhl

Reputation: 31465

1) read larger chunks at a time. Like 1MiB or 10MiB or the entire file in one go. Then process the data from the memory you read the file (or chunk) into. Or mmap the file.

2) call reserve() on your vertices vector before adding to it to reduce the number of allocations it has to make.

3) compile your code with optimizations enabled (aka release build).

Upvotes: 4

Related Questions