CODEWITHSUNDEEP
c++file
jndok
jndok

Reputation: 949

How to read a file in multiple chunks until EOF (C++)

So, here's my problem: I want to make a program that reads chunks of data from a file. Let's say, 1024 bytes per chunk. So I read the first 1024 bytes, perform various operations and then open the next 1024 bytes, without reading the old data. The program should keep reading data untile the EOF is reached.

I'm currently using this code:

std::fstream fin("C:\\file.txt");

vector<char> buffer (1024,0); //reads only the first 1024 bytes
fin.read(&buffer[0], buffer.size());

But how can I read the next 1024 bytes? I was thinking by using a for loop, but I don't really know how. I'm totally a noob in C++, so if anyone can help me out, that would be great. Thanks!

Upvotes: 15

Views: 34819

Answers (4)

freezing_
freezing_

Reputation: 1044

I think that will work

     #include <stdlib.h>
     #include <stdio.h>
     #include <string.h>
     #include <fstream>
    
    // Buffer size 16 Megabyte (or any number you like)
    size_t buffer_size = 1 << 24; // 20 is 1 Megabyte
    char* buffer = new char[buffer_size];

    std::streampos fsize = 0;
    std::ifstream file("c:\\file.bin", std::ios::binary);

    fsize = file.tellg();
    file.seekg(0, std::ios::end);
    fsize = file.tellg() - fsize;

    int loops = fsize / buffer_size;
    int lastChunk = fsize % buffer_size;

    for (int i = 0; i < loops; i++) {
        file.read(buffer, buffer_size);
        // DO what needs with the buffer
    }

    if (lastChunk > 0) {
        file.read(buffer, lastChunk);
        // DO what needs with the buffer
    }

    delete[] buffer;

Upvotes: 0

111111
111111

Reputation: 16168

You can do this with a loop:

std::ifstream fin("C:\\file.txt", std::ifstream::binary);
std::vector<char> buffer (1024,0); //reads only the first 1024 bytes

while(!fin.eof()) {
    fin.read(buffer.data(), buffer.size())
    std::streamsize s=fin.gcount();
    ///do with buffer
}

##EDITED

http://en.cppreference.com/w/cpp/io/basic_istream/read

Upvotes: 23

MateuszL
MateuszL

Reputation: 2993

Accepted answer doesn't work for me - it doesn't read last partial chunk. This does:

void readFile(std::istream &input, UncompressedHandler &handler) {
    std::vector<char> buffer (1024,0); //reads only 1024 bytes at a time
    while (!input.eof()) {
        input.read(buffer.data(), buffer.size());
        std::streamsize dataSize = input.gcount();
        handler({buffer.begin(), buffer.begin() + dataSize});
    }
}

Here UncompressedHandler accepts std::string, so I use constructor from two iterators.

Upvotes: 11

Mostafa 36a2
Mostafa 36a2

Reputation: 177

I think you missed up that there is a pointer points to the last place you've visit in the file , so that when you read for the second time you will not start from the first , but from the last point you've visit . Have a look to this code

std::ifstream fin("C:\\file.txt");
char buffer[1024]; //I prefer array more than vector for such implementation

fin.read(buffer,sizeof(buffer));//first read get the first 1024 byte

fin.read(buffer,sizeof(buffer));//second read get the second 1024 byte

so that how you may think about this concept .

Upvotes: 2

Related Questions