Reading (text) files in blocks

Question

I am currently all into I/O operations in c++ and I came across the method of reading and/or writing data in blocks of certain sizes (like 1024Bytes) rather than the whole file at once. Now there are a few questions to this concept in my mind:

1. What is the benefit of this? Is it only because there is a limit on how many bytes can be read at once? Or does this give a significant speed-up? Is it maybe only used for socket connections, or for files where the size is not known up-front?
2. If this is useful for all kind of file types (like a text file with config values in it), how can I process a block correctly? See my example below:

Imagine a config file which is somewhat structured like this:

[Engine]
bloom = true
AA = 16
[Keys]
jump = SPACE
quit = ESCAPE

Now one block I read contains this:

[Engine]
bloom = true
AA = 16
[Keys]
jump = SP

So my block is not large enough to store the last line completely. If I now use this block with my config reader class it will detect this line, but with a wrong value. How can I make sure that this is handled correctly? Maybe I am on a totally wrong path here, but I would appreciate some clarification.

Answer 1

You posit that this is some kind of choice you can make, but it's not: file data is always read in chunks. The only question is at what layer of abstraction are the chunks concatenated into a single output, for consumption by your business logic? Is it in your "user-space" code or hidden away behind standard library functions? As such, most of your question is moot.

Still, if you can keep that logic hidden away by using standard library functions, then it saves you a little bit of handholding code.

Compare this:

#include <string>
#include <fstream>

std::string readFromStream(std::istream& is)
{
   std::string result;

   char x;
   while (is.get(x))
      str += x;

   return result;
}

std::ifstream t("file.txt");
std::string str = readFromStream(t);

(or an equivalent that reads blocks of, say, 1024 bytes — the logic will be largely the same), with this:

#include <string>
#include <fstream>
#include <streambuf>

std::ifstream t("file.txt");
std::string str((std::istreambuf_iterator<char>(t)),
                 std::istreambuf_iterator<char>());