CODEWITHSUNDEEP
c++vectorfstreamc++14
RamblingMad
RamblingMad

Reputation: 5518

Most efficient way to read an entire file into vector of lines

I am trying to find a more efficient way to read an entire file into a vector of lines, defined as std::vector<std::string>.

Currently, I have written the naive:

std::ifstream file{filepath};
std::vector<std::string> lines;

std::string line;
while(std::getline(file, line)) lines.push_back(line);

But feel as though the extra copy in push_back and the vector reallocation for every line would be extremely detrimental to efficiency and am looking for a more modern c++ type of approach, such as using stream buffer iterators when copying bytes:

std::ifstream file{filepath};
auto filesize = /* get file size */;
std::vector<char> bytes;
bytes.reserve(filesize);
bytes.assign(std::istreambuf_iterator{file}, istreambuf_iterator{});

Is there any such way I could read a text file in by lines into a vector?

Upvotes: 2

Views: 1439

Answers (2)

R Sahu
R Sahu

Reputation: 206607

Something along the following code might work.

struct FileContents
{
   // Contents of the file.
   std::string contents;

   // Each line consists of two indices.
   struct line { std::string::size_type begin; std::string::size_type end;};

   // List of lines.
   std::vector<line> lines;
};

void readFileContents(std::string const& file,
                      FileContents& fileContents)
{
   std::ifstream ifs(file);
   if ( !ifs )
   {
      // Deal with error.
      return;
   }

   // Get the size of the file.
   ifs.seekg(0, std::ios::end);
   std::fstream::pos_type size = ifs.tellg();

   // Reserve space to read the file contents.
   fileContents.contents.assign(static_cast<std::string::size_type>(size), '\0');

   // Read the file contents.
   ifs.seekg(0, std::ios::beg);
   ifs.read(&fileContents.contents[0], size);
   ifs.close();

   // Divide the contents of the file into lines.
   FileContents::line line = {0, 0};
   std::string::size_type pos;
   while ( (pos = fileContents.contents.find('\n', line.begin)) != std::string::npos )
   {
      line.end = pos+1;
      fileContents.lines.push_back(line);
      line.begin = line.end;
   }
   line.end = fileContents.contents.size();
   if ( line.begin < fileContents.contents.size() )
   {
      fileContents.lines.push_back(line);
   }
}

Upvotes: 0

SergV
SergV

Reputation: 1277

There is very interesting and relative new approach - ranges. You can read very interesting articles by Eric Niebler:

Out Parameters, Move Semantics, and Stateful Algorithms about fast getlines alternative

Input Iterators vs Input Ranges

Upvotes: 0

Related Questions