CODEWITHSUNDEEP
c#linqreadlines
ykh
ykh

Reputation: 1825

Fast Reading File With Indices

I am trying to read a huge file. Previously i used File.ReadLines which will not read the entire data into memory. Now i need to access the previous & next line while iterating. I tried reading the entire file but got an OutofMemoryException.

I then started searching for a solution and found that i can use Linq. I tried using the following:

    var inputLines = File.ReadLines(filePath);
    int count = inputLines.Count();
    for (int i = 0; i < count; i++)
    {
        string line = inputLines.Skip(i).Take(1).First();
    }

This is very slow. I think each time it Skip and then Take, it keeps reading all the lines over and over again at each loop.

Is there a better way to do it ? The only solution i could think of right now is to split my file into smaller chunks so i can read it but i can't really believe that there is no workaround for this.

Upvotes: 0

Views: 42

Answers (1)

Mel Green
Mel Green

Reputation: 3801

Are you iterating through all the lines in the file and want to compare three lines (previous, current, and next) each time you look at a new line?

If so, how about just keep track of the last two lines as you iterate through all of them:

    string previousLine = null,
           currentLine = null;

    foreach (string nextLine in File.ReadLines(filePath))
    {
        if (previousLine == null)
        {
            // Do something special for the first two lines?
        }
        else
        {
            // You're past the first two lines and now have previous,
            // current, and next lines to compare.
        }

        // Save for next iteration
        previousLine = currentLine;
        currentLine = nextLine;
    }

Upvotes: 1

Related Questions