CODEWITHSUNDEEP
c#visual-studiofilevisual-studio-2012limit
Sina Sohi
Sina Sohi

Reputation: 2779

Max number of data in file in C#?

I'm trying to edit some data in a file with Visual Studio C#. I've tried using both

StreamReader and File.ReadAllLines / ReadAllText

Both results give me 3414 lines of content. I've jut used Split('\n') after "ReadAllText". But when I check the use the following command on linux I get the follow results:

cat phase1_promoter_data_PtoP1.txt | wc

Output:

 184829 164686174 1101177922

So about 185.000 lines and 165 million words. A word count on Visual Studio gives me about 19 million.

So my question is, am I reading the file wrong or does Visual Studio have a limit on how much data it will read at once? My file takes about about 1 GB space.

Here's the code I use:

try
{
    using (StreamReader sr = new StreamReader("phase1_promoter_data_PtoP1.txt"))
    {
        String line = sr.ReadToEnd();
        Console.WriteLine(line);
        String[,] data = new String[184829, 891];
        //List<String> data2 = new List<String>();

        string[] lol = line.Split('\n');

        for (int i = 0; i < lol.Length; i++)
        {
            String[] oneLine = lol[i].Split('\t');
            //List<String> singleLine = new List<String>(lol[i].Split('\t'));


            for (int j = 0; j < oneLine.Length; j++)
            {
                //Console.WriteLine(i + " - " + lol.Length + " - " + j + " - " + oneLine.Length);
                data[i,j] = oneLine[j];
            }
        }
        Console.WriteLine(data[3413,0]);
    }
}
catch (Exception e)
{                 
    Console.WriteLine(e.Message);
}

Upvotes: 1

Views: 345

Answers (1)

Magic Mick
Magic Mick

Reputation: 1523

The file in your dropbox contains 6043 lines.

Both 

Debug.Print(File.ReadAllLines(fPath).Count().ToString());

And

Debug.Print(File.ReadAllText(fPath).Split('\n').Count().ToString());

Show the same results (Using VS 2013 .NET 4.5)

I was able to cycle through each line with..

 using (var sr = new StreamReader(fPath))
 {
     while (!sr.EndOfStream)
     {
         Debug.Print(sr.ReadLine());
     }
 }

And

 foreach(string line in File.ReadAllLines(fPath))
 {
     Debug.Print(line);
 }

Instead of reading the entire file into a string at once, try one of the loops above and build an array as you cycle through.

Upvotes: 2

Related Questions