CODEWITHSUNDEEP
c#streamwriter
Andreas
Andreas

Reputation: 1285

Fastest way to write a list of integers to file

I have a list of integers that I need to write to a file.

I wonder if there is any faster way to write those integers to the file than this approach which takes 1.77 seconds?

(The list do have integers from the beginning because they are processed as integers in other functions that needs integers for fast calculations)

void writeToFile()
{
    List<int> list1 = new List<int>();
    for (int i = 0; i < 5000000; i++)
    {
        list1.Add(123);
    }

    //Approach takes: 1.77 seconds
    DateTime start = DateTime.Now;
    StreamWriter writer = null; FileStream fs = null;
    fs = new FileStream("C:/test.txt", FileMode.Append, FileAccess.Write, FileShare.ReadWrite); 
    writer = new StreamWriter(fs);
    for (int i = 0; i < list1.Count; i++)
    {
        writer.WriteLine(list1[i]);
    }
    writer.Close(); fs.Close();

    DateTime end = DateTime.Now;
    TimeSpan span = end - start;
    MessageBox.Show(span.TotalSeconds.ToString());
}

Upvotes: 0

Views: 833

Answers (2)

Andreas
Andreas

Reputation: 1285

This approach takes 0.4 seconds using the BinaryWriter VS StreamWriter(1.77 seconds).
I wonder if it is possible to improve the speed even more?

        void writeToFile()
        {
            List<int> list1 = new List<int>();
            for (int i = 0; i < 5000000; i++)
            {
                list1.Add(i);
            }

            //Approach takes: 0.4 seconds
            DateTime start = DateTime.Now;
            BinaryWriter writer = null; FileStream fs = null;
            fs = new FileStream("C:/test.txt", FileMode.Append, FileAccess.Write, FileShare.ReadWrite, 65536); writer = new BinaryWriter(fs);
            for (int i = 0; i < list1.Count; i++)
            {
                writer.Write(list1[i]);
            }
            writer.Close(); fs.Close();

            DateTime end = DateTime.Now;
            TimeSpan span = end - start;
            MessageBox.Show(span.TotalSeconds.ToString()); //Takes 0.4 seconds


            //Read the integers method
            using (BinaryReader b = new BinaryReader(File.Open("C:/test.txt", FileMode.Open)))
            {
                // 2.
                // Position and length variables.
                int pos = 0;
                // 2A.
                // Use BaseStream.
                int length = (int)b.BaseStream.Length;
                while (pos < length)
                {
                    // 3.
                    // Read integer.
                    int v = b.ReadInt32();
                    // 4.
                    // Advance our position variable.
                    pos += sizeof(int);
                }
            }
        }

Upvotes: 0

Guillaume S.
Guillaume S.

Reputation: 1545

Using BufferedStream won't help because FileStream already has a buffer.

Using BinaryWriter can help because you will write 4 bytes (one integer) instead of 6* bytes (three chars of 2 bytes, and we should count the newlines too) 5 000 000 times. However the file written will be binary. It won't be human-readable anymore.

In your example, because you always write the same number 123, I think using compression will do some magic too ;)

Upvotes: 1

Related Questions