CODEWITHSUNDEEP
c#winformsweb-applications
Tanya
Tanya

Reputation: 1621

How to increase the writing speed of the streamwriter?

How to raise the speed of the streamwriter to write a 83MB csv file. I have inceresed the buffersize to 65536 but its also consuming more time. How to improve the speed.

StreamWriter writer =new streamWriter(
      new FileStream(filePath, FileMode.CreateNew), Encoding.UTF8, 65536))
string str=string.Empty;
while((str = reader.ReadLine())!=null)
    writer.WriteLine(str)}        
writer.Close()

Upvotes: 1

Views: 6942

Answers (3)

Theodore Zographos
Theodore Zographos

Reputation: 2395

Depending on the number of lines your CSV file contains, you could possibly end up with a loop that executes millions of times. Not a good idea to access the disk that many times.

The easy way out (if you have memory) is to read the entire CSV file in a string[] in memory (using File.ReadAll() i think), do your processing and write it all once (File.WriteAll() i think). This will greatly increase your performance.

The other way out is to use asynchronous read/writes, increase buffer size AND create a mechanism to read bigger chunks of data. Having a big buffer if you are only reading 1 line will not help you.

Upvotes: 2

Dan Byström
Dan Byström

Reputation: 9244

Based on your comment showing what your code is actualy doing:

File.WriteAllText( filePath, reader.ReadToEnd() );

Upvotes: 0

mroach
mroach

Reputation: 2478

I would try raising the buffer even more. At that buffer size it's going to take over 8200 writes to create the whole file. Try a buffer around 256K or 512K.

Upvotes: 0

Related Questions