CODEWITHSUNDEEP
c#csvhelper
Jasin
Jasin

Reputation: 21

CsvHelper doesn't write to file as expected

Below is the code I am using, I expect that each line will be wrote to the file during each iteration of the loop. What I am finding is that if I end the program early using ctrl-c or just closing the window the file is empty.

I need to write to the file quite often, I have 33k items to add to the file and its almost inevitable something goes wrong before the 33k items are added to the CsvWriter. If I short circuit the loop to say 20 items and let it finish, the output file is correct. How do I solve this issue?

int i = 0;
IList<string> pcodes = FindAutobidPcodes();
using (StreamWriter sw = new StreamWriter($"HPH_Catalog_{DateTime.Now.Ticks}.csv"))
using (CsvWriter cw = new CsvWriter(sw, CultureInfo.InvariantCulture))
{
    cw.Configuration.RegisterClassMap<ProductMap>();
    foreach (string code in pcodes)
    {
        string search = $"&search=%7b\"catalog\"%3a\"{catalog}\"%2c\"text\"%3a%7b\"{code}\"%3a\"{code}\"%7d%7d";
        searchQuery.Query = $"{RandomString()}{limit}{page}{search}{gtemplate}&token={token}";
        WebPage webPage = QueryProductData(searchQuery.ToString());
        cw.WriteRecord(Product.ParseJSON(webPage));
        cw.NextRecord();
        i++;
        if(i == 20) { return; }
    }
}

Upvotes: 2

Views: 1212

Answers (1)

Alex Leo
Alex Leo

Reputation: 2851

What you can do is flushing the StreamWriter after each iteration eg:

    static void Main(string[] args)
    {
        using (StreamWriter sw = new StreamWriter($"HPH_Catalog_{DateTime.Now.Ticks}.csv"))
        using (CsvWriter cw = new CsvWriter(sw, CultureInfo.InvariantCulture))
        {
            for (int i = 0; i < 34000; i++)
            {
                cw.WriteField(i);
                cw.NextRecord();

                sw.Flush();
                System.Threading.Thread.Sleep(100);
            }
        }
    }

You can also encapsulate the inner foor-each loop into a try statement- if something goes wrong whilst looping then in the catch statement flush and dispose.

Upvotes: 3

Related Questions