CODEWITHSUNDEEP
c#xmltask-parallel-library
SanhitaPC
SanhitaPC

Reputation: 1

Parallel Reading multiple XML files and inserting in database

As performance improvement, I want to read 10 XML files in parallel and insert data in database.

1.How to achieve reading and database insert in parallel? i.e. when one file is reading insert data of already read XML file.

2.How to achieve processing multiple XML files(including reading and database insert) in parallel. i.e.Processing 10 files at same time. Language C#

Used Task.Factory.StartNew(() => Method1(), CancellationToken.None, 
                           TaskCreationOptions.LongRunning, TaskScheduler.Default);

but method inside Method1() getting called only once.

Upvotes: 0

Views: 918

Answers (1)

Alexander Petrov
Alexander Petrov

Reputation: 14231

Use standard producer/consumer pattern.

var xmlFiles = new BlockingCollection<XDocument>();

var readFiles = Task.Run(() =>
{
    try
    {
        foreach (var file in Directory.EnumerateFiles(".", "*.xml"))
            xmlFiles.Add(XDocument.Load(file));
    }
    finally { xmlFiles.CompleteAdding(); }
});

var processFiles = Task.Run(() =>
{
    foreach (var xml in xmlFiles.GetConsumingEnumerable())
    {
        // Insert data to database
    }
});

Task.WaitAll(readFiles, processFiles);

This method provides independent reading files from disk and writing data to the database.

Upvotes: 1

Related Questions