CODEWITHSUNDEEP
c#.netmultithreadingftpconsole-application
peter
peter

Reputation: 8662

How to fasten the work by using Threads in c#?

I have thousands of files located in FTP server. My task is to download the files from ftpserver, then unzip the file, then process the file. For downloading i am using Tamir library and for unzipping i am using Ionic.zip and then processing the files.

When i used threads, downloading files from FTP server stopped, don't know the reason, Maybe FTP server is not allowing to download file by using threads. Then i used thread only for unzipping the file and for processing. This also failed with an error like

The process cannot access the file 'file ' because it is being used by another process`.

So now i am doing everything sequentially. Prototype of code is as shown below

 static void Main(string[] args)
        {
            string FTPpah = "d://Testpath";
            DonloadUnzipProcessFile(FTPpah);
        }

        private static void DonloadUnzipProcessFile(string FTPpah)
        {
            string Localpath = @"e://testpath";
            //Using Tamir libraryr
            DownloadFile(FTPpah,Localpath);
            //Ionic.zip library
            UnzipFile(Localpath);
            //c#code
            ProcessFile(Localpath);
        }

Is there any way i can improve this task by using Threads or Process?

EDIT

downloading from FTP server can not be done by threads? If so i am thinking of unzipping and processing by using task. So i will create 10 task (TPL) each will take 10 files at a time and unzip, then ten task will process, are such scenarios possible?

Upvotes: 1

Views: 149

Answers (2)

Soleil
Soleil

Reputation: 7286

First, tasks are NOT necessarily threads. (What is the difference between task and thread?)

Second, I wouldn't recommend to use threads, but Tasks or Parallel.Foreach, since they have their own optimization, unless you have something very specific to achieve through threads.

For your scenario, I would do this: create a class ProcessFile that will download, unzip, process one file, and trigger an event; have a Enumerable/List of n (say 10) instances of ProcessFile; the class that will manage those ProcessFile would react to the event by adding a new instance so you keep n active files being processed.

Good luck

Upvotes: 2

Mrinal Kamboj
Mrinal Kamboj

Reputation: 11478

Following shall be your code to create Asynchronous version, which can do the file download in the background. You can do it for 1000s of file, it will never clog the system, will have a very high throughput, since everything will happen in the Background and will be very fast.

async Task Main()
{
    // List of FTP Path and Local file Path for processing
    var ftpFilesForProcessing = new Dictionary<string, string>
    {
      {"Ftp_Path_1","Local_Path_1"},
      {"Ftp_Path_2","Local_Path_2"},
      {"Ftp_Path_3","Local_Path_3"},
    };

    // FTP Files with Task for Async processing
    var ftpFilesTaskForProcessing = new Dictionary<string, Task<string>> ();

    // Add a relevant Task to each file processing
    foreach (var file in ftpFilesForProcessing)
        ftpFilesTaskForProcessing[file.Key] = FtpRead(file.Key,file.Value);

    // All the FTP downloads will be processed here Asynchronously, then it 
       will proceed with the remaining logic
    await Task.WhenAll(ftpFilesTaskForProcessing.Values);

    // Unzip All files Asynchronously

    // Process Data using Task Parallel Library     
}

// Read the Ftp file to a local file
public async Task<string> FtpRead(string ftpPath, string localPath)
{
    // Create FTP Request object  
    FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create(ftpPath);

    // Set FTP Request Object properties
    ftpRequest.KeepAlive = false;
    ftpRequest.UseBinary = true;
    ftpRequest.Method = WebRequestMethods.Ftp.DownloadFile;

    // This example assumes the FTP site uses anonymous logon.  
    ftpRequest.Credentials = new NetworkCredential("<Username>", "<Password>"); 

    var ftpWebResponse = await ftpRequest.GetResponseAsync();

    Stream ftpResponseStream = ((FtpWebResponse)ftpWebResponse).GetResponseStream();

    StreamReader ftpStreamReader = new StreamReader(ftpResponseStream);

    StreamWriter ftpStreamWriter = new StreamWriter(localPath);

    var fileData = await ftpStreamReader.ReadToEndAsync();

    await ftpStreamWriter.WriteAsync(fileData);

    ftpStreamReader.Close();
    ftpResponseStream.Close();

    return localPath;
}

Upvotes: 2

Related Questions