CODEWITHSUNDEEP
c#multithreading
Kevin Walter
Kevin Walter

Reputation: 7166

Simultaneous downloads with Tasks and WebClients

I am doing something completely wrong or just don't understand the concept of multithreading/tasking. I want, lets just say, 5 tasks running simultaneously downloading strings from the internet. On the console it should instantly place the 5 logs but they are not instant. Sorry for my horrible explanation but hopefully my code will clear things up. 

class ThreadedDownloadWorker
{
    public Dictionary<string,string> urls { get; set; }
    public List<Task> tasks = new List<Task>();
    public string downloadGuid;
    public int progress = 0;
    public MainForm main;
    public int amountOfPages;

    public ThreadedDownloadWorker(MainForm main, int amountOfPages)
    {
        this.main = main;
        this.amountOfPages = amountOfPages;
    }

    public void startDownloadStrings(int maxAmountOfThreads, string downloadGuid)
    {
        this.downloadGuid = downloadGuid;
        for (int i = 0; i < maxAmountOfThreads; i++) 
        {
            if (urls.Count > 0) 
            {
                KeyValuePair<string, string> selectedUrl = urls.First();
                Task task = Task.Factory.StartNew(() => doDownloadStrings());
                tasks.Add(task);
            }
        }
    }

    public void doDownloadStrings() 
    {
        KeyValuePair<string, string> selectedUrl = urls.First();
        urls.Remove(selectedUrl.Key);
        WebClient wc = new WebClient();
        Console.WriteLine("Started download: " + selectedUrl.Key);
        string downloadedString = wc.DownloadString(selectedUrl.Value);
        SettingsAndData.saveDownloadPage(downloadGuid, selectedUrl.Key + ".json", downloadedString);
        progress++;
        this.main.blogs[this.main.selectedGuid].status = "Downloaded " + progress + " / " + this.amountOfPages.ToString();
        this.main.reloadBlogGridThreaded();
        wc.Dispose();
        if (urls.Count > 0) 
        {
            //doDownloadStrings();
        }
    }
}

And this is how I call this code:

var amountOfPages = 28;     
Classes.ThreadedDownloadWorker threadedWorker = new Classes.ThreadedDownloadWorker(this, amountOfPages);
threadedWorker.urls = new Dictionary<string, string>();
for(int i = 0; i < amountOfPages; i++)
{
    threadedWorker.urls.Add((i * 50).ToString(), "http://justarandompage.bla.bla/?offset="+ (i * 50).ToString());
}
threadedWorker.startDownloadStrings(5, selectedGuid);

From my understanding this should start 5 download tasks at once. But when I check the console they are not instantly started, but instead start one after another, with maybe a 500ms delay between each.

Upvotes: 0

Views: 162

Answers (1)

Roy
Roy

Reputation: 196

try using service point manager, and set default connection limit

https://msdn.microsoft.com/en-us/library/system.net.servicepointmanager%28v=vs.110%29.aspx

this will remove the limitation of only two connections

Upvotes: 1

Related Questions