Adding pause and continue ability in my downloader

Question

I am creating a downloader in C#. I am using WebClient class. To pause downloading on a button click I could just think of using Thread. So when I created thread and attached it with my file downloading as below

WebClient web = new WebLCient();
Thread dwnd_thread = new Thread(Program.web.DownloadFileAsync(new Uri(Program.src), Program.dest));

it is giving me following errors: "The best overloaded method match for 'System.Threading.Thread.Thread(System.Threading.ThreadStart)' has some invalid arguments" and "Argument '1': cannot convert from 'void' to 'System.Threading.ThreadStart'".

Then I thought if I pause my system main thread then it could block my whole process for that I used below line of code

System.Threading.Thread.Sleep(100);

but it is doing nothing at all. Can somebody tell me what could be a better approach for pause/downloading and how to use thread to pause my downloading process.

Answer 1

I've taken @ebb's solution (which works great!) and adapted it a bit. It now:

• Can accept a stream as an input
• Can report progress via IProgress
• Some other minor tweaks

public class FileDownload
{
    private volatile bool _allowedToRun;
    private Stream _sourceStream;
    private string _sourceUrl;
    private string _destination;
    private bool _disposeOnCompletion;
    private int _chunkSize;
    private IProgress<double> _progress;
    private Lazy<long> _contentLength;

    public long BytesWritten { get; private set; }
    public long ContentLength { get { return _contentLength.Value; } }

    public bool Done { get { return ContentLength == BytesWritten; } }

    public FileDownload(Stream source, string destination, bool disposeOnCompletion = true, int chunkSizeInBytes = 10000 /*Default to 0.01 mb*/, IProgress<double> progress = null)
    {
        _allowedToRun = true;

        _sourceStream = source;
        _destination = destination;
        _disposeOnCompletion = disposeOnCompletion;
        _chunkSize = chunkSizeInBytes;
        _contentLength = new Lazy<int>(() => Convert.ToInt32(GetContentLength()));
        _progress = progress;

        BytesWritten = 0;
    }

    public FileDownload(string source, string destination, int chunkSizeInBytes = 10000 /*Default to 0.01 mb*/, IProgress<double> progress = null)
    {
        _allowedToRun = true;

        _sourceUrl = source;
        _destination = destination;
        _chunkSize = chunkSizeInBytes;
        _contentLength = new Lazy<int>(() => Convert.ToInt32(GetContentLength()));
        _progress = progress;

        BytesWritten = 0;
    }

    private long GetContentLength()
    {
        if (_sourceStream != null)
            return _sourceStream.Length;
        else
        {
            var request = (HttpWebRequest)WebRequest.Create(_sourceUrl);
            request.Method = "HEAD";

            using (var response = request.GetResponse())
                return response.ContentLength;
        }
    }

    private async Task Start(int range)
    {
        if (!_allowedToRun)
            throw new InvalidOperationException();

        if (_sourceStream != null)
        {
            using (var fs = new FileStream(_destination, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
            {
                while (_allowedToRun)
                {
                    var buffer = new byte[_chunkSize];
                    var bytesRead = await _sourceStream.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);

                    if (bytesRead == 0) break;

                    await fs.WriteAsync(buffer, 0, bytesRead);
                    BytesWritten += bytesRead;
                    _progress?.Report((double)BytesWritten / ContentLength);
                }

                await fs.FlushAsync();
            }

            //Control whether the stream should be disposed here or outside of this class
            if (BytesWritten == ContentLength && _disposeOnCompletion)
                _sourceStream?.Dispose();
        }
        else
        {
            var request = (HttpWebRequest)WebRequest.Create(_sourceUrl);
            request.Method = "GET";
            request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
            request.AddRange(range);

            using (var response = await request.GetResponseAsync())
            {
                using (var responseStream = response.GetResponseStream())
                {
                    using (var fs = new FileStream(_destination, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
                    {
                        while (_allowedToRun)
                        {
                            var buffer = new byte[_chunkSize];
                            var bytesRead = await responseStream.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);

                            if (bytesRead == 0) break;

                            await fs.WriteAsync(buffer, 0, bytesRead);
                            BytesWritten += bytesRead;
                            _progress?.Report((double)BytesWritten / ContentLength);
                        }

                        await fs.FlushAsync();
                    }
                }
            }
        }
    }

    public Task Start()
    {
        _allowedToRun = true;
        return Start(BytesWritten);
    }

    public void Pause()
    {
        _allowedToRun = false;
    }
}

Answer 2

I have adapted it in order to deal with files > 2GB and deals with broken download, so that for any reasons the download was stopped and you need to try again, at next download attempt, instead of starting again from scratch it will continue where it was stopped. All above solutions from @derekantrican, @ebb and myself will only work if server supports Range requests. You might need to test first if server accepts range requests before to proceed with this solution. https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

 public class FileDownload
 {
    private volatile bool _allowedToRun;
    private readonly string _sourceUrl;
    private readonly string _destination;
    private readonly int _chunkSize;
    private readonly IProgress<double> _progress;
    private readonly Lazy<long> _contentLength;

    public long BytesWritten { get; private set; }
    public long ContentLength => _contentLength.Value;

    public bool Done => ContentLength == BytesWritten;

    public FileDownload(string source, string destination, int chunkSizeInBytes = 10000 /*Default to 0.01 mb*/, IProgress<double> progress = null)
    {
        if(string.IsNullOrEmpty(source))
            throw new ArgumentNullException("source is empty");
        if (string.IsNullOrEmpty(destination))
            throw new ArgumentNullException("destination is empty");

        _allowedToRun = true;
        _sourceUrl = source;
        _destination = destination;
        _chunkSize = chunkSizeInBytes;
        _contentLength = new Lazy<long>(GetContentLength);
        _progress = progress;

        if (!File.Exists(destination))
            BytesWritten = 0;
        else
        {
            try
            {
                BytesWritten = new FileInfo(destination).Length;
            }
            catch
            {
                BytesWritten = 0;
            }
        }
    }

    private long GetContentLength()
    {
        var request = (HttpWebRequest)WebRequest.Create(_sourceUrl);
        request.Method = "HEAD";

        using (var response = request.GetResponse())
            return response.ContentLength;
    }

    private async Task Start(long range)
    {
        if (!_allowedToRun)
            throw new InvalidOperationException();

        if(Done)
            //file has been found in folder destination and is already fully downloaded 
            return;

        var request = (HttpWebRequest)WebRequest.Create(_sourceUrl);
        request.Method = "GET";
        request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
        request.AddRange(range);

        using (var response = await request.GetResponseAsync())
        {
            using (var responseStream = response.GetResponseStream())
            {
                using (var fs = new FileStream(_destination, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
                {
                    while (_allowedToRun)
                    {
                        var buffer = new byte[_chunkSize];
                        var bytesRead = await responseStream.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);

                        if (bytesRead == 0) break;

                        await fs.WriteAsync(buffer, 0, bytesRead);
                        BytesWritten += bytesRead;
                        _progress?.Report((double)BytesWritten / ContentLength);
                    }

                    await fs.FlushAsync();
                }
            }
        }
    }

    public Task Start()
    {
        _allowedToRun = true;
        return Start(BytesWritten);
    }

    public void Pause()
    {
        _allowedToRun = false;
    }
}

Answer 3

As there's no standard way of pause/resume a download request, you'll have to implement your own mechanism. Below is a block of code, containing an example of how such a mechanism could look. The class FileDownload takes 3 parameters:

• source - url to the file, to download.

• destination - where to save the file.

• chunkSize - how many bytes to read, before checking whether to pause or continue the download.

---

public class FileDownload
{
    private volatile bool _allowedToRun;
    private string _source;
    private string _destination;
    private int _chunkSize;

    private Lazy<int> _contentLength;

    public int BytesWritten { get; private set; }
    public int ContentLength { get { return _contentLength.Value; } }

    public bool Done { get { return ContentLength == BytesWritten; } }

    public FileDownload(string source, string destination, int chunkSize)
    {
        _allowedToRun = true;

        _source = source;
        _destination = destination;
        _chunkSize = chunkSize;
        _contentLength = new Lazy<int>(() => Convert.ToInt32(GetContentLength()));

        BytesWritten = 0;
    }

    private long GetContentLength()
    {
        var request = (HttpWebRequest)WebRequest.Create(_source);
        request.Method = "HEAD";

        using (var response = request.GetResponse())
            return response.ContentLength;
    }

    private async Task Start(int range)
    {
        if (!_allowedToRun)
            throw new InvalidOperationException();

        var request = (HttpWebRequest)WebRequest.Create(_source);
        request.Method = "GET";
        request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
        request.AddRange(range);

        using (var response = await request.GetResponseAsync())
        {
            using (var responseStream = response.GetResponseStream())
            {
                using (var fs = new FileStream(_destination, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
                {
                    while (_allowedToRun)
                    {
                        var buffer = new byte[_chunkSize];
                        var bytesRead = await responseStream.ReadAsync(buffer, 0, buffer.Length);

                        if (bytesRead == 0) break;

                        await fs.WriteAsync(buffer, 0, bytesRead);
                        BytesWritten += bytesRead;
                    }

                    await fs.FlushAsync();
                }
            }
        }
    }

    public Task Start()
    {
        _allowedToRun = true;
        return Start(BytesWritten);
    }

    public void Pause()
    {
        _allowedToRun = false;
    }
}

Usage:

static void Main(string[] args)
{
    var fw = new FileDownload("http://download.microsoft.com/download/E/E/2/EE2D29A1-2D5C-463C-B7F1-40E4170F5E2C/KinectSDK-v1.0-Setup.exe", @"D:\KinetSDK.exe", 5120);

    // Display progress...
    Task.Factory.StartNew(() =>
    {
        while (!fw.Done)
        {
            Console.SetCursorPosition(0, Console.CursorTop);
            Console.Write(string.Format("ContentLength: {0} | BytesWritten: {1}", fw.ContentLength, fw.BytesWritten));
        }
    });

    // Start the download...
    fw.Start();

    // Simulate pause...
    Thread.Sleep(500);
    fw.Pause();
    Thread.Sleep(2000);

    // Start the download from where we left, and when done print to console.
    fw.Start().ContinueWith(t => Console.WriteLine("Done"));

    Console.ReadKey();
}

Answer 4

unfortunately WebClient has no ways to pause downloading. So, You must use WebRequest on background thread and pause getting response stream with flag. Here is sample code. But make sure that you can't pause unlimited, because TCP connection will be closed if there has nothing transfer for a while. So if resume downloading fail, you must restart downloading again.

public class DownloadJob {

    public delegate void DownloadCompletedDelegate(Stream stream);
    //completion download  event
    public event DownloadCompletedDelegate OnDownloadCompleted;

    //sync object
    private object _lock = new object();
    //pause flag
    private bool _bPause = false;

    //Pause download
    public void Pause() {

        lock (_lock) {
            _bPause = true;
        }
    }

    //Resume download
    public void Resume() {

        lock (_lock) {
            _bPause = false;
        }
    }

    //Begin download with URI
    public void BeginDowload(Uri uri) {

        //Create Background thread
        Thread downLoadThread = new Thread(
            delegate() {

                WebRequest pWebReq = WebRequest.Create(uri);
                WebResponse pWebRes = pWebReq.GetResponse();

                using (MemoryStream pResultStream = new MemoryStream())
                using (Stream pWebStream = pWebRes.GetResponseStream()) {

                    byte[] buffer = new byte[256];

                    int readCount = 1;
                    while (readCount > 0) {

                        //Read download stream 
                        readCount = pWebStream.Read(buffer, 0, buffer.Length);
                        //Write to result MemoryStream
                        pResultStream.Write(buffer, 0, readCount);

                        //Waiting 100msec while _bPause is true
                        while (true) {

                            lock (_lock) {
                                if (_bPause == true) {
                                    Thread.Sleep(100);
                                }
                                else {
                                    break;
                                }
                            }
                        }

                        pResultStream.Flush();
                    }

                    //Fire Completion event
                    if (OnDownloadCompleted != null) {
                        OnDownloadCompleted(pResultStream);
                    }
                }
            }
        );

        //Start background thread job
        downLoadThread.Start();
    }
}