SSH.Net Async file download

Question

I am trying to download files asynchronously from an SFTP-server using SSH.NET. If I do it synchronously, it works fine but when I do it async, I get empty files. This is my code:

var port = 22;
string host = "localhost";
string username = "user";
string password = "password";
string localPath = @"C:\temp";

using (var client = new SftpClient(host, port, username, password))
{
    client.Connect();
    var files = client.ListDirectory("");

    var tasks = new List<Task>();

    foreach (var file in files)
    {                        
        using (var saveFile = File.OpenWrite(localPath + "\\" + file.Name))
        {
            //sftp.DownloadFile(file.FullName,saveFile); <-- This works fine
            tasks.Add(Task.Factory.FromAsync(client.BeginDownloadFile(file.FullName, saveFile), client.EndDownloadFile));
        }                        
    }

    await Task.WhenAll(tasks);
    client.Disconnect();

}

Answer 1

I found that this solution (mentioned in another answer) did work, but it was extremely slow. I've tried switching the BufferSize a bunch of times but the performance was nothing compared to the default DownloadFile method.

I've found another solution which allows the download to be run asynchronously without the bottleneck:

public static class SftpClientExtensions
{
    public static async Task DownloadFileAsync(this SftpClient client, string source, string destination, CancellationToken cancellationToken)
    {
        TaskCompletionSource<bool> tcs = new();

        using (FileStream saveFile = File.OpenWrite(destination))
        // cancel the tcs when the token gets cancelled
        using (cancellationToken.Register(() => tcs.TrySetCanceled()))
        {
            // begin the asynchronous operation
            client.BeginDownloadFile(source, saveFile, result =>
            {
                try
                {
                    // Try to complete the operation
                    client.EndDownloadFile(result);

                    // indicate success
                    tcs.TrySetResult(true);
                }
                catch (OperationCanceledException)
                { 
                    // properly handle cancellation
                    tcs.TrySetCanceled();
                }
                catch (Exception ex)
                {
                    // handle any other exceptions
                    tcs.TrySetException(ex);
                }
            }, null);

            // await the task to complete or be cancelled
            await tcs.Task;

            // since we catch the cancellation exepction in the task, we need to throw it again
            cancellationToken.ThrowIfCancellationRequested();
        }
    }
}

Answer 2

They finally updated the methods a while back but didn't add explicit DownloadFileAsync as was originally requested. You can see the discussion here:

https://github.com/sshnet/SSH.NET/pull/819

Where the solution provided is:

/// <summary>
/// Add functionality to <see cref="SftpClient"/>
/// </summary>
public static class SftpClientExtensions
{
    private const int BufferSize = 81920;

    public static async Task DownloadFileAsync(this SftpClient sftpClient, string path, Stream output, CancellationToken cancellationToken)
    {
        await using Stream remoteStream = await sftpClient.OpenAsync(path, FileMode.Open, FileAccess.Read, cancellationToken).ConfigureAwait(false);
        await remoteStream.CopyToAsync(output, BufferSize, cancellationToken).ConfigureAwait(false);
    }

    public static async Task UploadFileAsync(this SftpClient sftpClient, Stream input, string path, FileMode createMode, CancellationToken cancellationToken)
    {
        await using Stream remoteStream = await sftpClient.OpenAsync(path, createMode, FileAccess.Write, cancellationToken).ConfigureAwait(false);
        await input.CopyToAsync(remoteStream, BufferSize, cancellationToken).ConfigureAwait(false);
    }
}

Answer 3

Because saveFile is declared in a using block, it is closed right after you start the task, so the download can't complete. Actually, I'm surprised you're not getting an exception.

You could extract the code to download to a separate method like this:

var port = 22;
string host = "localhost";
string username = "user";
string password = "password";
string localPath = @"C:\temp";

using (var client = new SftpClient(host, port, username, password))
{
    client.Connect();
    var files = client.ListDirectory("");

    var tasks = new List<Task>();

    foreach (var file in files)
    {                        
        tasks.Add(DownloadFileAsync(file.FullName, localPath + "\\" + file.Name));
    }

    await Task.WhenAll(tasks);
    client.Disconnect();

}

...

async Task DownloadFileAsync(string source, string destination)
{
    using (var saveFile = File.OpenWrite(destination))
    {
        var task = Task.Factory.FromAsync(client.BeginDownloadFile(source, saveFile), client.EndDownloadFile);
        await task;
    }
}

This way, the file isn't closed before you finish downloading the file.

---

Looking at the SSH.NET source code, it looks like the async version of DownloadFile isn't using "real" async IO (using IO completion port), but instead just executes the download in a new thread. So there's no real advantage in using BeginDownloadFile/EndDownloadFile; you might as well use DownloadFile in a thread that you create yourself:

Task DownloadFileAsync(string source, string destination)
{
    return Task.Run(() =>
    {
        using (var saveFile = File.OpenWrite(destination))
        {
            client.DownloadFile(source, saveFile);
        }
    }
}