Move files from Azure Storage blob to an Ftp server

Question

I need to upload a few files from Azure Storage to an external Ftp server.

Is there any way with Azure to uplodad these files directly without download them before ?

Answer 1

I came across this question whilst looking for the same answer, I came up with the following solution:

• Get the Azure file as a Stream [Handled by Azure Functions for you]
• Using WebClient Upload the Stream

This allowed me to transfer the file directly from Blob Storage to an FTP client. For me the Azure Blob file as a Stream was already done as I was creating an Azure Function based on a blob trigger.

I then converted the Stream to a MemoryStream and passed that to WebClient.UploadData() as a byte array [very roughly something like]:

// ... Get the Azure Blob file in to a Stream called myBlob
// As mentioned above the Azure function does this for you:
// public static void Run([BlobTrigger("containerName/{name}", Connection = "BlobConnection")]Stream myBlob, string name, ILogger log)

public void UploadStreamToFtp(Stream file, string targetFilePath)
    {
        using (MemoryStream ms = new MemoryStream())
        {
            // As memory stream already handles ToArray() copy the Stream to the MemoryStream
            file.CopyTo(ms);

            using (WebClient client = new WebClient())
            {
                // Use login credentails if required
                client.Credentials = new NetworkCredential("username", "password");

                // Upload the stream as Data with the STOR method call
                // targetFilePath is a fully qualified filepath on the FTP, e.g. ftp://targetserver/directory/filename.ext
                client.UploadData(targetFilePath, WebRequestMethods.Ftp.UploadFile, ms.ToArray());
            }
        }
    }

Answer 2

You will need to use two classes/libraries and create two methods here:

1. WebClient class to download the file from the blob storage to your local drive
2. FTP library such WinSCP to move the file

WebClient Class: You need to supply the URI parameter with the format: https://[accountname].blob.core.windows.net/[containername]/[filetodownloadincludingextension]

The download location must then be a variable as the origin location of the file to be uploaded to your FTP server.

        string uri = "https://[accountname].blob.core.windows.net/[containername]/[filetodownloadincludingextension]/";
        string file = "file1.txt";
        string downloadLocation = @"C:\";

        WebClient webClient = new WebClient();
        Log("Downloading File from web...");
        try
        {
            webClient.DownloadFile(new Uri(uri+file), downloadLocation);
            Log("Download from web complete");
            webClient.Dispose();
        }
        catch (Exception ex)
        {
            Log("Error Occurred in downloading file. See below for exception details");
            Log(ex.Message);
            webClient.Dispose();
        } 
        return downloadLocation + file;

Once downloaded in your local drive, you need to upload it to your FTP/SFTP server. You may use the library of WinSCP for this:

        string absPathSource = downloadLocation + file;
        string destination = "/root/folder"; //this basically is your FTP path

    // Setup session options
        SessionOptions sessionOptions = new SessionOptions
        {

            Protocol = Protocol.Sftp,
            HostName = ConfigurationManager.AppSettings["scpurl"],
            UserName = ConfigurationManager.AppSettings["scpuser"],
            Password = ConfigurationManager.AppSettings["scppass"].Trim(),
            SshHostKeyFingerprint = ConfigurationManager.AppSettings["scprsa"].Trim()
        };

        using (Session session = new Session())
        {

            //disable version checking
            session.DisableVersionCheck = true;

            // Connect
            session.Open(sessionOptions);

            // Upload files
            TransferOptions transferOptions = new TransferOptions();
            transferOptions.TransferMode = TransferMode.Binary;

            TransferOperationResult transferResult;
            transferResult = session.PutFiles(absPathSource, destination, false, transferOptions);

            // Throw on any error
            transferResult.Check();

            // Print results
            foreach (TransferEventArgs transfer in transferResult.Transfers)
            {
                //Console.WriteLine("Upload of {0} succeeded", transfer.FileName);
            }
        }

You may include a File.Delete code at the end of the upload to FTP code if you want to delete the file from your local hard drive after the upload.