Using the native .Net FTP library progress bar does not show correct progress

Question

To make sure I'm doing EVERYTHING correct and in hopes of helping someone else out I have decided to create a test app to upload a file and post it here seeking help on my issue. Hope it helps someone else while helping me! ;)

Worth Noting: - I'm testing this over a slow connection and have limited the speed of the ftp user account to about 10kb/s just to test this out.

Problem: When uploading a file using my newly created thread (function UploadAFile->UF_Thread), I send a packet in the snip-it below and even though the requestStream.Write() returns and allows the thread to continue my server shows the packet is still sending. By the time that 100 packets are sent 3 seconds go by in my loop that sends all the packets while on the server it takes about 26 seconds to upload the file (in which time my app just sits and waits for it to finish showing no progress at all because 23 seconds ago it finished according to the GUI).

Expected Results: I need to find a way to set bStillSending to true until the packet finishes. I want to wait for each packet.

start snip-it

DateTime startTimestamp = DateTime.Now;
requestStream.Write(buffer, 0, bytesRead);

//Wait for packet to get to server
bool bStillSending = false;
do
{
  Thread.Sleep(50);
} while (bStillSending);

bytesRead = sourceStream.Read(buffer, 0, ufParams.PacketSize);
lBytesSent += bytesRead;

TimeSpan tsUploadTime = DateTime.Now.Subtract(startTimestamp);
if (tsUploadTime.Milliseconds > 0) lstSpeedInBytesPerMillisecond.Add(bytesRead / tsUploadTime.Milliseconds);
else lstSpeedInBytesPerMillisecond.Add(bytesRead);
sSpeed = ConvertBytesToLargestType(lstSpeedInBytesPerMillisecond.Last());
Raise_UploadFile_Progress_Event(bytesRead, lBytesSent, sourceStream.Length, sSpeed);

end snip-it

start of entire application

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Net;
using System.Threading;
using System.Windows;

namespace WindowsFormsApplication1
{
    class UploadFile
    {
        #region Private class variables
        private ManualResetEvent m_MRE_StopUpload = new ManualResetEvent(false);
        #endregion

        #region Private class structures
        private class UploadFile_PARAMS
        {
            #region Local private class variables
            private int m_iThreadID = 0;
            private ManualResetEvent m_MRE_StopUpload = new ManualResetEvent(false);
            private string m_sServer = "";
            private string m_sUser = "";
            private string m_sPass = "";
            private string m_sSourcePath = "";
            private string m_sDestPath = "";
            private int m_iPacketSize = 512;
            #endregion

            #region Properties
            public int ThreadID
            {
                get
                {
                    return m_iThreadID;
                }
                set
                {
                    m_iThreadID = value;
                }
            }

            public ManualResetEvent MRE_StopUpload
            {
                get
                {
                    return m_MRE_StopUpload;
                }
                set
                {
                    m_MRE_StopUpload = value;
                }
            }

            public string Server
            {
                get
                {
                    return m_sServer;
                }
                set
                {
                    m_sServer = value;
                }
            }

            public string User
            {
                get
                {
                    return m_sUser;
                }
                set
                {
                    m_sUser = value;
                }
            }

            public string Pass
            {
                get
                {
                    return m_sPass;
                }
                set
                {
                    m_sPass = value;
                }
            }

            public string SourcePath
            {
                get
                {
                    return m_sSourcePath;
                }
                set
                {
                    m_sSourcePath = value;
                }
            }

            public string DestPath
            {
                get
                {
                    return m_sDestPath;
                }
                set
                {
                    m_sDestPath = value;
                }
            }

            public int PacketSize
            {
                get
                {
                    return m_iPacketSize;
                }
                set
                {
                    m_iPacketSize = value;
                }
            }
            #endregion
        }
        #endregion

        #region Public class methods
        //Public method UploadFile
        public void UploadSingleFile(string sServer, string sUser, string sPass, string sSourcePath, string sDestPath, int iPacketSize)
        {
            UploadFile_PARAMS ufParams = new UploadFile_PARAMS();
            ufParams.Server = sServer;
            ufParams.User = sUser;
            ufParams.Pass = sPass;
            ufParams.SourcePath = sSourcePath;
            ufParams.DestPath = sDestPath;
            ufParams.PacketSize = iPacketSize;
            ufParams.MRE_StopUpload = m_MRE_StopUpload;

            UploadAFile(ufParams);
        }

        //Public method CancelUpload
        public void CancelUpload()
        {
            m_MRE_StopUpload.Set();
        }
        #endregion

        #region Public class events
        //Public event UploadFile_Progress
        public delegate void UploadFile_Progress_EventHandler(long lBytesSent, long lTotalBytesSent, long lTotalFileSizeBytes, string sSpeed);
        public event UploadFile_Progress_EventHandler UploadFile_Progress;
        private void Raise_UploadFile_Progress_Event(long lBytesSent, long lTotalBytesSent, long lTotalFileSizeBytes, string sSpeed)
        {
            if (this.UploadFile_Progress != null)
            {
                //The event handler exists so raise the event to it
                this.UploadFile_Progress(lBytesSent, lTotalBytesSent, lTotalFileSizeBytes, sSpeed);
            }
        }

        //Public event UploadFile_Complete
        public delegate void UploadFile_Complete_EventHandler(long lTotalBytesSent, long lTotalFileSizeBytes, bool bError, string sAverageSpeed, TimeSpan tsTimeItTookToUpload);
        public event UploadFile_Complete_EventHandler UploadFile_Complete;
        private void Raise_UploadFile_Complete_Event(long lTotalBytesSent, long lTotalFileSizeBytes, bool bError, string sAverageSpeed, TimeSpan tsTimeItTookToUpload)
        {
            if (this.UploadFile_Complete != null)
            {
                //The event handler exists so raise the event to it
                this.UploadFile_Complete(lTotalBytesSent, lTotalFileSizeBytes, bError, sAverageSpeed, tsTimeItTookToUpload);
            }
        }

        //Public event UploadFile_Error
        public delegate void UploadFile_Error_EventHandler(string sErrorMessage);
        public event UploadFile_Error_EventHandler UploadFile_Error;
        private void Raise_UploadFile_Error_Event(string sErrorMessage)
        {
            if (this.UploadFile_Error != null)
            {
                //The event handler exists so raise the event to it
                this.UploadFile_Error(sErrorMessage);
            }
        }
        #endregion

        #region Private class functions
        //Private class function ConvertBytesToLargestType
        private string ConvertBytesToLargestType(long lBytesAMillisecond)
        {
            //Convert the byte count to the highest type
            double dKiloBytesAMillisecond = (double)lBytesAMillisecond / (double)1024;
            double dMegaBytesAMillisecond = dKiloBytesAMillisecond / (double)1024;
            double dGigaBytesAMillisecond = dMegaBytesAMillisecond / (double)1024;

            if (dGigaBytesAMillisecond >= 1)
            {
                return dGigaBytesAMillisecond.ToString("#.##") + "Gb/s";
            }

            if (dMegaBytesAMillisecond >= 1)
            {
                return dMegaBytesAMillisecond.ToString("#.##") + "Mb/s";
            }

            if (dKiloBytesAMillisecond >= 1)
            {
                return dKiloBytesAMillisecond.ToString("#") + "kb/s";
            }

            return lBytesAMillisecond.ToString("#") + "b/s";
        }
        #endregion

        #region Private class multi-threading functions
        //Thread creation for UploadAFile
        System.Threading.Thread threadUF;
        private void UploadAFile(UploadFile_PARAMS ufParams)
        {
            //Make sure to destory any old objects
            if (threadUF != null)
            {
                if (threadUF.IsAlive) threadUF.Abort();
                while (threadUF.IsAlive) Application.DoEvents();
                threadUF = null;
                GC.Collect();
            }

            //Make sure the params object is not null
            if (ufParams == null) return;

            //Create the new thread
            threadUF = new System.Threading.Thread(new System.Threading.ParameterizedThreadStart(UF_Thread));
            threadUF.IsBackground = true;
            ufParams.ThreadID = threadUF.ManagedThreadId;

            //Start the thread
            threadUF.Start(ufParams);
        }
        private void UF_Thread(Object oParameter)
        {
            //Extract (cast) the params
            UploadFile_PARAMS ufParams = (UploadFile_PARAMS)oParameter;

            FtpWebRequest request;
            FtpWebResponse response;

            Stream sourceStream = new MemoryStream();
            Stream requestStream = sourceStream;

            long lBytesSent = 0;
            string sSpeed = "";
            TimeSpan tsTimeToUploadFile;
            DateTime dtStartedUpload = DateTime.Now;
            List<long> lstSpeedInBytesPerMillisecond = new List<long>();

            try
            {
                request = (FtpWebRequest)WebRequest.Create("ftp://" + ufParams.Server + ufParams.DestPath);
                request.Method = WebRequestMethods.Ftp.UploadFile;
                request.Credentials = new NetworkCredential(ufParams.User, ufParams.Pass);

                sourceStream = new FileStream(ufParams.SourcePath, FileMode.Open);

                requestStream = request.GetRequestStream();
                request.ContentLength = sourceStream.Length;

                byte[] buffer = new byte[ufParams.PacketSize];
                int bytesRead = sourceStream.Read(buffer, 0, ufParams.PacketSize);

                do
                {
                    DateTime startTimestamp = DateTime.Now;
                    requestStream.Write(buffer, 0, bytesRead);

                    //Wait for packet to get to server
                    bool bStillSending = false;
                    do
                    {
                      Thread.Sleep(50);
                    } while (bStillSending);

                    bytesRead = sourceStream.Read(buffer, 0, ufParams.PacketSize);
                    lBytesSent += bytesRead;

                    TimeSpan tsUploadTime = DateTime.Now.Subtract(startTimestamp);
                    if (tsUploadTime.Milliseconds > 0) lstSpeedInBytesPerMillisecond.Add(bytesRead / tsUploadTime.Milliseconds);
                    else lstSpeedInBytesPerMillisecond.Add(bytesRead);
                    sSpeed = ConvertBytesToLargestType(lstSpeedInBytesPerMillisecond.Last());
                    Raise_UploadFile_Progress_Event(bytesRead, lBytesSent, sourceStream.Length, sSpeed);
                } while (bytesRead > 0 && ufParams.MRE_StopUpload.WaitOne(0) == false);

                if (ufParams.MRE_StopUpload.WaitOne(0)) throw new System.Net.WebException("Upload was canceled!");

                //Close the stream to show we are finished with the file
                requestStream.Close();

                response = (FtpWebResponse)request.GetResponse();
                if (lstSpeedInBytesPerMillisecond.Count > 0)
                {
                    sSpeed = ConvertBytesToLargestType(Convert.ToInt64(lstSpeedInBytesPerMillisecond.Average()));
                }
                else sSpeed = "0 b/s";
                tsTimeToUploadFile = DateTime.Now.Subtract(dtStartedUpload);
                Raise_UploadFile_Complete_Event(lBytesSent, sourceStream.Length, false, sSpeed, tsTimeToUploadFile);
            }
            catch (Exception ex)
            {
                Raise_UploadFile_Error_Event(ex.GetType().Name + ": " + ex.Message);
                if (lstSpeedInBytesPerMillisecond.Count > 0)
                {
                    sSpeed = ConvertBytesToLargestType(Convert.ToInt64(lstSpeedInBytesPerMillisecond.Average()));
                }
                else sSpeed = "0 b/s";
                tsTimeToUploadFile = DateTime.Now.Subtract(dtStartedUpload);
                Raise_UploadFile_Complete_Event(lBytesSent, sourceStream.Length, true, sSpeed, tsTimeToUploadFile);
            }
            finally
            {
                request = null;

                sourceStream.Close();
                sourceStream = null;

                requestStream.Close();
                requestStream = null;
            }
        }
        #endregion
    }

    public partial class Form1 : Form
    {
        //Define the upload object as a global form object
        UploadFile upload;

        public Form1()
        {
            InitializeComponent();
        }

        #region Buttons
        private void button1_Click(object sender, EventArgs e)
        {
            OpenFileDialog openFileDialog1 = new OpenFileDialog();

            openFileDialog1.InitialDirectory = "c:\\";
            openFileDialog1.Filter = "txt files (*.txt)|*.txt|All files (*.*)|*.*";
            openFileDialog1.FilterIndex = 2;
            openFileDialog1.RestoreDirectory = true;

            if (openFileDialog1.ShowDialog() == DialogResult.OK)
            {
                if (File.Exists(openFileDialog1.FileName))
                {
                    label1.Text = openFileDialog1.FileName;
                    listView1.Items.Add("Selected file to upload.");
                }
                else
                {
                    label1.Text = "";
                    listView1.Items.Add("File choosen does not exist!");
                }
            }
        }

        private void button2_Click(object sender, EventArgs e)
        {
            if (!File.Exists(label1.Text))
            {
                listView1.Items.Add(DateTime.Now.ToString("hh:mm:ss:f") + " Error!  File selected does not exist!");
                return;
            }

            listView1.Items.Add(DateTime.Now.ToString("hh:mm:ss:f") + " Starting the upload process...");

            progressBar1.Minimum = 0;
            progressBar1.Maximum = 100;

            upload = new UploadFile();
            upload.UploadFile_Progress += new UploadFile.UploadFile_Progress_EventHandler(upload_UploadFile_Progress);
            upload.UploadFile_Complete += new UploadFile.UploadFile_Complete_EventHandler(upload_UploadFile_Complete);
            upload.UploadFile_Error += new UploadFile.UploadFile_Error_EventHandler(upload_UploadFile_Error);
            upload.UploadSingleFile("ftp.xentar.com", "arvo", "S1mP13", label1.Text, "/arvo/" + Path.GetFileName(label1.Text), 1024);
        }

        private void button3_Click(object sender, EventArgs e)
        {
            listView1.Items.Add(DateTime.Now.ToString("hh:mm:ss:f") + " Canceling upload...");
            upload.CancelUpload();
        }
        #endregion

        #region UploadFile object events
        private void upload_UploadFile_Error(string sErrorMessage)
        {
            listView1.Items.Add(DateTime.Now.ToString("hh:mm:ss:f") + " Error!  " + sErrorMessage);
        }

        private void upload_UploadFile_Complete(long lTotalBytesSent, long lTotalFileSizeBytes, bool bError, string sAverageSpeed, TimeSpan tsTimeItTookToUpload)
        {
            if (bError)
            {
                AddItemToLog("Error uploading file!");
            }
            else
            {
                AddItemToLog("Upload complete!");
                AddItemToLog("It took " + tsTimeItTookToUpload.ToString("mm") + " minutes and " + tsTimeItTookToUpload.ToString("ss") + " seconds to upload the file.");
                UpdateProgressBar(100, lTotalBytesSent);
            }
        }

        private void upload_UploadFile_Progress(long lBytesSent, long lTotalBytesSent, long lTotalFileSizeBytes, string sSpeed)
        {
            int iProgressComplete = Convert.ToInt32(((double)lTotalBytesSent / (double)lTotalFileSizeBytes) * 100);
            UpdateProgressBar(iProgressComplete, lTotalBytesSent);
        }
        #endregion

        #region Thread-safe functions
        private void AddItemToLog(string sLogEntry)
        {
            //Send to tread safe call
            ThreadSafe_AddItemToLog(sLogEntry);
        }
        delegate void ThreadSafe_AddItemToLog_Callback(string sLogEntry);
        private void ThreadSafe_AddItemToLog(string sLogEntry)
        {
            if (this.InvokeRequired)
            {
                ThreadSafe_AddItemToLog_Callback d =
                    new ThreadSafe_AddItemToLog_Callback(ThreadSafe_AddItemToLog);
                try
                {
                    this.Invoke(d, new object[] { sLogEntry });
                }
                catch
                {
                    //ObjectDisposedException
                }
            }
            else
            {
                //Send the call on to the normal function
                ThreadSafe_AddItemToLog_Proc(sLogEntry);
            }
        }
        private void ThreadSafe_AddItemToLog_Proc(string sLogEntry)
        {
            listView1.Items.Add(DateTime.Now.ToString("hh:mm:ss:f") + " " + sLogEntry);
        }

        private void UpdateProgressBar(int iProgressBarValue, long lTotalBytesSent)
        {
            //Send to tread safe call
            ThreadSafe_UpdateProgressBar(iProgressBarValue, lTotalBytesSent);
        }
        delegate void ThreadSafe_UpdateProgressBar_Callback(int iProgressBarValue, long lTotalBytesSent);
        private void ThreadSafe_UpdateProgressBar(int iProgressBarValue, long lTotalBytesSent)
        {
            if (this.InvokeRequired)
            {
                ThreadSafe_UpdateProgressBar_Callback d =
                    new ThreadSafe_UpdateProgressBar_Callback(ThreadSafe_UpdateProgressBar);
                try
                {
                    this.Invoke(d, new object[] { iProgressBarValue, lTotalBytesSent });
                }
                catch
                {
                    //ObjectDisposedException
                }
            }
            else
            {
                //Send the call on to the normal function
                ThreadSafe_UpdateProgressBar_Proc(iProgressBarValue, lTotalBytesSent);
            }
        }
        private void ThreadSafe_UpdateProgressBar_Proc(int iProgressBarValue, long lTotalBytesSent)
        {
            label4.Text = lTotalBytesSent.ToString();
            progressBar1.Value = iProgressBarValue;
        }
        #endregion
    }
}

Answer 1

Setting the speed artificially like that won't have the same effect as a slow link. Limiting in the client limits the amount it reads from the buffer, but won't actually change the rate it's sent at until the buffers are filled up on both ends.

If you try uploading a much larger file, you should see the upload stall once they've filled up the buffers, then it carry on at the expected rate.

.Write() will block until it's written to the socket, and if you want it to be async, you'll need to use the async .BeginWrite() calls.

Answer 2

Why not use a background worker to handle the progressbar, then if you want it to wait, then it does not bog down your main thread.