Reputation: 137
Multiple Threading
I am developing an application (in C#). Which is basically a file uploading/downloading application (just like filezila).
But there is one special feature on that (on file Downloading) :-
We can divide a file in various segments(up to 100). And can repeat this process by various Cycles (means we can divide a file in various segments and can repeat this process more then one time(by Cycles),at same time).
I am using Threading concept for that.In the application there are some progress bars and labels that are showing the uploading/downloading status.
My problem is that :-
If i do this process (downloading) for more numbers for example :-
100 Cycles and (the cycles can be more then 100)
100 Segments
Means we are diving a file into 100 parts (segment) and the same process will work 100 times(Cycles).
Means 100 * 100 =10,000 files will download at the time by same process.
But i am facing problem in that if i take big numbers for this task then there should be more threads will opens.
And application will hangs if there are 100 TCP Ports/Threads are running simultaneously.
I tried so many things for that but i did not got success yet.Like -
Asynchronous Delegates,ThreadPool etc.
I am facing a major problem in updating a progress bars and labels (because the current functionality is working with Asynchronous Delegates).
Please suggest me some way to resolve this issue.
Upvotes: 1
Views: 401
Answers (3)
Reputation: 108995
To asynchronously update the UI from a worker thread (whether explicitly created, a work item queued to the thread pool or the callback from an asynchronous API) use Control.BeginInvoke.
This will not block the worker thread while waiting for the UI thread to process the action.
But remember the UI thread cannot assume a specific ordering of asynchronous operations completing (two downloads of equal side started in a particular order are unlikely to complete in that order: too many external factors affect the speed of downloads).
Finally most BeginABC methods require a call to the corresponding EndABC, but (helpfully) Control.BeginInvoke does not.
Upvotes: 0
Reputation: 11
Hello Erno i am sending some code for better understanding :-
this is the Downloading function :
Public Sub FileDownload()
txtBytesToDownload.Text = totalBytesToDownload.ToString("N", nfi) 'set textbox with total file size
del = New MyDelegate(AddressOf DownloadDataInRange)
callback = New AsyncCallback(AddressOf CalcCallback)
TotalBytesToDownloadForEachCycle = totalBytesToDownload
downloadedBytes = 0
dlStopWatch = Stopwatch.StartNew()
Dim flag As Integer = 0
Dim segment As Integer = nuSegment.Value 'total segments
Dim cyl = cycleNumericUpDown.Value 'total cycles
'ServicePointManager.DefaultConnectionLimit = segment
If UtilityFunctions.CheckRangeSupported(dlUrlHttpTextBox.Text) Then
Dim segmentSize As Integer = 0
segmentSize = TotalBytesToDownloadForEachCycle / segment
' For cy As Integer = 0 To (cyl - 1)
For ctr As Integer = 0 To (segment - 1) 'loop for total Segments
GC.Collect()
flag += 1
Dim fileValues As New FileValues()
If ctr = 0 Then
fileValues.StartPoint = 0 'starting point of each segment (in bytes)
Else
fileValues.StartPoint = segmentSize * ctr + 1
End If
fileValues.EndPoint = fileValues.StartPoint + segmentSize 'end point of each segment (in bytes)
If (ctr = (segment - 1)) Then
fileValues.EndPoint += TotalBytesToDownloadForEachCycle Mod segment
End If
fileValues.URL = dlUrlHttpTextBox.Text 'downloading file url
Dim str As String = ctr.ToString() + "_" + flag.ToString()
' del.BeginInvoke(fileValues, callback, Nothing)
newThread = New System.Threading.Thread(AddressOf DownloadBytes) 'Thread starts here
newThread.Priority = ThreadPriority.Highest
newThread.IsBackground = True
newThread.Start(fileValues)
'ThreadPool.QueueUserWorkItem(AddressOf DownloadBytes, fileValues)
fileValues = Nothing
Next
' ProgressBarTotal.Value += 1
' Next
Else
Dim fileValues As New FileValues()
fileValues.StartPoint = 0
fileValues.EndPoint = TotalBytesToDownloadForEachCycle
fileValues.URL = dlUrlHttpTextBox.Text
Dim newThread As New System.Threading.Thread(AddressOf DownloadBytes)
newThread.Name = "Thread1"
newThread.Start(fileValues)
'ThreadPool.QueueUserWorkItem(AddressOf DownloadBytes, fileValues)
fileValues = Nothing
End If
Console.WriteLine("Http Downloload End")
Debug.Print("thread List 1 count =" + _threadsList1.Count.ToString() + ", thread list 2 count =" + _threadsList2.Count.ToString() + ", thread list 3 count =" + _threadsList3.Count.ToString())
'MessageBox.Show("finished - flag=" + flag.ToString())
'After all Cycles Complete
'startButton.Enabled = True
'abortButton.Enabled = False
'skipButton.Enabled = False
'DataGridViewDLOrPing.Enabled = True
'DataGridViewUL.Enabled = True
'protocolComboBox.Enabled = True
'modelComboBox.Enabled = True
'testTypeComboBox.Enabled = True
'measurementComboBox.Enabled = True
'cycleNumericUpDown.Enabled = True
'DelayNumericUpDown.Enabled = True
'InputBoxPrivilege()
End Sub
This is the function that Thread will call to read the bytes :
Public Sub DownloadBytes(ByVal p_fileValues As FileValues)
Dim httpWebRequest As HttpWebRequest
Dim httpWebResponse As HttpWebResponse
Dim responseStream As Stream
Dim threadName = threadID
Try
Console.WriteLine("Start " + Thread.CurrentThread.Name)
httpWebRequest = DirectCast(WebRequest.Create(p_fileValues.URL), HttpWebRequest)
'httpWebRequest.ProtocolVersion = HttpVersion.Version11
httpWebRequest.KeepAlive = False
httpWebRequest.AddRange(p_fileValues.StartPoint, p_fileValues.EndPoint)
httpWebRequest.Credentials = CredentialCache.DefaultCredentials
httpWebRequest.Proxy = WebRequest.DefaultWebProxy
httpWebResponse = CType(httpWebRequest.GetResponse(), HttpWebResponse)
responseStream = httpWebResponse.GetResponseStream()
Dim bytesSize As Integer = 0
' A buffer for storing and writing the data retrieved from the server
Dim downBuffer As Byte() = New Byte(2047) {}
Dim bytesAlreadyDownloaded As Int64 = p_fileValues.StartPoint
' Loop through the buffer until the buffer is empty
While (True)
'end while loop if the Abort button is clicked
If (isActionAborted = True) Then
Exit While
End If
'currentCycle is > total Cycles ,then end while
If (currentCycleDownload > cycleNumericUpDown.Value) Then
Exit While
End If
bytesSize = responseStream.Read(downBuffer, 0, downBuffer.Length)
bytesAlreadyDownloaded += bytesSize
'speedtimer.Start()
If (bytesSize <= 0) Then
If (downloadedBytes < totalBytesToDownload) Then
Me.Invoke(New UploadProgressCallback(AddressOf Me.UpdateDownloadProgress), New Object() {(totalBytesToDownload - downloadedBytes), totalBytesToDownload})
End If
Exit While
End If
' Invoke the method that updates the form's label and progress bar
Me.Invoke(New DownloadProgressCallback(AddressOf Me.UpdateDownloadProgress), New Object() {bytesSize, TotalBytesToDownloadForEachCycle})
If (bytesAlreadyDownloaded > p_fileValues.EndPoint) Then
'Console.WriteLine("Downloading part files Exit " + p_fileValues.StartPoint.ToString() + "," + p_fileValues.EndPoint.ToString())
Exit While
End If
End While
responseStream.Flush()
responseStream.Close()
httpWebResponse.Close()
responseStream.Dispose()
responseStream = Nothing
httpWebResponse = Nothing
Console.WriteLine("End " + Thread.CurrentThread.Name)
'ProgressDownload.Value = Convert.ToInt32((downloadedBytes * 100) / totalBytesToDownload)
' downloadedBytesTextBox.Text = downloadedBytes.ToString("N", nfi)
Catch ex As Exception
Console.WriteLine("Error " + Thread.CurrentThread.Name)
Console.WriteLine(ex)
Finally
Console.WriteLine("Finally " + Thread.CurrentThread.Name)
GC.Collect()
' GC.WaitForPendingFinalizers()
'newThread.Abort()
'Try
'Thread.CurrentThread.Abort()
'Catch ex1 As Exception
'End Try
End Try
'Return 1
End Sub
This function will updates the progress bars and the other labels on the form :
Private Sub UpdateDownloadProgress(ByVal BytesRead As Int64, ByVal TotalBytes As Int64)
If Not swDL Is Nothing AndAlso swDL.IsRunning Then
swDL.Stop()
If swDL.ElapsedMilliseconds > 0 Then
resultGrid.Rows.Item(HandoverGridCounter - 1).Cells(4).Value &= "DL: " & swDL.ElapsedMilliseconds & " ms"
End If
swDL = Nothing
End If
If (dlCurrentspeed > 0) Then
'txtCurrentSpeedDL.Text = Math.Round((dlCurrentspeed / 1024), 0) & " KB/s"
End If
downloadedBytes += BytesRead
If downloadedBytes >= totalBytesToDownload Then
downloadedBytes = totalBytesToDownload
End If
ProgressDownload.Value = Convert.ToInt32((downloadedBytes * 100) / TotalBytes)
downloadedBytesTextBox.Text = downloadedBytes.ToString("N", nfi) ' & " bytes"
If totalBytesToDownload = 0 Then
totalBytesToDownload = TotalBytes
txtBytesToDownload.Text = totalBytesToDownload.ToString("N", nfi)
End If
If downloadedBytes >= totalBytesToDownload Then
dlCurrentspeed = 0
dlStopWatch.Stop()
testEndTickDownload = My.Computer.Clock.TickCount
testDeltaDownload = (testEndTickDownload - testStartTickDownload) / 1000
If DLtimedOut = True Then
DownloadCompleted(TestStatus.Cancelled.ToString(), "")
Else
DownloadCompleted(TestStatus.Completed.ToString(), "")
End If
If currentCycleDownload <= cycleNumericUpDown.Value Then
If protocolComboBox.Text = "HTTP" Then
StartHttpDownload()
Else
StartFtpDownload()
End If
End If
End If
End Sub
Upvotes: 1
Reputation: 50672
Because all these threads want to pass data to the UI thread to show the progress, the UI thread is becoming the bottleneck of your application.
You could add a timer that every now and then causes the UI thread to inspect the progress of the jobs.
This way the UI thread is not driven by the huge number of threads but by time.
Post some code if you need more details.
Upvotes: 2