stigzler
stigzler

Reputation: 993

Baffled by Async Await for HttpClient.GetStringAsync

I'm trying to design an approach to downloading XML delivered by a web API. The API allows for multi-threading - so in my usage case, I can use 7 threads at once (I don't understand the technicalities - is this 7 sockets or simultaneous connections?). I haven't factored this into my code below, but would just use a parallel foreach for the 7 simultaneous httpclient queries.

I'm really struggling to understand Await Async and most tutorials seems to be in c#. I'm afraid I'm vb (I do know you can convert, but ruins the learning a little).

I've also read that you shouldn't open up a new instance of httpClient for each GetStringAsync (e.g. via a using statement) and should instead have a global instance and reuse that.

So, two problems - how to design the HttpClient helper class and how to utilise it.

On the first, I have gotten to this stage:

Imports System.Net.Http

Public Class HttpClientHelper

    Private Shared ReadOnly _httpClient As HttpClient = New HttpClient()

    Public Async Function SendRequestToEndpoint(url As String) As Task(Of String)

        Dim retString As String = Await _httpClient.GetStringAsync(url)

        Return retString

    End Function

End Class

On the second, I just got lost! Totally baffled by Await/Async. so can't even get it to compile:

Private Async Sub HttpClientTest(urls As List(Of String))

    'HTTPCLIENT TESTS
    Dim hch As New HttpClientHelper
    Dim MainThreadString As String = Nothing
    For i = 0 To urls.Count - 1

        Await Task.Run(Function()
                           Dim retStr As String = hch.SendRequestToEndpoint(urls(i))
                           'I need to perform actions on main thread variables with retStr - e.g. updating a database
                           MainThreadString = retStr
                       End Function)
        Debug.WriteLine("URL: " & urls(i) & vbCr & MainThreadString)

    Next

End Sub

I just pass in a list of URLs to this method.

This is just a simplified version of the final process. I would essentially be passing the returned xml into another method to update the main database on the main thread.

Sorry if this doesn't make sense, I'm just really struggling to get Async theory!

=========================================================

UPDATE: Following Caius Jard's answer, I updated my code to what I thought would be logical:

Public Class HttpClientHelper

    Private Shared ReadOnly _httpClient As HttpClient = New HttpClient()
    
    Public Async Function SendRequestToEndpoint(url As String) As Task(Of String)

        Dim retString As String = Await _httpClient.GetStringAsync(url)
        Return retString

    End Function

End Class

Public Class 

    Private Sub Main()

        Dim urls As New List(Of String) From {"https://www.google.com", "https://www.bbc.co.uk", "https://stackoverflow.com"}
        HttpClientTest(urls)
        
    End Sub
    
    Private Async Sub HttpClientTest(urls As List(Of String))

        Dim hch As New HttpClientHelper
        For i = 0 To urls.Count - 1

            Dim str As String = Await hch.SendRequestToEndpoint(urls(i))
            Debug.WriteLine(vbCr & vbCr & "URL: " & urls(i) & vbCr & str)

        Next
    End Sub
    
End Class

However, there are still problems:

  1. Nothing is written to the debug console.
  2. Also, using the debugger yields some weird stepping, which I assume is a result of trying to step through asynchronous code?
  3. How come you have to also mark HttpClientTest as async as essentially this method only calls an async Function and isn't async in itself (or does the very act of calling an async method within, albeit with await, render it as async as well?)

I did try:

Await HttpClientTest(urls)

but it wouldn't compile...

Pesky async...

Upvotes: 0

Views: 761

Answers (1)

Caius Jard
Caius Jard

Reputation: 74605

Async/Await is like:

  • You're playing Xbox
  • A friend you haven't seen for years drops in (pre COVID :) ) for coffee
  • This chat is gonna take a while, so your save your game and turn off the Xbox, save electricity etc
  • You have your chat, few hours later friend goes home
  • You go back to the Xbox, turn it on, restore the game and carry on playing where you left off

When you have a method that runs async, if you Await it then VB saves the state, sets the job going (big download, gonna take a while) and goes back to what it was doing before - probably drawing the UI, if this download was initiated from a button click handler

When the big download is done, vb comes back to where it was at the time the code started awaiting, restores the state and carries on executing the code with everything as it was, and bonus; the download is complete so the results of the Task are available

When a function/sub uses the Await keyword, it needs to be marked Async (Public Async Sub ClickHandler(...) Handles MyButton.Click - the async marking meaning you're indicating to the compiler that it will have to insert the savegame facility into the method (it writes a lot of stuff you can't see, to make all this work)

When a Function returns a Task, or a Task(Of Something) it can be Awaited. The Task represents the work, and is like a container for the result. At its simplest you could conceive that the Await keyword waits for a Task to complete, then extracts the result and returns it. It can only extract the result when it's ready. Sometimes there are tasks that don't have a result; await will wait for them to finish but not extract any result from them:

Dim downloadedString As Strin = Await httpClient.GetStringAsync(...)

Await myStringWriter.WriteStringAsync(downlaodedString) 'writes data to disk, no result

One key thing to note (and it's a little bit like Inception) is that using Await requires/causes your function to return a Task, and it's not that Task that you're awaiting; it's a different Task whose job it is to wait for the Task you're awaiting to complete and give you the result it promised. You've used Await, you've had to declare your function Async, and you've had to say it returns Task(Of String). httpCLient's GetStringAsync returned you a Task(Of String) and Awaiting it will eventually result in a string, but while the waiting is happening, a Task is created in your function (not the Task that was created in the GetStringAsybc function) that allows whoever called your function to wait for your function to finish waiting for GetStringAsync to do its work. It doesn't stack up (and become a Task(Of Task(Of String)) because Await digs the string out, then packs it into a Task(Of String) that whoever called your function will unpack, and maybe it will pack it up.. and so on.. This process of packing and unpacking is very lightweight, so you shouldn't care too much about streamlining it not to occur, and trying to save on it introduces problems elsewhere so we tend to leave it alone. If you're interested to read more on it you can read about eliding async/await

--

It's important to note that there isn't necessarily any multithreading here. To do that work in the background it might not create what you'd think of as a new thread, but that doesn't matter - it gives the impression of being multithreaded because your program doesn't hang while you wait for the download, because it isn't the UI thread doing the work; the UI thread is free to process window messages and generally everything else it normally does


Microsoft tend to name their Async methods with an Async suffix. This is not because they are declared as Async necessarily, but because they behave in an asyncronous way. Just because something is named ..Async doesn't mean you have to Await it. You could just return the Task it creates, and whatever called your method could await that returned task instead. There are other situations where you might not want to Await an ..Async function directly, but instead capture the Task it returns, and do something else before you await (or never await). Remember that Await unpacks a Task and digs the result out so you could do this:

Dim myTask = client.DownloadStringAsync(...) 'Not awaited here, myTask is a Task(Of String)
Dim x = CalculatePiTo3BillionDecimalPlacesAndReturnXWhenDone()
Dim stringResult = Await myTask 

Probably the download already finished while we were calculating pi so this await won't spend any time waiting, it just digs the string result out of the task. The important part is that the download was going on, and the pi thing was going on "simultaneously" - however it was managed internally

If you were to start 7 downloads and stash all the Tasks in an array/list etc, then you can say:

Dim mySevenTasks As New List(Of Task(Of String))
mySevenTasks.Add(client.DownloadAsync(url1))
mySevenTasks.Add(client.DownloadAsync(url2))
mySevenTasks.Add(client.DownloadAsync(..))
mySevenTasks.Add(client.DownloadAsync(url7))
Await Task.WhenAll(mySevenTasks)

Task.WhenAll will create a task whose job it is to wait for all the tasks in a collection to complete. The runtime will start doing the 7 tasks as you've asked, when you ask but only go back to its former job (e.g. drawing the UI) when it encounters your Await. When all the tasks complete, the code resumes from where it left off and you can pull your results out of the now-completed Tasks stashed in the collection. Note that you probably wouldn't do this in this fashion (starting 7 simultaneously, but you could use some semaphore scheduling mechanism to start a few, and have the others wait until some free up) but I presented this in this way to hopefully make it simple to visualize.

Upvotes: 2

Related Questions