CODEWITHSUNDEEP
multithreadingasynchronousf#
reborn programmer
reborn programmer

Reputation: 514

F# spawn a new async request every second

My goal (before I expand on it) is to start up an Async task (in this case, fetching a URL) once a second. The idea is that if the Async task takes 3 seconds, it would behave like this:

0sec: Start thread 0
1sec: Start thread 1
2sec: Start thread 2
3sec: Start thread 3
3sec: Finish thread 0
4sec: Start thread 4
4sec: Finish thread 1

Here's my code so far. The workTime and Async.Sleep code was added to make my problem more apparent.

let FetchUrlAsync (thread: int) (url: string) =
    async {
        let req = WebRequest.Create(url)
        let! resp = req.AsyncGetResponse()
        use stream = resp.GetResponseStream()
        use reader = new System.IO.StreamReader(stream)

        let workTime = System.Random().Next(5000,8000)
        sprintf "start work %i" thread |> Console.WriteLine
        do! Async.Sleep workTime
        sprintf "end work %i (time taken: %i)" thread workTime |> Console.WriteLine
        return reader.ReadToEnd()
    }

Seq.initInfinite(fun i -> FetchUrlAsync i "http://www.google.com"
                          |> Async.StartAsTask
                          |> ignore
                          Thread.Sleep 1000
                )
|> Seq.iter(id)

The issue is that there seems to be a maximum of two threads running at the same time. Why is this happening and what am I doing wrong?

Also, this is my first time doing anything with async. Please let me know if you see any way to improve my code (is using Thread.Sleep the right thing to do here?)

Upvotes: 2

Views: 157

Answers (1)

Ringil
Ringil

Reputation: 6527

The problem is System.Net.ServicePointManager.DefaultConnectionLimit is by default set to 2. Raise its value higher with something like this before your requests:

System.Net.ServicePointManager.DefaultConnectionLimit <- 10

Upvotes: 4

Related Questions