CODEWITHSUNDEEP
httpf#
N0ug4t
N0ug4t

Reputation: 191

Cannot download attachment file - cURL works just fine - Missing a header?

I want to download a file - I can get the response body with cURL just fine. However, with fsharp HttpClient I get an empty response body and no file. I have looked around and have no idea why this request is not getting a good response back

Here's my fsharp minimally reproducible example:

// Program.fs
open System
open System.IO
open System.Net.Http

let getAsync (client:HttpClient) (url:string) = 
  async {
    let! response = client.GetAsync(url) |> Async.AwaitTask
    response.EnsureSuccessStatusCode () |> ignore
    let! content = response.Content.ReadAsStringAsync() |> Async.AwaitTask
    printfn "%O" response
    printfn "%s" content
    return content
  }

[<EntryPoint>]
let main argv =
  use httpClient = new HttpClient()
  async {
    let! csv = getAsync httpClient "https://huskers.com/calendar.ashx/calendar.csv?sport_id=27"
    printfn "%s" csv
  } |> Async.RunSynchronously
  0 // return an integer exit code

fSharp output:

StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
  Server: Microsoft-IIS/8.5
  Date: Sat, 20 Feb 2021 01:05:19 GMT
  Content-Length: 0
}

cURL output:

$ curl -v https://huskers.com/calendar.ashx/calendar.csv?sport_id=27
### SSL Handshake omitted for brevity ###
> GET /calendar.ashx/calendar.csv?sport_id=27 HTTP/1.1
> Host: huskers.com
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Type: text/csv
< Server: Microsoft-IIS/8.5
< Content-Disposition: attachment; filename=Add2Calendar.csv
< Date: Sat, 20 Feb 2021 01:06:04 GMT
< Content-Length: 2248
< 
Event,Start Date,Start Time,End Date,End Time,Location,Category,Description,Facility
"Wrestling vs Minnesota","1/8/2021","8:00PM","1/8/2021","11:00 PM","Lincoln, Neb.","
... Rest of response here ...

Upvotes: 2

Views: 194

Answers (1)

Brian Berns
Brian Berns

Reputation: 17038

That particular server requires you to specify a user agent:

client.DefaultRequestHeaders.Add("User-Agent", "F# rocks!")
let! response = client.GetAsync(url) |> Async.AwaitTask

Upvotes: 2

Related Questions