CODEWITHSUNDEEP
httphaskellhttpshttp-conduitnetwork-conduit
user2512324
user2512324

Reputation: 791

Haskell simpleHTTP of Network.HTTP.Conduit performing slowly for get requests

In my haskell code I've imported Network.HTTP.Conduit as

import Network.HTTP.Conduit

In my main function I make a simple GET request using simpleHTTP

main = do
     response <- simpleHttp "https://github.com/trending?l=ruby"
     return ()

It took 6 minutes 42 secs to complete 100 api requests

time for NUM in `seq 1 1 100`; do ./Testhttp; done
real    6m42.839s
user    0m12.115s
sys 0m2.652s

whereas the ruby alternative just takes 153 secs for 100 api calls using Net::HTTP.get(URI.parse("https://github.com/trending?l=ruby"))

Am I doing something wrong in my haskell code? What are the performant and efficient alternatives to simpleHTTP?

Upvotes: 1

Views: 821

Answers (2)

Reid Barton
Reid Barton

Reputation: 15009

My guess would be that the Haskell library you are using is doing something like an IPv6 DNS request that times out before falling back to IPv4, and Go and Ruby are doing IPv4 requests directly. A few seconds per request is a likely DNS timeout duration and doesn't have any other possible explanation that I can see.

Upvotes: 1

Taylor Fausak
Taylor Fausak

Reputation: 1106

The documentation for simpleHttp says:

Note: This function creates a new Manager. It should be avoided in production code.

Your code creates a new manager for each request. If you change it to reuse a single manager, it could be a lot faster. You can use newManager to create a manager. For example:

import Network.HTTP.Conduit
main = do
    request <- parseUrl "https://github.com/trending?l=ruby"
    manager <- newManager conduitManagerSettings
    _response <- httpLbs request manager
    return ()

Upvotes: 1

Related Questions