CODEWITHSUNDEEP
httpslibcurl
rincewind
rincewind

Reputation: 1183

Using libcurl from multiple threads - how to get the best performance?

My program uses libcurl to talk to a server (always the same one) over HTTPS. The easiest way is to always create a new curl handle for a request. But by doing so I'm not taking advantage of curl connection pooling/reuse, if I understand it right.

Could it make sense to create a pool of curl handles, so that any given thread could borrow a handle from, perform a request, and then put it back into the pool?

If the answer is yes, then a related question: given a curl handle, how to find out if for a new request it will reuse the connection, or will have to create a new one (hence it doesn't make sense to put it back into the pool in the scenario described above)?

Upvotes: 2

Views: 2326

Answers (1)

Daniel Stenberg
Daniel Stenberg

Reputation: 58114

You can't figure out before if a connection is there to be re-used as there's no libcurl API for that.

The reason is quite simply that there are many factors involved in judging that decision that aren't figured out until you ask for the transfer to happen - and the fact that it can still fail and then libcurl will (transparently) create a new connection anyway.

If you really must use multi-threading then I'd recommend you splitting up the requests/threads on some logic you have or know so that the different threads handle different hosts or similar, to improve the chances of connection-reuse for each handle.

If you can consider another option, using a single thread that runs the libcurl multi interface could be a way to increase your chances of connection re-use.

Upvotes: 4

Related Questions