how to handle https request response on native socket asynchronously in C++

Question

In my C++ application, I have implemented "Reactor" design pattern using epoll. I have the Reactor class where I register all Fd (socketFd/EventFd/TimerFd) and respective handler functions. When the application starts, I run the Reactor's event loop.

In this application, I have to send https request and process the https response received. Right now I am using CurlPP (libcurl c++ wrapper) to send this https request. But these are synchronous and blocking call. It is hampering my performance.

Now I want to handle these https request/response asynchronously using my Reactor class. For that, I will be creating native socket and connecting to http server. Register this socketFd and respective handler function in Reactor.

To send https request on this native socket, I need some library which will take https payload as input and give me back https (SSL encrypted) http request. So that I can write this https request directly on native socket.

Same thing to handle the https response. The read/recv call on socket will return me the buffer. I need some library which will take this buffer and give me the received https payload.

I did check the APIs available in libcurl. But libcurl do not give such feature. It does have multi handle feature, but the control of socket there is with libcurl itself.

Can you please suggest any library which can provide me such feature? Also, if you have any other approach to handle these https requests asynchronously then please suggest that also.

Answer 1

To send https request on this native socket, I need some library which will take https payload as input and give me back https (SSL encrypted) http request. So that I can write this https request directly on native socket.

Same thing to handle the https response. The read/recv call on socket will return me the buffer. I need some library which will take this buffer and give me the received https payload.

No. This will cause you massive pain and will never work correctly. SSL is not a block encryption protocol. It's a stream protocol. You will never be able to get a working implementation this way. You may get one that seems to work most of the time, but then you'll find that it breaks when, for example, a local send triggers an SSL renegotiation.

You must treat SSL as a black box that you must never, ever try to look through. Because SSL connects two bidirectional stream protocols, the black box has four (notional) ports:

1. It has an "encrypted data in" port. When you receive data from the socket, you must give it to this port.

2. It has an "encrypted data out" port. When data is sent from this port, you must send it to the socket.

3. It has a "plaintext in" port. When you want to send data to the other side, you must send it to this port.

4. It has a "plaintext out" port. When data is decrypted, it is given to you on this port.

You must treat these ports as independent and you must not assume that, for example, after you give data to the "plaintext in" port, data will come out the "encrypted data out" port. Why? Because the engine might still be waiting for a renegotiation to complete. You must not assume that because you gave data to the "encrypted data in" port, that means data will come out the "plaintext out" port. Why? Because it could be negotiation data or it could be part of a block.

You are thinking about the problem wrong. Go back to the drawing board, keeping in mind that you are implementing a bidirectional byte-stream protocol that is layered on top of another bidirectional byte-stream protocol.

This is knowledge earned through decades of painful experience -- ignore it at your peril.

Answer 2

Quick-and-dirty way:

One way would be to make these requests in a separate thread(s) using the same libcurl (with libcurl thread reporting reply back to main thread, where reply is processed). If performance becomes an issue, threadpool should help. That being said, such threading approach won't provide you with absolutely-best performance.