CODEWITHSUNDEEP
sslproxy
Sabya
Sabya

Reputation: 11894

Decoupling SSL load from a curl client

I have a syncer process and it makes curl HTTPS calls to Twitter, Facebook, etc to gather data for our users and update the DB (stored in local file system). There is a HTTP server running on the same system which queries this db and returns results to the user when the request.

The problem is that the sycer process is eating up too much CPU/IO due to SSL negotiation happening while calling Twitter, Facebook, etc APIs. This is making user experience a bit slow. So I want this SSL negotiations to happen on another system from where the HTTP server is running. And I can not take the syncer process outside because it needs local file system access to update the DB.

So I was thinking of building a kind of proxy server. The syncer process will make HTTP calls to the proxy server. The proxy server will make HTTPS calls to Twitter, Facebook, etc and returns the results to the syncer process.

Is any software anything existing for this? I don't want to reinvent the wheel if something is already there.

Upvotes: 1

Views: 248

Answers (2)

Sabya
Sabya

Reputation: 11894

I ended up writing a simple proxy server using nodejs. Clients which need to call HTTPS apis will pass all the details in JSON format to this proxy server. This proxy server will in turn call HTTPS apis and return the result to clients.

Upvotes: 0

dnet
dnet

Reputation: 1429

Stunnel does almost exactly what you ask if the following conditions are met.

  • One stunnel instance must be running for each service (either from inetd or in standalone daemon-like mode).
  • The system running cURL must manually resolve the hostname of the target service to the IP address of the host running stunnel (e.g. /etc/hosts) in order to use it.

I used stunnel in a similar way for connecting to IRC servers using SSL with SSL-capable clients. It works like a charm, one important thing to be careful about is the incompatibilities between the 3.x and newer versions. If you're planning for a new system, you should probably use the new ones.

Upvotes: 0

Related Questions