CODEWITHSUNDEEP
sslhaproxyamazon-elb
Ian Atkin
Ian Atkin

Reputation: 6346

Weird behavior/error trying to use HAProxy to forward requests to ELBs

We have an architecture where have several APIs on multiple hosts. Each API sits behind an AWS ELB. We want to put a proxy in front that will route requests based on URI to the ELB.

What I have so far mostly works, but about 3 out of 10 requests result in the following error (using cURL, but the problem isn't cURL):

curl: (35) Unknown SSL protocol error in connection to test-router.versal.com:-9847

I have a feeling that the ELB is the culprit. The SSL is terminated there.

Here is our HAProxy config:

global
  log 127.0.0.1   local0
  log 127.0.0.1   local1 notice
  maxconn 10240
  user haproxy
  group haproxy
  daemon
  debug
  stats socket /var/run/haproxy.sock
  log-send-hostname test-router.domain.com
  description "HAProxy on test-router.domain.com"

defaults
  log     global
  mode    http
  option  httplog
  option  dontlognull
  retries 3
  option redispatch
  option forwardfor
  option httpclose
  option dontlognull
  option tcpka
  maxconn 10240
  timeout connect 10000ms
  timeout client 600000ms
  timeout server 600000ms

frontend public
  bind *:80
  bind *:443
  acl elb0 path_beg /elb0
  acl elb1 path_beg /elb1
  use_backend elb0 if elb0
  use_backend elb1 if elb1

  bind 0.0.0.0:443 ssl crt /etc/ssl/cert.pem no-sslv3 ciphers AES128+EECDH:AES128+EDH

backend elb0
  server server_vcat elb0.domain.com:443 ssl verify none

backend elb1
  server server_laapi elb1.domain.com:443 ssl verify none

Upvotes: 0

Views: 1302

Answers (1)

Michael - sqlbot
Michael - sqlbot

Reputation: 178974

The SSL from curl is not terminated at the ELB. It is terminated at HAProxy, in your configuration...

bind 0.0.0.0:443 ssl crt /etc/ssl/cert.pem no-sslv3 ciphers AES128+EECDH:AES128+EDH

...and then an entirely different SSL session is established by HAProxy on the connection to the ELB:

server ... ssl verify none

An SSL problem with ELB could not possibly be propagated back to curl through HAProxy in this configuration.

The problem is in your HAProxy configuration, here:

bind *:443

Remove that line. It's redundant (and incorrect).

You are telling HAProxy to bind to port 443 twice: once speaking SSL, and once not speaking SSL.

So, statistically, on approximately 50% of your connection attempts, curl finds HAProxy is not speaking SSL on port 443 -- it's just speaking HTTP, and curl can't (and shouldn't) handle that gracefully.

I believe this (mis)configuration goes undetected by HAProxy, not because of an actual bug, but rather because of the way some things are implemented in the HAProxy internals, related to multi-process deployments and hot reloads, in which case it would be valid to have HAProxy bound to the same socket more than once.

Upvotes: 1

Related Questions