CODEWITHSUNDEEP
websockethaproxysockjs
ChrisG
ChrisG

Reputation: 149

Websocket handshake hangs with HAProxy

With the latest Chrome browser, I am trying to use sockjs client to communicate with a backend server which is behind haproxy. On my localhost (without haproxy in the middle), this is working fine - the client can connect, send and receive messages using websocket protocol. For example:

conn.onopen = function() {
    if (conn.readyState === SockJS.OPEN) {
        conn.send("hello server");
        console.log("msg sent");
    }
};

Once I deploy it on the server with HAProxy, strange thing happens that sockjs thinks the connection is open (as in conn.readyState === SockJS.OPEN and 'msg sent' appears in the console log), however, websocket handshake simply hangs and msg is never received by the server. Below is what I see in the haproxy log:

Oct 23 09:08:25 localhost.localdomain haproxy[14121]: 129.xx.xxx.105:55000 [23/Oct/2012:09:08:24.459] public www/content 777/0/0/1/778 200 375 - - ---- 3/3/0/1/0 0/0 "GET /sockjs/info HTTP/1.1"
Oct 23 09:10:54 localhost.localdomain haproxy[14121]: 129.xx.xxx.105:55015 [23/Oct/2012:09:08:25.398] public www/content 0/0/0/1/149017 101 147 - - CD-- 4/4/0/0/0 0/0 "GET /sockjs/478/kyi342s8/websocket HTTP/1.1"

Notice that the second log msg only appears when I shutdown the backend server behind haproxy. Before the shutdown, no error in the log but the websocket handshake doesn't complete and no msg is received by the server.

Using the Chrome's developer tools, in the Network tab, I see the following:

Request URL:ws://www.mysite.com/sockjs/478/kyi342s8/websocket
Request Method:GET
Status Code:101 Switching Protocols

Request Headers
Connection:Upgrade
Host:www.mysite.com
Origin:http://www.mysite.com
Sec-WebSocket-Extensions:x-webkit-deflate-frame
Sec-WebSocket-Key:TFEIKYhlqWWBZKlXzXAuWQ==
Sec-WebSocket-Version:13
Upgrade:websocket
(Key3):00:00:00:00:00:00:00:00

Response Headers
Connection:Upgrade
Sec-WebSocket-Accept:D+s3va02KH6QTso24ywcdxcfDgM=
Upgrade:websocket
(Challenge Response):00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00

And both 'Type' and 'Time Latency' of the websocket object show 'Pending' under the network tab of developer tools.

Finally, this is my haproxy config (version 1.4.22):

global
        log 127.0.0.1   local1 info
        log 127.0.0.1   local1 notice
        #log loghost    local0 info
        maxconn 4096
        chroot /usr/share/haproxy
        uid 99
        gid 99
        daemon
        #debug
        #quiet

defaults
        log             global
        mode            http
        option          httplog
        option          dontlognull
        retries         3
        option          redispatch
        maxconn         500
        timeout connect 6s

frontend public
        mode    http
        bind    *:80
        timeout client  300s
        option  http-server-close
        #option         http-pretend-keepalive
        # define ACLs
        acl host_static hdr_beg(host) -i static. data.
        acl host_www hdr_beg(host) -i www.
        acl url_static path_end .ico .txt .pdf .png .jpg .css .js .csv
        acl is_stats path_beg /haproxy/stats
        # define rules
        use_backend nginx if host_static or host_www url_static
        use_backend stats if is_stats
        default_backend www

backend nginx
        timeout server 20s
        server nginx 127.0.0.1:8484

backend stats
        stats enable
        stats uri /haproxy/stats

backend www
        timeout server 300s
        option forwardfor
        #no option httpclose
        option http-server-close
        server sockcontent 127.0.0.1:8080

Does anyone know why this is happening? Is it due to some haproxy config, or even some general network settings of the server (e.g iptables)?

P.S. I've tried enabling http-pretend-keepalive in haproxy, but it does not work.

Upvotes: 5

Views: 7428

Answers (3)

Marek
Marek

Reputation: 3509

Like @Joes said, it's a fault of misbehaving firewalls. Some say that for example FortiGate causes that.

More discussion: https://github.com/sockjs/sockjs-client/issues/94

Serving SockJS over SSL seems to solve the problem.

Upvotes: 2

Joes
Joes

Reputation: 1818

Most likely, you have transparent firewall in your network and it messes up websocket connections going to port 80.

To verify that's the case:

  1. Make haproxy listen on different port and try if it started working;
  2. Try accessing your service outside of your network.

If it starts working, then there's something wrong with your network.

Unfortunately, SockJS won't use fallback transport in this case, as client thinks it connected, but connection was not really established.

As possible solution: make haproxy listen on two ports, on port 80 for your Web traffic and different port for SockJS traffic. This guarantees that transparent HTTP proxies won't mess with your websocket connections.

Upvotes: 1

Willy Tarreau
Willy Tarreau

Reputation: 3424

I have already encountered this case, I don't remember what server it was, but it was not totally compliant with the WebSocket spec. Indeed, it failed because it found the "close" token in the Connection header, along with the "Upgrade" token, while the spec says that the "Upgrade" token is needed and (fortunately) does not suggest to reject other ones.

In theory, if you use "option http-pretend-keep-alive" as you have commented, it should work. At least it did for me. But maybe there is a different issue here.

Upvotes: 1

Related Questions