CODEWITHSUNDEEP
websocketcompressiondeflate
Andrew W. Phillips
Andrew W. Phillips

Reputation: 3601

Better websocket compression using inter-message deflate

I believe that all modern browsers have support for websocket compression in the form of "permessage-deflate". However, as I understand it (and as implied by the name) this only compresses each message individually. In my case (and, I would assume, quite commonly) this is of little use as I send many small JSON messages.

Is there any current or proposed browser facility for continuous compression? My research has found the experimental websocket extensions called deflate-stream and deflate-application-data, but they seem to have died. For example, deflate-stream was removed from FireFox in 2011. This is odd as it would be very useful.

My current bottleneck is network bandwidth and I think my application could greatly benefit from continuous (ie "inter-message") compression. For example, my most common message is usually sent at least once a second and for tens of thousands of websockets this starts to overload a 100Mbit connection. It looks something like this (w/o whitespace):

{
    "Response": "i",
    "Data": {
        "AlwaysSameId": 12345678,
        "Type": 0,
        "Style": 0,
        "Name": "long.similar.names.often.the.same",
        "Time": 1532492191000000
    }
}

where most fields are always or usually the same and only the time field is always changing (but still similar). My tests shows that with this sort of data using "inter-message" compression could compress the JSON by 95% (that's 20X smaller!), but "per-message" compression gives little or no reduction (since the deflate dictionary has to be sent each time).

I could implement my own "inter-message" compression at a higher level. (I have done this sort of thing before in C using zlib 1.2.3) It might be a bit tedious though as I have to implement both ends, or at least the direction with most data - ie, compress in backend (Golang) and decompress in frontend (Javascript).

I am just after advice and clarification on what's happening with websocket compression.

Upvotes: 3

Views: 571

Answers (1)

swpalmer
swpalmer

Reputation: 4380

The specification allows for the sliding window to span messages, but also to negotiate to have it not.

See https://datatracker.ietf.org/doc/html/rfc7692#section-7.1.1

Upvotes: 0

Related Questions