CODEWITHSUNDEEP
javascriptpythoncompression
user984003
user984003

Reputation: 29597

Javascript compression in memory. Decompress in Python after Post. Must also handle non-ascii

I need to be able to compress a string in Javascript, but without saving a temporary file. I am then going to send this compressed data via a POST. I will receive it in Python so I need to be able to decompress it there. I implemented the following, http://rosettacode.org/wiki/LZW_compression, only to discover that it only works on ascii-characters. I am going to be reading webpages and never know what characters I'll be getting.

(The reason I need to do this is because the strings can become quite long and therefore take too long for slow networks to post.)

Upvotes: 1

Views: 1067

Answers (3)

Mark Adler
Mark Adler

Reputation: 112502

Use deflate in javascript and zlib in python. (LZW is ancient and obsolete -- modern methods are much better.) In between use base 85 encoding, picking 85 ASCII characters than experimentation or standards documentation indicate can make it through POST unscathed. Base 85 is simply where each character is a digit in a base 85 number, where five such digits encode 32 bits.

Upvotes: 0

LSerni
LSerni

Reputation: 57418

You can try base64-encoding the string beforehand (this will yield a compressed stream from 1.5 to twice the size it would have if it had been possible to compress it directly).

There is another implementation (this of gzip Deflate algorithm) here.

Or you might try and escape the non-ASCII characters by replacing them with \xNN (NN = hex code of character). Of course you will also have to escape the slash .

Anyway, you are unlikely to achieve more than about a 10X increase in speed, and I fear this would be more than balanced by the encoding overhead. Without knowing more about the use case, I'd suggest going with Deflate.

Upvotes: 2

Jon Clements
Jon Clements

Reputation: 142216

From OP comment.

The javascript reads the DOM elements and sends that. It won't work to point to the source for various reasons, a main one being that I need the elements created by javascript on the page. I also need the computed style that the browser calculates for me.

One solution would be to automate a browser using Selenium with Python and then retrieve the DOM from that.

Upvotes: 0

Related Questions