CODEWITHSUNDEEP
javascriptnode.jsgzip
Get Off My Lawn
Get Off My Lawn

Reputation: 36299

node gzip unzip file into variable

I am using gzip to zip a json string and then I save it to a file like this:

fs.writeFile(`${craneProjectDir}/tree.out`, JSON.stringify(workspaceTree), (err) => {
    var gzip = zlib.createGzip();
    var inp = fs.createReadStream(`${craneProjectDir}/tree.out`);
    var out = fs.createWriteStream(`${craneProjectDir}/tree`);
    inp.pipe(gzip).pipe(out);
    fs.unlinkSync(`${craneProjectDir}/tree.out`);
});

It seems to work because it creates the file, but I am not sure because I cannot read it back into a variable. This is what I have tried doing:

var treeStream = fs.createReadStream(`${craneProjectDir}/tree`);
zlib.unzip(treeStream, (err, buffer) => {
    if(err){
        connection.console.log((util.inspect(err, false, null)));
    }
    workspaceTree = JSON.parse(buffer.toString());
});

I am getting the follow error:

[TypeError: Invalid non-string/buffer chunk]

Upvotes: 2

Views: 2726

Answers (2)

Get Off My Lawn
Get Off My Lawn

Reputation: 36299

I found out that I could just use readFile, and put its contents into a Buffer like so:

fs.readFile(`${craneProjectDir}/tree`, (err, data) => {
    var treeStream = new Buffer(data);
    zlib.unzip(treeStream, (err, buffer) => {
        if (err) {
            connection.console.log((util.inspect(err, false, null)));
        }
        workspaceTree = JSON.parse(buffer.toString());
        notifyClientOfWorkComplete();
    });
});

Upvotes: 0

user4466350
user4466350

Reputation:

The problem is maybe that you delete the source file before the content has been fully written on disk.

fs.writeFile(`${craneProjectDir}/tree.out`, JSON.stringify(workspaceTree), (err) => {
    var gzip = zlib.createGzip();
    var inp = fs.createReadStream(`${craneProjectDir}/tree.out`);
    var out = fs.createWriteStream(`${craneProjectDir}/tree`);
    inp.pipe(gzip).pipe(out).on('close', function () {;
      fs.unlinkSync(`${craneProjectDir}/tree.out`);
    });
});

Take care to bind close and not end. close means that the underlying file descriptor is closed, end means the stream has finish to process the data.

When extracting the new file, you have made two mistakes,

  • you used zlib.unzip, but the content was gzipped, so you must use one of gunzip methods.
  • you have used zlib.unzip which expects a buffer but you sent it a stream.

To gunzip the content,

  var d = '';
  fs.createReadStream(`${craneProjectDir}/tree`)
  .pipe(zlib.createGunzip())
  .on('data', function (data){
    d += data.toString()
  })
  .on('end', function (){
    console.log(d);
  })

BTW, i suspect you don t need this temp file ;)

Upvotes: 5

Related Questions