CODEWITHSUNDEEP
node.jsgoogle-cloud-platformgoogle-cloud-storagegoogle-cloud-functions
Viento
Viento

Reputation: 119

Ungzip files from Cloud Storage via Cloud Functions

I have gzip files in my Google Cloud Storage and I have to check the checksum of the file that is gzipped inside with Cloud Functions.

I started to work with the unzip example from this example, but it only works with ZIP files not gzip:

gcsSrcObject.createReadStream()
    .pipe(unzipper.Parse())
    .pipe(stream.Transform({
        objectMode: true,
        transform: function (entry, e, callback) {
            var filePath = entry.path;
            var type = entry.type;
            var size = entry.size;
            console.log(`Found ${type}: ${filePath}`);
            console.log(`Unzipping to: ${TEMP}/${prefix}/${filePath}`)
            var gcsDstObject = dstBucket.file(`${TEMP}/${prefix}/${filePath}`);
            entry
                .pipe(gcsDstObject.createWriteStream())
                .on('error', function (err) {
                    console.log(`Error: ${err}`);
                })
                .on('finish', function () {
                    console.log('Complete');
                    callback();
                });
        }
    }));

I have also read the documentation about the native storage functions(gsutil cp), but it only let you to GZIP files in copy from local.

Upvotes: 1

Views: 2562

Answers (1)

pantulis
pantulis

Reputation: 1705

Remember that if your files are stored in GCS with "Content-Encoding: gzip" then you can decompress them on the fly transparently. GCS calls this "transcoding"

https://cloud.google.com/storage/docs/transcoding

Upvotes: 1

Related Questions