CODEWITHSUNDEEP
node.jszip
commadelimited
commadelimited

Reputation: 5119

Need to ZIP an entire directory using Node.js

I need to zip an entire directory using Node.js. I'm currently using node-zip and each time the process runs it generates an invalid ZIP file (as you can see from this Github issue).

Is there another, better, Node.js option that will allow me to ZIP up a directory?

EDIT: I ended up using archiver

writeZip = function(dir,name) {
var zip = new JSZip(),
    code = zip.folder(dir),
    output = zip.generate(),
    filename = ['jsd-',name,'.zip'].join('');

fs.writeFileSync(baseDir + filename, output);
console.log('creating ' + filename);
};

sample value for parameters:

dir = /tmp/jsd-<randomstring>/
name = <randomstring>

UPDATE: For those asking about the implementation I used, here's a link to my downloader:

Upvotes: 182

Views: 237482

Answers (16)

therightstuff
therightstuff

Reputation: 1007

I came across @zip-js/zip-js, and I found it really easy to work with. Here's a sample from an article I wrote on it:

export async function blobToBase64(blob: Blob): Promise<string> {
    const buffer = await blob.arrayBuffer();
    return Buffer.from(buffer).toString("base64");
}

export async function blobToFile(blob: Blob, path: string): Promise<void> {
    const buffer = await blob.arrayBuffer();
    fs.writeFileSync(path, Buffer.from(buffer));
    return;
}

export async function zipFolder(sourcePath: string): Promise<Blob> {
    const blobWriter = new BlobWriter("application/zip");
    const writer = new ZipWriter(blobWriter);

    // Recursive function to walk through the directory and add files/folders to the zip
    const walk = async (dir: string, writer: ZipWriter<Blob>) => {
        const files = fs.readdirSync(dir);
        for (const file of files) {
            const fullPath = path.join(dir, file);
            const relativePath = path.relative(sourcePath, fullPath);
            if (fs.statSync(fullPath).isDirectory()) {
                // Add folder
                await writer.add(`${relativePath}/`, undefined);
                await walk(fullPath, writer); // Recursively add folder contents
            } else {
                const fileBlob = new Blob([fs.readFileSync(fullPath)]);
                // Add file
                await writer.add(relativePath, new BlobReader(fileBlob));
            }
        }
    };

    await walk(sourcePath, writer);
    await writer.close();

    return await blobWriter.getData());
}

Upvotes: 0

sachin kumar
sachin kumar

Reputation: 1

const express = require("express");

const { createGzip } = require("node:zlib");
const { pipeline } = require("node:stream");
const { createReadStream, createWriteStream } = require("node:fs");

const app = express();
app.use(express.json());
app.get("/", async (req, res) => {
  const gzip = createGzip();
  const source = createReadStream("hotellists.json");
  const destination = createWriteStream("hotellists.json.gz");

  pipeline(source, gzip, destination, (err) => {
    if (err) {
      console.error("An error occurred:", err);
      process.exitCode = 1;
    }
    
  });
});
app.listen(2020, () => console.log("server in on"));

Upvotes: 0

Ondrej Kvasnovsky
Ondrej Kvasnovsky

Reputation: 4643

I have found this small library that encapsulates what you need.

npm install zip-a-folder

const zipAFolder = require('zip-a-folder');
await zipAFolder.zip('/path/to/the/folder', '/path/to/archive.zip');

https://www.npmjs.com/package/zip-a-folder

Upvotes: 3

hfc
hfc

Reputation: 788

As today, I'm using AdmZip and works great:

const AdmZip = require('adm-zip');
export async function archiveFile() {
  try {
    const zip = new AdmZip();
    const outputDir = "/output_file_dir.zip";
    zip.addLocalFolder("./yourFolder")
    zip.writeZip(outputDir);
  } catch (e) {
    console.log(`Something went wrong ${e}`);
  }
}

Upvotes: 4

Govind Rai
Govind Rai

Reputation: 15780

Use Node's native child_process api to accomplish this.

No need for third party libs. Two lines of code.

const child_process = require("child_process");
child_process.execSync(`zip -r <DESIRED_NAME_OF_ZIP_FILE_HERE> *`, {
  cwd: <PATH_TO_FOLDER_YOU_WANT_ZIPPED_HERE>
});

The example above showcases the synchronous API. You can also use child_process.exec(path, options, callback) if you want async behavior. There are a lot more options you can specify other than cwd to further fine-tune your request.

If you don't have the ZIP utility:

This question is specifically asks about the zip utility for archiving/compression purposes. Therefore, this example assumes you have the zip utility installed on your system. For completeness sakes, some operating systems may not have utility installed by default. In that case you have at least three options:

  1. Work with the archiving/compression utility that is native to your platform

    Replace the shell command in the above Node.js code with code from your system. For example, linux distros usually come with tar/gzip utilities:

    tar -cfz <DESIRED_NAME_OF_ZIP_FILE_HERE> <PATH_TO_FOLDER_YOU_WANT_ZIPPED_HERE>.

    This is a nice option as you don't need to install anything new onto your operating system or manage another dependency (kind of the whole point for this answer).

  2. Obtain the zip binary for your OS/distribution.

    For example on Ubuntu: apt install zip.

    The ZIP utility is tried and tested for decades, it's fairly ubiquitous and it's a safe choice. Do a quick google search or go to the creator, Info-ZIP's, website for downloadable binaries.

  3. Use a third party library/module (of which there are plenty on NPM).

    I don't prefer this option. However, if you don't really care to understand the native methods and introducing a new dependency is a non-issue, this is also a valid option.

Upvotes: 51

commadelimited
commadelimited

Reputation: 5119

I ended up using archiver lib. Works great.

Example

var file_system = require('fs');
var archiver = require('archiver');

var output = file_system.createWriteStream('target.zip');
var archive = archiver('zip');

output.on('close', function () {
    console.log(archive.pointer() + ' total bytes');
    console.log('archiver has been finalized and the output file descriptor has closed.');
});

archive.on('error', function(err){
    throw err;
});

archive.pipe(output);

// append files from a sub-directory, putting its contents at the root of archive
archive.directory(source_dir, false);

// append files from a sub-directory and naming it `new-subdir` within the archive
archive.directory('subdir/', 'new-subdir');

archive.finalize();

Upvotes: 199

Jesus is Lord
Jesus is Lord

Reputation: 15399

import ... from answer based on https://stackoverflow.com/a/51518100

To zip single directory

import archiver from 'archiver';
import fs from 'fs';

export default zipDirectory;

/**
 * From: https://stackoverflow.com/a/51518100
 * @param {String} sourceDir: /some/folder/to/compress
 * @param {String} outPath: /path/to/created.zip
 * @returns {Promise}
 */
function zipDirectory(sourceDir, outPath) {
  const archive = archiver('zip', { zlib: { level: 9 }});
  const stream = fs.createWriteStream(outPath);

  return new Promise((resolve, reject) => {
    archive
      .directory(sourceDir, false)
      .on('error', err => reject(err))
      .pipe(stream)
    ;

    stream.on('close', () => resolve());
    archive.finalize();
  });
}

To zip multiple directories:

import archiver from 'archiver';
import fs from 'fs';

export default zipDirectories;

/**
 * Adapted from: https://stackoverflow.com/a/51518100
 * @param {String} sourceDir: /some/folder/to/compress
 * @param {String} outPath: /path/to/created.zip
 * @returns {Promise}
 */
function zipDirectories(sourceDirs, outPath) {
  const archive = archiver('zip', { zlib: { level: 9 }});
  const stream = fs.createWriteStream(outPath);

  return new Promise((resolve, reject) => {
    var result = archive;
    sourceDirs.forEach(sourceDir => {
      result = result.directory(sourceDir, false);
    });
    result
      .on('error', err => reject(err))
      .pipe(stream)
    ;

    stream.on('close', () => resolve());
    archive.finalize();
  });
}

Upvotes: 1

D.Dimitrioglo
D.Dimitrioglo

Reputation: 3663

I'm not going to show something new, just wanted to summarise the solutions above for those who like Promises as much as I do 😉.

const archiver = require('archiver');

/**
 * @param {String} sourceDir: /some/folder/to/compress
 * @param {String} outPath: /path/to/created.zip
 * @returns {Promise}
 */
function zipDirectory(sourceDir, outPath) {
  const archive = archiver('zip', { zlib: { level: 9 }});
  const stream = fs.createWriteStream(outPath);

  return new Promise((resolve, reject) => {
    archive
      .directory(sourceDir, false)
      .on('error', err => reject(err))
      .pipe(stream)
    ;

    stream.on('close', () => resolve());
    archive.finalize();
  });
}

Hope it will help someone 🤞

Upvotes: 137

user672770
user672770

Reputation: 11

I ended up wrapping archiver to emulate JSZip, as refactoring through my project woult take too much effort. I understand Archiver might not be the best choice, but here you go.

// USAGE:
const zip=JSZipStream.to(myFileLocation)
    .onDone(()=>{})
    .onError(()=>{});

zip.file('something.txt','My content');
zip.folder('myfolder').file('something-inFolder.txt','My content');
zip.finalize();

// NodeJS file content:
    var fs = require('fs');
    var path = require('path');
    var archiver = require('archiver');

  function zipper(archive, settings) {
    return {
        output: null,
        streamToFile(dir) {
            const output = fs.createWriteStream(dir);
            this.output = output;
            archive.pipe(output);

            return this;
        },
        file(location, content) {
            if (settings.location) {
                location = path.join(settings.location, location);
            }
            archive.append(content, { name: location });
            return this;
        },
        folder(location) {
            if (settings.location) {
                location = path.join(settings.location, location);
            }
            return zipper(archive, { location: location });
        },
        finalize() {
            archive.finalize();
            return this;
        },
        onDone(method) {
            this.output.on('close', method);
            return this;
        },
        onError(method) {
            this.output.on('error', method);
            return this;
        }
    };
}

exports.JSzipStream = {
    to(destination) {
        console.log('stream to',destination)
        const archive = archiver('zip', {
            zlib: { level: 9 } // Sets the compression level.
        });
        return zipper(archive, {}).streamToFile(destination);
    }
};

Upvotes: 0

jicler
jicler

Reputation: 823

Since archiver is not compatible with the new version of webpack for a long time, I recommend using zip-lib.

var zl = require("zip-lib");

zl.archiveFolder("path/to/folder", "path/to/target.zip").then(function () {
    console.log("done");
}, function (err) {
    console.log(err);
});

Upvotes: 5

Harsha Biyani
Harsha Biyani

Reputation: 7268

You can try in a simple way:

Install zip-dir :

npm install zip-dir

and use it

var zipdir = require('zip-dir');

let foldername =  src_path.split('/').pop() 
    zipdir(<<src_path>>, { saveTo: 'demo.zip' }, function (err, buffer) {

    });

Upvotes: 0

Dreams
Dreams

Reputation: 6122

This is another library which zips the folder in one line : zip-local

var zipper = require('zip-local');

zipper.sync.zip("./hello/world/").compress().save("pack.zip");

Upvotes: 23

caiocpricci2
caiocpricci2

Reputation: 7788

Archive.bulk is now deprecated, the new method to be used for this is glob:

var fileName =   'zipOutput.zip'
var fileOutput = fs.createWriteStream(fileName);

fileOutput.on('close', function () {
    console.log(archive.pointer() + ' total bytes');
    console.log('archiver has been finalized and the output file descriptor has closed.');
});

archive.pipe(fileOutput);
archive.glob("../dist/**/*"); //some glob pattern here
archive.glob("../dist/.htaccess"); //another glob pattern
// add as many as you like
archive.on('error', function(err){
    throw err;
});
archive.finalize();

Upvotes: 13

Raf
Raf

Reputation: 7649

To pipe the result to the response object (scenarios where there is a need to download the zip rather than store locally) 

 archive.pipe(res);

Sam's hints for accessing the content of the directory worked for me. 

src: ["**/*"]

Upvotes: 4

Sam Ghaderyan
Sam Ghaderyan

Reputation: 144

To include all files and directories:

archive.bulk([
  {
    expand: true,
    cwd: "temp/freewheel-bvi-120",
    src: ["**/*"],
    dot: true
  }
]);

It uses node-glob(https://github.com/isaacs/node-glob) underneath, so any matching expression compatible with that will work.

Upvotes: 9

Xiaoxin
Xiaoxin

Reputation: 164

Adm-zip has problems just compressing an existing archive https://github.com/cthackers/adm-zip/issues/64 as well as corruption with compressing binary files.

I've also ran into compression corruption issues with node-zip https://github.com/daraosn/node-zip/issues/4

node-archiver is the only one that seems to work well to compress but it doesn't have any uncompress functionality.

Upvotes: 4

Related Questions