Steven Kull
Steven Kull

Reputation: 161

Bz2 every file in a dir

I am running centos and I have around 1,300 files in a folder that each need to be bzipped individually. What would be the easiest way of handing this?

Upvotes: 12

Views: 22656

Answers (5)

sagi
sagi

Reputation: 5747

If all the files are in a single directory then:

bzip2 *

Is enough. A more robust approach is:

find . -type f -exec bzip2 {} +

Which will compress every file in the current directory and its sub-directories, and will work even if you have tens of thousands of files (using * will break if there are too many files in the directory).

If your computer has multiple cores, then you can improve this further by compressing multiple files at once. For example, if you would like to compress 4 files concurrently, use:

find . -type f -print0 | xargs -0 -n1 -P4 bzip2

In case you get a permission denied error, put another sudo before xargs:

find . -type f -print0 | sudo xargs -0 -n1 -P4 bzip2

Upvotes: 27

vikrant rana
vikrant rana

Reputation: 4679

I have written below script to bzip2 files to another directory

#!/bin/bash
filedir=/home/vikrant_singh_rana/test/*

for filename in $filedir; do
name=$(basename "$filename" | sed -e 's/\.[^.]*$//')
bzip2 -dc $filename > /home/vikrant_singh_rana/unzipfiles/$name
done

my sample file name was like

2001_xyz_30Sep2020_1400-30Sep2020_1500.csv.bz2

I was not able to get any direct command, hence made this. This is working fine as expected.

Upvotes: 1

Shaheen Ghiassy
Shaheen Ghiassy

Reputation: 7517

To bzip2 on a multi-core Mac, you can issue the following command (when you're inside the folder you want to bzip)

find . -type f -print0 | xargs -0 -n1 -P14 /opt/local/bin/bzip2

This will bzip every file recursively inside the folder your terminal is in using 14 CPU cores simultaneously.

You can adjust how many cores to use by editing

 -P14

If you don't know where the bzip2 binary is, you can issue the following command to figure it out

which bzip2

The output of that command is what you can replace

/opt/local/bin/bzip2

with

Upvotes: 3

user405725
user405725

Reputation:

find PATH_TO_FOLDER -maxdepth 1 -type f -exec bzip2 -zk {} \;

Upvotes: 0

Carlos Campderrós
Carlos Campderrós

Reputation: 22982

From inside the directory: bzip2 *

Upvotes: 0

Related Questions