CODEWITHSUNDEEP
pythonbashimagejpeg
mmz
mmz

Reputation: 1151

How to reduce file size of millions of images?

I have a few million images stored as jpgs. I'd like to reduce the size of each jpg by 80%. Here's a bash loop that I'm currently using (I'm on MacOS):

for i in *jpg; do convert "$i" -quality 80% "${i%.jpg}.jpg"; done;

The above line converts the images sequentially. Is there a way to parallelize and thus speed up this conversion? I don't need to use bash, just want to find the fastest way to make the conversion.

Upvotes: 1

Views: 382

Answers (3)

Ole Tange
Ole Tange

Reputation: 33748

Using GNU Parallel it looks like this:

parallel convert {} -quality 80% {.}_80.jpg ::: *jpg

If the million jpgs are in the same dir, the above line will be too long. Then try:

printf '%s\0' *.jpg | parallel -0 convert {} -quality 80% {.}_80.jpg

Upvotes: 0

Cyrus
Cyrus

Reputation: 88929

Parallelise the execution of convert with GNU xargs. This will run 10 convert processes simultaneously and restart more processes if less than 10 are running simultaneously until 10 are running simultaneously again.

printf "%s\n" *.jpg | xargs -P 10 -I {} convert {} -quality 80% {}

xargs replaces all {} in convert command with the file name that comes from stdin.

I assume that your file names do not contain a line break. The original files are overwritten.

Upvotes: 0

Omar Aflak
Omar Aflak

Reputation: 2962

Using Python you can do it this way:

import glob
import shlex
import subprocess
from tqdm.contrib.concurrent import thread_map

def reduce_file(filepath):
    output = f"{filepath}_reduced.jpg"
    cmd = f"convert {filepath} -quality 80% {output}"
    subprocess.run(shlex.split(cmd))

list(thread_map(reduce_file, glob.glob("./images/*.jpg")))

Given that your images are in images/*.jpg.

Upvotes: 1

Related Questions