CODEWITHSUNDEEP
linuxmultithreadingbashshellubuntu
Patartics Milán
Patartics Milán

Reputation: 4938

Shell script: Parallelizing commands in bash script under Ubuntu linux

I am creating a bash script to losslessly compress jpeg and png files, so I use two package under Ubuntu named jpegoptim and optipng. A will use the script with 2 or 4 core processors. I would like to use the full capacity of my CPU.

The problem is that optipng does not support multithreading by default (it only uses one CPU core), so I decided, I launch 2 or 4 parallel processes to compress the images faster. I already sorted the image files to 4 almost equal arrays (based on number of pixels), and now I need to run the processes parallelized.

I am trying parallelizing the processes with the & character at the end of the command, but it does not do the job parallel.

optipngout=$(optipng -$pnglevel -dir $outdir ${threaddata_1[@]} &)
optipngout=$(optipng -$pnglevel -dir $outdir ${threaddata_2[@]} &)
optipngout=$(optipng -$pnglevel -dir $outdir ${threaddata_3[@]} &)
optipngout=$(optipng -$pnglevel -dir $outdir ${threaddata_4[@]} &)

I have to catch the output of the command, so I think the problem is with the $() structure.

Upvotes: 2

Views: 1134

Answers (2)

Patartics Milán
Patartics Milán

Reputation: 4938

I solved my problem with the taskset and & commands. Thanks for @jedwards

The final code looks like this:

outstr=$(taskset 0xFFFFFFFF optipng -$pnglevel -dir $outdir ${threaddata_1[@]}) &
outstr=$(taskset 0xFFFFFFFF optipng -$pnglevel -dir $outdir ${threaddata_2[@]}) &
outstr=$(taskset 0xFFFFFFFF optipng -$pnglevel -dir $outdir ${threaddata_3[@]}) &
outstr=$(taskset 0xFFFFFFFF optipng -$pnglevel -dir $outdir ${threaddata_4[@]}) &

Upvotes: 2

Lars Kotthoff
Lars Kotthoff

Reputation: 109232

Have a look at Gnu parallel. It allows you to do what you want with only small modifications to your existing script.

Upvotes: 3

Related Questions