CODEWITHSUNDEEP
processbackgroundparallel-processingcsh
Nomadiq
Nomadiq

Reputation: 360

How to limit the background processes launched by C shell?

I process 100 files in a directory with a command called process and as I want to parallel this process as much as possible. So, I issue the following commands in a C shell and it works great:

foreach F (dir/file*.data)

process $F > $F.processed &

echo $F

end

All 100 processes launch at once in the background, maximizing the usage of all my cores.

Now I want to use only a half of my cores (2 out of 4) at once. Is there an elegant way to do this?

Upvotes: 2

Views: 290

Answers (1)

Ole Tange
Ole Tange

Reputation: 33740

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

parallel -j 50% 'process {} > {}.processed; echo {}' ::: dir/file*.data

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Upvotes: 1

Related Questions