CODEWITHSUNDEEP
gnu-parallel
Ryan Berckmans
Ryan Berckmans

Reputation: 111

How to stop parallel from reporting "No more processes" with "-X" option?

Working off this example: http://www.gnu.org/software/parallel/man.html#EXAMPLE:-Speeding-up-fast-jobs

When I run:

seq -w 0 9999 | parallel touch pict{}.jpg
seq -w 0 9999 | parallel -X touch pict{}.jpg

Success! However, add another 9 and BOOM:

$ seq -w 0 99999 | parallel --eta -X touch pict{}.jpg parallel: Warning: No more processes: Decreasing number of running jobs to 3. Raising ulimit -u or /etc/security/limits.conf may help. Computers / CPU cores / Max jobs to run 1:local / 4 / 3 parallel: Warning: No more processes: Decreasing number of running jobs to 2. Raising ulimit -u or /etc/security/limits.conf may help. parallel: Warning: No more processes: Decreasing number of running jobs to 1. Raising ulimit -u or /etc/security/limits.conf may help. parallel: Error: No more processes: cannot run a single job. Something is wrong.

I would expect parallel -X to run no more jobs than I have cpu cores, and to cram as many parameters onto each job as the max command line length permits. How am I running out of processes?

My environment:

Upvotes: 3

Views: 1631

Answers (1)

Ole Tange
Ole Tange

Reputation: 33715

Your expectation is 100% correct. What you are seeing is clearly a bug - probably due to GNU Parallel not being well tested on OSX. Please follow http://www.gnu.org/software/parallel/man.html#REPORTING-BUGS and file a bug report.

Upvotes: 2

Related Questions