CODEWITHSUNDEEP
ruby-on-railsrubymultithreadingdebian
map7
map7

Reputation: 5126

can't create Thread: Resource temporarily unavailable

I'm running parallel tests for a ruby on rails 5.2.x project under Debian 10 on a powerful server (16 core, 32threads, 70GB ram) and I keep getting this error;

#<Thread:0x000000000b9667d8@/usr/local/rbenv/versions/2.6.6/lib/ruby/2.6.0/webrick/utils.rb:163 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
    1: from /usr/local/rbenv/versions/2.6.6/lib/ruby/2.6.0/webrick/utils.rb:187:in `watch'
/usr/local/rbenv/versions/2.6.6/lib/ruby/2.6.0/webrick/utils.rb:187:in `start': can't create Thread: Resource temporarily unavailable (ThreadError)

I'm running ruby 2.6.6 through rbenv, google-chrome version: '86.0.4240.183-1 amd64'. Sometimes it works and sometimes it doesn't.

I've checked the thread limits on my system;

Global max threads

% cat /proc/sys/kernel/threads-max
628996

User max threads

ulimit -a

-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-m: resident set size (kbytes)      unlimited
-u: processes                       314498
-n: file descriptors                1024
-l: locked-in-memory size (kbytes)  64
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 314498
-q: bytes in POSIX msg queues       819200
-e: max nice                        0
-r: max rt priority                 0
-N 15:                              unlimited

Update: Re-enact the error through a simple example

I created this little script and kept adjusting the threads until it complains with the same error.

#!/usr/bin/env ruby
THREADS = 600
threads = []
pids = []

THREADS.times do
  t= Thread.new do 
    pid = spawn("echo 'hi';sleep 10;echo 'bye'")
    pids.push(pid)
    sleep 10
  end
  threads.push(t)
end

threads.each do |thread|
  thread.join
end

pids.each do |pid|
  `kill #{pid}`
end

There are a lot of processes going on this computer and I think in normal operation I'm sitting close to the limit so when I go to run my tests it usually fails to create threads.

Running this script as different users returns different results

As my user it fails when I set the threads to 600, but if I login to a user which has nothing open then I can run 5000 threads no problem.

Upvotes: 2

Views: 5827

Answers (1)

map7
map7

Reputation: 5126

Found the issue, as all my sessions have been running through ssh (over x2go) as I'm working remote like everyone in 2020, I had to set the following for the ulimits to work;

/etc/ssh/sshd_config

UsePAM yes

After logout and back in I can now run my threads test without a problem, I even tried with 9000 threads and it worked.

Upvotes: 2

Related Questions