CODEWITHSUNDEEP
phpmultithreadingpthreads
Peter VARGA
Peter VARGA

Reputation: 5186

Running 600+ threads with PHP pthreads - what about the overhead

I have a server with 2 physical CPU which have together 24 cores and 10 GB RAM. The PHP program is calculating a statistic and I could run each section totally independent of the others. Once all calculations finished I have only to "merge" them.

Therefore I had the idea to perform each calculation phase in a separate thread created/controlled by "pthread". Each calculation takes around 0,10 seconds but the amount of the calculations lets it take that long when they are serialized.

My questions:

  1. Is there a limitation when creating a new "thread" with "pthreads"?
  2. What is the overhead when creating a new thread? I must consider this to avoid a new delay.

I can imagine that for several seconds the load would be very high but then it ends suddenly once each calculation finished. This is not the problem. It is "my" server and I do not have to take care regarding other users [or when it is a shared server].

Upvotes: 0

Views: 602

Answers (1)

Peter VARGA
Peter VARGA

Reputation: 5186

While "waiting" for an answer :-) I started to rewrite the class.

I can summarize it like this:

  1. There is no way to start 600 threads at once. I expected it but I wanted to know where is the limit. My configuration "allowed" around 160 threads to be started.
  2. When starting more than these 150 threads the PHP script stopped working without any further notice.
  3. As Franz Gleichmann pointed out the whole process took longer when starting lot of threads. I found out that starting 20 threads has the best performance.
  4. The achieved performance gain is between 20% and 50% - I am satisfied.
  5. I don't know if it is a bug in the pthread library but I could not access any class members. I had to move the class members inside the function. Due to the fact the calculation is in one function it did not bother me and I do not investigate it further.

Upvotes: 1

Related Questions