CODEWITHSUNDEEP
pythonmultithreadingmultiprocessing
Gie Grajo
Gie Grajo

Reputation: 207

How to run t threads to a specific CPU?

I am new at multiprocessing. So, I have t threads and p CPU (machine threads). What I want is to equally distribute the threads into each CPU. But I am puzzling about the

I saw this documentation on how to to set the CPU affinity mask of a process indicated by the specified process id. os.sched_setaffinity() method

I'm currently using multiprocessing.ThreadPool method for threading (see the code below).

        pool = ThreadPool(processes=t)
        
        for j in range(t):
            threads[j] = pool.apply_async(pearson_cor, args=(j, split_x[j], y, n, len(split_x[j])))
        for j in range(t):
            correct_rs.extend(threads[j].get())
        pool.close()
        pool.join()

Additional questions. If I understood it correctly I need the process ID of the threads to assign specific CPU into them. Is this correct?

Also, can we get the thread process ID before running it?

Upvotes: 0

Views: 767

Answers (1)

relent95
relent95

Reputation: 4731

At least in Linux, you can set the pid argument of os.sched_setaffinity() to 0 or threading.get_native_id(), to change the affinity of the current thread.

If you want to do that with ThreadPool, do like this for example.

def init_worker_thread():
   masks = ...
   os.sched_setaffinity(0, masks)
pool = ThreadPool(processes=t, init_worker_thread)

Upvotes: 1

Related Questions