CODEWITHSUNDEEP
pythonmultithreadingmultiprocessing
abhay kanade
abhay kanade

Reputation: 33

Python multithreading,multi processing

I have a requirement which requires me to run 480 threads and 16 processes( each process will have 30 threads, so it's 480). I have heard that due to GIL, python does not have a good support for multi threading! Is there any way to execute the application with above requirements efficiently in python? Thanks in advance.

Upvotes: 0

Views: 72

Answers (2)

plamut
plamut

Reputation: 3206

It depends on your code and the type of the problem you are trying to solve. Python GIL applies to CPU-bound threads, i.e. threads that want to do CPU-intensive tasks*.

However, if your threads are I/O-bound, i.e. spend most of the time waiting for input/output, the GIL would not be a problem because while waiting for the I/O operation to complete, threads can do not need a lock on the Python interpreter. An example would be waiting for a network operation to complete (downloading a file, for example). You can easily fire multiple threads and download the files simultaneously.

*footnote: even for CPU-bound tasks, the GIL only applies to Python code. If a thread uses a C-extension (i.e. library written in C), it won't need the GIL while executing, because it won't be running Python instructions.

Edit: Reading your comment saying that you will be invoking the POS simulator functions via sockets... that's an I/O operation. While the server will be executing function calls, Python (the client) will simply wait doing nothing, and the threads will not need to hold GIL most of the time.

Upvotes: 2

Ayy
Ayy

Reputation: 488

This should give you an idea of how it's done.

import multiprocessing
import threading

def thread():
    pass


def process():
    threads = []
    for i in range(30):
        t = threading.Thread(target=thread)
        threads.append(t)
        t.start()
    for t in threads:
        t.join()

if __name__ == '__main__':
    processes = []
    for i in range(16):
        p = multiprocessing.Process(target=process)
        processes.append(p)
        p.start()
    for p in processes:
        p.join()

Upvotes: 0

Related Questions