CODEWITHSUNDEEP
pythonmultithreadingpool
Ramon Medeiros
Ramon Medeiros

Reputation: 2631

How to use pool of threads in python, in this case

Suppose I want to process strings in a file, in each line, which are not dependent between each other. How I usually do:

def process_line(filelines, i):
    #process lines
import threading
threads = []
lines = open(file).readlines()
for i in range(0,len(lines)):
    threads.append(threading.Thread(target=process_line, args=(lines,i,))

[t.start() for t in threads]
[t.join() for t in threads]

Upvotes: 0

Views: 74

Answers (1)

Ron Serruya
Ron Serruya

Reputation: 4426

from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor() as executor:
    [executor.submit(proccess_line, lines, i) for i in range(len(lines))]

ThreadPoolExecutor docs: https://docs.python.org/3/library/concurrent.futures.html

Note that it's better to use processes instead of threads for these kinds of tasks (replace ThreadPoolExecutor with ProccessPoolExecutor

Upvotes: 1

Related Questions