CODEWITHSUNDEEP
pythonmultithreading
zulln
zulln

Reputation: 49

Read a file and do something, multithreading

This source is just an exemple:

inputf = open('input', 'r')
outputf = open('output', 'a')

for x in inputf:
    x = x.strip('\n')
    result = urllib2.urlopen('http://test.com/'+x).getcode()
    outputf.write(x+' - '+result+'\n')

I want to add threading to this to check a few URLs at the same time. The user should everytime decide how many threads he want to use. The order on the output is not important.

What is the best and most beautiful way for that?

Upvotes: 3

Views: 108

Answers (1)

Cameron Sparr
Cameron Sparr

Reputation: 3981

I like multiprocessing.pool.ThreadPool (or multiprocessing.pool.Pool)

like:

from multiprocessing.pool import ThreadPool

n_threads = 5
pool = ThreadPool(processes=n_threads)

threads = [pool.apply_async(some_function, args=(arg1,)) for arg1 in args]

pool.close()
pool.join()

results = [result.get() for result in threads]

Upvotes: 4

Related Questions