CODEWITHSUNDEEP
pythonmultithreadingmultiprocessingpython-2.6
DilTeam
DilTeam

Reputation: 2661

Run certain number of threads/processes simultaneously

I need to run certain number of threads or processes simultaneously until all of them are completed. How do I do that in Python 2.6. (Note: I believe Python 3.0 has ThreadPoolExecutor but that doesn't help me because we are on 2.6).

I tried the following:

    pool = multiprocessing.Pool(2)

    try:
        for table_name in table_list:
            pool.apply_async(Processor(table_name, self.type, 
                self.properties).exec())
    except KeyboardInterrupt:
        pool.terminate()
    else:
        pool.close()
    pool.join()

but it's processing each table sequentially. What I want is, if there are 10 tables & Pool size is 5, then it should first start 5 threads. Once one of them completes it should pick the next one, then next and so on, till all of them are processed.

How can I do this in Python 2.6?

Upvotes: 0

Views: 71

Answers (1)

Dan D.
Dan D.

Reputation: 74675

pool.apply_async(Processor(table_name, self.type, self.properties).exec())

Immediately invokes .exec you need to delay so instead pass the .exec function:

pool.apply_async(Processor(table_name, self.type, self.properties).exec)

See apply_async(func[, args[, kwds[, callback]]])

Upvotes: 1

Related Questions