python threading issue

Question

I want to have the result of my threads in a list.

I have the following sample code:

def parallelizer_task(processor,input,callback):
    output = processor(input)
    if callback:
        callback(output)
    return output

class ThreadsParallelizer(Parallelizer):

    def parallelize(self,processors,input=None,callback=None):
        threads = []
        for processor in processors:
            t = threading.Thread(target=parallelizer_task,args=(processor,input,callback))
            threads.append(t)
            t.start()
        return threads
parallelizer = ThreadsParallelizer

But I have the output of threads list as ;

* <Thread(Thread-1, started 4418719744)>
* <Thread(Thread-2, started 4425617408)>
* <Thread(Thread-3, started 4429950976)>

Is there a way to have the threads result in the list?

Answer 1

Yes, for that you can use for example join. It will force the main thread to wait until child threads finish the work. You can then store the data in threading.Thread objects, something like this:

def parallelizer_task(processor,input,callback):
    output = processor(input)
    if callback:
        callback(output)

    # Attach result to current thread
    thread = threading.currentThread()
    thread.result = output


class ThreadsParallelizer(Parallelizer):

    def parallelize(self,processors,input=None,callback=None):
        threads = []
        for processor in processors:
            t = threading.Thread(...)
            threads.append(t)
            t.start()

        # wait for threads to finish
        for th in threads:
            th.join()

        # do something with results
        results = [th.result for th in threads]
        return results