CODEWITHSUNDEEP
pythonpython-multiprocessing
dylan7
dylan7

Reputation: 813

Python Multiprocessing conflicting adds to List

I am using the python Process class to have multiple objects work on multiple pictures and then when a given object is done append the picture to a shared list.

However, the result is that if three objects are created I only see one picture in the list at a time. I never remove the pictures from the list. It seems that each .append() of a new pic is removing the previous picture. How would I go about dealing with this?

Thank you.

Upvotes: 2

Views: 2105

Answers (1)

Alexander
Alexander

Reputation: 109686

Because of the Global Interpreter Lock (GIL), Python Lists and Dictionaries are thread-safe within the same process. Perhaps you can dispatch each process a list or dictionary container to store the response of that process and then append these items back to your list in the main process after process.join()? That way you won't be sharing the same list across multiple processes, but will instead join them after they complete each process.

You can also use a Queue, which is both thread safe and process safe.

Upvotes: 1

Related Questions