CODEWITHSUNDEEP
pythonmultiprocessingglobal-variablesshared-memory
Benji
Benji

Reputation: 71

Shared memory global list while running multiprocessing in python

I'm currently coding a chatbot for my streaming. Since it needs to do multiple things at once I'm using the multiprocessing module, that way it can still respond to commands and do functions at the same time. My problem now is that I have one process dedicated to some web scraping and another one to look at chat and respond if a command is being typed. My thoughts were, if I append the information from one process to a global list, and then when the command is being typed in chat, the other process can use the information in the appended list. Well, this didn't work and I learned that this is because the 2 processes don't have shared memory, although both having access to the same list, they are both copies of the list, so even if one appends, in the other process' case, it will still be empty. I've come across a few questions regarding this here on stack overflow, but the examples are very specific and since I'm fairly new to coding still, I had a hard time figuring out how to apply it to my own code. For this exact reason, I've simplified the problem so it can help others who are in a similar situation, by having my example broad enough and simple enough for anyone to understand it once they read the solution. Thus this is not the code I'm actually using for my chatbot, but one that mimics the problem.

import multiprocessing as mp
import time

globalList = []

def readList():
    while True:
        time.sleep(2)
        if globalList:
            print(globalList)
        else:
            print("List is Empty")
            print(globalList)
def writeList():
    while True:
        time.sleep(3)
        globalList.append("Item")
        print(globalList)

if __name__ == '__main__':
    p1 = mp.Process(target=readList)
    p2 = mp.Process(target=writeList)
    p1.start()
    p2.start()

When running this code you can see that the writeList function will keep adding another item to the list, but the readList function will keep showing an empty list.

I hope some master wiz out there can help me with this problem.

Upvotes: 2

Views: 797

Answers (2)

alex_noname
alex_noname

Reputation: 32233

In Python processes cannot straightforwardly access global mutable objects created by other processes. For this, you can use, for example, a multiprocessing.Manager and its proxy objects. Your adapted example:

import multiprocessing as mp
import time


def readList(shared_list):
    while True:
        time.sleep(2)
        if shared_list:
            print(shared_list)
        else:
            print("List is Empty")
            print(shared_list)


def writeList(shared_list):
    while True:
        time.sleep(3)
        shared_list.append("Item")
        print(shared_list)


if __name__ == '__main__':
    manager = mp.Manager()
    shared_list = manager.list()
    p1 = mp.Process(target=readList, args=(shared_list,))
    p2 = mp.Process(target=writeList, args=(shared_list,))
    p1.start()
    p2.start()
    p1.join()
    p2.join()

Upvotes: 1

go2nirvana
go2nirvana

Reputation: 1648

You can not have that by normal means. Processes have their own memory space. Threads, on the other hand, have same memory space and are ran within one process.

For more, please, reffer to this answer Multiprocessing vs Threading Python

Upvotes: 0

Related Questions