CODEWITHSUNDEEP
pythonshared-memory
Geeocode
Geeocode

Reputation: 5797

Does any inter-module shared memory based solution exist in python?

I made a code in python using theano. Because theano uses GPU, hence I wanted to utilize the time it uses to computation with GPU for computing an other similarly computing time hungry process via CPU.

I worked a lot with multiprocessing module, but unfortunately I found, that theano doesn't work along with multiprocessing module resulting in some locking error. Thus I altered my code and wanted to run the theano based module (modul1.py) and the other one(modul2.py) along each other in two separate modules without multiprocessing.

Aftermath: I have to share data among separate modules. I tried pickling, but is too slow for my solution, I waste the time I would gain with "multiprocessing" because of the file operations.

Hence I need a solution, with which I can exchange data between modules inside of MEMORY.

Note: I need only simplex data exchange i.e. I want to read only from one of the modules.

Upvotes: 0

Views: 66

Answers (1)

valentin
valentin

Reputation: 3608

Maybe mmap can help you , it is in standard library.

Documentation: https://docs.python.org/2/library/mmap.html

and an example: http://blog.schmichael.com/2011/05/15/sharing-python-data-between-processes-using-mmap/

Upvotes: 1

Related Questions