CODEWITHSUNDEEP
pythonserializationimportray
Quan Hoang
Quan Hoang

Reputation: 93

Using a class whose methods are decorated by @ray.remote in another directory

I'm learning how to use the python package Ray to parallel my code. I'm facing a problem with a class whose methods are decorated with @ray.remote.

It is okay for me to import the class in the same folder and execute the method in the class. However, after importing the class to another directory, it raises ModuleNotFoundError when the method is called.

According to the trackback of the error, it seems like the issue is about the deserialization process when the program can not find the module it requires.

A minimal working example of my issue is shown below.

Structure of the directory.
folder/
    main/
        a_class.py
        process_in_main.py
    processing/
        process_not_in_main.py

Content of three scripts

# a_class.py
import ray

class Foo:
    @ray.remote
    def single_function(self):
        return None

    def combined_function(self):
        ray.init()
        results_id = [Foo.single_function.remote(self) for i in range(5)]
        results = ray.get(results_id)
        ray.shutdown()
        return None



# process_in_main.py, it works fine
import a_class
instance = a_class.Foo()
instance.combined_function()



# process_not_in_main.py
import sys
sys.path.append('../')
from main import a_class

instance = a_class.Foo()
instance.combined_function() # it will raise ModuleNotFoundError: No module named 'main'

Any help would be appreciated. Thanks in advance.

Upvotes: 0

Views: 2471

Answers (1)

Felix Kleine Bösing
Felix Kleine Bösing

Reputation: 606

Please check that the imports are really resolvable. Both scripts work for me, when I set the Pythonpath to the parent of folder and use absolute imports that start with "folder".

Dockerised example here (since I´m at windows at the moment) : https://github.com/FelixKleineBoesing/stackoverflowSnippets/tree/master/question59809169

Upvotes: 0

Related Questions