CODEWITHSUNDEEP
pythonpicklejoblib
Julkar9
Julkar9

Reputation: 2110

joblib: cannot load dumped joblib file from another directory

I am trying to load a dumped joblib file from another directory, it's giving the following error

Traceback (most recent call last):
  File "D:\dir\loader.py", line 5, in <module>
    foob = joblib.load(r'D:\foob.joblib')
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\site-packages\joblib\numpy_pickle.py", line 585, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\site-packages\joblib\numpy_pickle.py", line 504, in _unpickle
    obj = unpickler.load()
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\pickle.py", line 1088, in load
    dispatch[key[0]](self)
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\pickle.py", line 1376, in load_global
    klass = self.find_class(module, name)
  File "C:\programming\programmeFiles\anaconda\envs\tensor_gpu\lib\pickle.py", line 1426, in find_class
    __import__(module, level=0)
ModuleNotFoundError: No module named 'foo'

Minimal code to reproduce the error
The directory looks like this

D-
   \
    foo.py
    dumper.py
    foob.joblib
    dir-
        \
         loader.py

foo.py

class Foo:
    def __init__(self):
        self.s = "here at Foo"
    def __repr__(self):
        return "fooObject"

dumper.py

import joblib
import foo

foob = foo.Foo()
joblib.dump(foob,'foob.joblib')

loader.py

import joblib
foob = joblib.load(r'D:\foob.joblib')
print(foob)

If I try to load the foob.joblib from the same directory(D) it works fine, the error occurs while running loader.py from different directory

Upvotes: 3

Views: 4372

Answers (1)

fnokke
fnokke

Reputation: 1161

joblib.load() requires any modules required for unpickling to be in the path. So you need to add the location of foo.py the system path at loading time somehow.

For instance add:

import sys
import os
this_dir = os.path.dirname(__file__) # Path to loader.py
sys.path.append(os.path.join(this_dir, <rel_path_to_foo.py>))

to the top of loader.py

Upvotes: 4

Related Questions