CODEWITHSUNDEEP
pythonpickle
simona
simona

Reputation: 2181

pickling and unpickling user-defined class

I have a user-defined class 'myclass' that I store on file with the pickle module, but I am having problem unpickling it. I have about 20 distinct instances of the same structure, that I save in distinct files. When I read each file, the code works on some files and not on others, when I get the error:

'module' object has no attribute 'myclass'

I have generated some files today and some other yesterday, and my code only works on the files generated today (I have NOT changed class definition between yesterday and today).

I was wondering if maybe my method is not robust, if I am not doing things as I should do, for example maybe I cannot pickled user-defined class, and if this is introducing some randomness in the process.

Another issue could be that the files that I generated yesterday were generated on a different machine --- because I work on an academic cluster, I have some login nodes and some computing nodes, that differs by architecture. So I generated yesterday files on the computing nodes, and today files on the login nodes, and I am reading everything on the login nodes.

As suggested in some of the comments, I have installed dill and loaded it with import dill as pickle. Now I can read the files from computing nodes to login nodes of the same cluster. But if I try to read the files generated on the computing node of one cluster, on the login node of another cluster I cannot. I get KeyError: 'ClassType' in _load_type(name) in dill.py

Can it be because the python version is different? I have generated the files with python2.7 and I read them with python3.3.

EDIT:

I can read the pickled files, if I use everywhere python 2.7. Sadly, part of my code, written in python 3, is not automatically compatible with python 2.7 :(

Upvotes: 3

Views: 12977

Answers (1)

Dunes
Dunes

Reputation: 40713

Can you from mymodule import myclass? Pickling does not pickle the class, just a reference to it. To load a pickled object python must be able to find the class that was to be used to create the object.

eg.

import pickle

class A(object):
    pass

obj = A()
pickled = pickle.dumps(obj)

_A = A; del A # hide class

try:
    pickle.loads(pickled)
except AttributeError as e:
    print(e)

A = _A # unhide class
print(pickle.loads(pickled))

Upvotes: 4

Related Questions