CODEWITHSUNDEEP
pythonstreamdeserializationpickledill
Troll_Hunter
Troll_Hunter

Reputation: 515

Pickle Error when loading an object in python?

When I try to load a file in python I get this error from dill:

Traceback (most recent call last):
  File "LArSoftSGD.py", line 154, in <module>
    stat_bundle = train_batch_iterator(clf, TOTAL_TRAINED_EVENTS)
  File "LArSoftSGD.py", line 121, in train_batch_iterator
    plot_data.append((test_batch_iterator(clf), tte))
  File "LArSoftSGD.py", line 91, in test_batch_iterator
    minibatch_test = dill.load(stream)
  File "/home/jdoe/.local/lib/python3.4/site-packages/dill/dill.py", line 199, in load
obj = pik.load()
  File "/usr/lib/python3.4/pickle.py", line 1036, in load
dispatch[key[0]](self)
KeyError: 95

Does anybody have any idea what is going wrong? Here is some of my code where the error happened:

for file in glob.glob('./SerializedData/Batch8202015_1999/*'):
    with open(file, 'rb') as stream:
        minibatch_test = dill.load(stream)

Upvotes: 0

Views: 1159

Answers (1)

Mike McKerns
Mike McKerns

Reputation: 35247

I'm the dill author. This does not seem to be a serialization error, it's most likely a coding error.

Essentially, the dispatch is a dictionary of pickled object types and their pickling functions, with the keys being object types. You have a 95 as a key… which is definitely not what the dispatch expects.

>>> import dill
>>> dill.dill.pickle_dispatch_copy
{<type 'unicode'>: <function save_unicode at 0x10c8f1cf8>, <type 'dict'>: <function save_dict at 0x10c8f1f50>, <type 'int'>: <function save_int at 0x10c8f1b18>, <type 'long'>: <function save_long at 0x10c8f1b90>, <type 'list'>: <function save_list at 0x10c8f1e60>, <type 'str'>: <function save_string at 0x10c8f1c80>, <type 'function'>: <function save_global at 0x10c8f5140>, <type 'instance'>: <function save_inst at 0x10c8f50c8>, <type 'type'>: <function save_global at 0x10c8f5140>, <type 'NoneType'>: <function save_none at 0x10c8f1a28>, <type 'bool'>: <function save_bool at 0x10c8f1aa0>, <type 'tuple'>: <function save_tuple at 0x10c8f1d70>, <type 'float'>: <function save_float at 0x10c8f1c08>, <type 'classobj'>: <function save_global at 0x10c8f5140>, <type 'builtin_function_or_method'>: <function save_global at 0x10c8f5140>}

You have something like this:

>>> key = [95]
>>> dispatch = {}
>>> dispatch[key[0]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 95

Your code to load the pickled object looks fine… however, and while that might be where the error is being thrown, it's not where the bug actual is. There's not enough information from what you've given above to uncover that.

Upvotes: 3

Related Questions