CODEWITHSUNDEEP
pythonpicklenamedtuple
eqzx
eqzx

Reputation: 5579

namedtuple pickling fails when variable name doesn't match typename

The python code below fails with the error pickle.PicklingError: Can't pickle <class '__main__.SpecialName'>: it's not found as __main__.SpecialName

import pickle
from collections import namedtuple

different_SpecialName = namedtuple('SpecialName', 'foo bar')
def save():
    foo = different_SpecialName(1, 2)
    with open('foosave.pkl', 'w') as f:
        pickle.dump(foo, f)

if __name__ == '__main__':
    save()

This seems like bad behaviour of the pickle module, as it depends on the correctness of a variable name. Changing different_SpecialName to SpecialName and re-running the code allows it to complete successfully. Changing the code to the below, where a variable with SpecialName is instantiated to be the same value as different_SpecialName, also lets the code run successfully

import pickle
from collections import namedtuple

different_SpecialName = namedtuple('SpecialName', 'foo bar')

## create new variable with 'correct' name
SpecialName = different_SpecialName

def save():
    # foo = different_SpecialName(1, 2)
    foo = SpecialName(1, 2)
    with open('foosave.pkl', 'w') as f:
        pickle.dump(foo, f)

if __name__ == '__main__':
    save()

My questions: is this fundamentally a pickle (and cPickle) bug? It seems like pickle shouldn't be looking up the class definition by using the name of the variable (although, I'm not sure what else it could do). Or, instead, is this an issue with the namedtuple API? I browsed the namedtuple documentation and couldn't find anything that explicitly told me to name my namedtuple variables the same as my typename argument (the first argument to the namedtuple() function)

Upvotes: 0

Views: 329

Answers (1)

ShadowRanger
ShadowRanger

Reputation: 155438

It's not a bug. pickle requires that

the class definition must be importable and live in the same module as when the object was stored.

From the perspective of the namedtuple's __reduce__ method, the type name is SpecialName (that's what you passed it after all). So when unpickling, it will try to import the module it was declared in and look for SpecialName. But since you didn't save it as SpecialName, it can't find it.

Without resorting to namedtuples, you can produce the exact same problem with:

class Foo:
    pass
Bar = Foo
del Foo

and trying to pickle and unpickle a Bar(); under the hood, you've effectively done the same thing with your mismatched names for a namedtuple.

Upvotes: 1

Related Questions