CODEWITHSUNDEEP
pythonpicklenamedtuplepython-3.5
marcotama
marcotama

Reputation: 2062

Why can't I pickle a typing.NamedTuple while I can pickle a collections.namedtuple?

Why can't I pickle a typing.NamedTuple while I can pickle a collections.namedtuple? How can I manage to do pickle a NamedTuple?

This code shows what I have tried so far:

from collections import namedtuple
from typing import NamedTuple

PersonTyping = NamedTuple('PersonTyping', [('firstname',str),('lastname',str)])
PersonCollections = namedtuple('PersonCollections', ['firstname','lastname'])

pt = PersonTyping("John","Smith")
pc = PersonCollections("John","Smith")


import pickle
import traceback

try:
    with open('personTyping.pkl', 'wb') as f:
        pickle.dump(pt, f)
except:
    traceback.print_exc()
try:
    with open('personCollections.pkl', 'wb') as f:
        pickle.dump(pc, f)
except:
    traceback.print_exc()

Output on the shell:

$ python3 prova.py 
Traceback (most recent call last):
  File "prova.py", line 16, in <module>
    pickle.dump(pt, f)
_pickle.PicklingError: Can't pickle <class 'typing.PersonTyping'>: attribute lookup PersonTyping on typing failed
$

Upvotes: 12

Views: 1517

Answers (1)

Ashwini Chaudhary
Ashwini Chaudhary

Reputation: 250991

It's a bug. I have opened a ticket on it: http://bugs.python.org/issue25665

The issue is that namedtuple function while creating the class sets its __module__ attribute by looking up __name__ attribute from the calling frame's globals. In this case the caller is typing.NamedTuple.

result.__module__ = _sys._getframe(1).f_globals.get('__name__', '__main__')

So, it ends up setting it up as 'typing' in this case.

>>> type(pt)
<class 'typing.PersonTyping'>  # this should be __main__.PersonTyping
>>> type(pc)
<class '__main__.PersonCollections'>
>>> import typing
>>> typing.NamedTuple.__globals__['__name__']
'typing'

Fix:

Instead of this the NamedTuple function should set it itself:

def NamedTuple(typename, fields):

    fields = [(n, t) for n, t in fields]
    cls = collections.namedtuple(typename, [n for n, t in fields])
    cls._field_types = dict(fields)
    try:
        cls.__module__ = sys._getframe(1).f_globals.get('__name__', '__main__')
    except (AttributeError, ValueError):
        pass
    return cls

For now you can also do:

PersonTyping = NamedTuple('PersonTyping', [('firstname',str),('lastname',str)])
PersonTyping.__module__ = __name__

Upvotes: 6

Related Questions