CODEWITHSUNDEEP
pythongenericspython-3.6python-typing
pdowling
pdowling

Reputation: 500

Generic NamedTuple in Python 3.6

I'm trying to create a generic version of a NamedTuple, as follows:

T1 = TypeVar("T1")
T2 = TypeVar("T2")

class Group(NamedTuple, Generic[T1, T2]):
    key: T1
    group: List[T2]

g = Group(1, [""])  # expecting type to be Group[int, str]

However, I get the following error:

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

I'm not sure how else to achieve what I'm trying to do here, or if this might be a bug in the typing mechanism on some level.

Upvotes: 28

Views: 8278

Answers (2)

MarcinKonowalczyk
MarcinKonowalczyk

Reputation: 2788

Old question, but I think I've come across something which works pretty well. Based on this answer, i've come up with the following:

from typing import TYPE_CHECKING, TypeVar, Generic, Tuple
from typing_extensions import reveal_type

T1 = TypeVar("T1")
T2 = TypeVar("T2")

if TYPE_CHECKING:
    # This is how Group looks like to the type checker
    class Group(Tuple[T1, T2], Generic[T1, T2]):
        def __new__(cls, key: T1, group: T2) -> "Group[T1, T2]": ...
        @property
        def key(self) -> T1: ...
        @property
        def group(self) -> T2: ...

else:
    # This is the actual implementation of Group, aka literally a tuple with two boltons
    class Group(tuple):
        def __new__(cls, key, group):
            return tuple.__new__(cls, (key, group))

        @property
        def key(self):
            return self[0]

        @property
        def group(self):
            return self[1]


# Can unpack/index the Group because its literally a tuple at runtime
G1 = Group(1, "hello")
key1, grp1 = G1
key1 = G1[0]
grp1 = G1[1]
key1 = G1.key
grp1 = G1.group

# Types are ok
if TYPE_CHECKING:
    reveal_type(key1)  # Revealed type is "builtins.int"
    reveal_type(grp1)  # Revealed type is "builtins.str"
    reveal_type(G1.key)  # Revealed type is "builtins.int"
    reveal_type(G1.group)  # Revealed type is "builtins.str"
    reveal_type(G1[0])  # Revealed type is "builtins.int"
    reveal_type(G1[1])  # Revealed type is "builtins.str"

# And it *is* a generic
G2 = Group(False, 1.0)
key2, msg2 = G2
if TYPE_CHECKING:
    reveal_type(key2)  # Revealed type is "builtins.bool"
    reveal_type(msg2)  # Revealed type is "builtins.float"

mypy playground

Aka just straight up lying to the type checker in a very careful way. Obv careful with such tricks. Type checker will not protect you from yourself here, so best stick with very simple behaviours and write plenty of what-you'd-otherwise-think-trivial tests.

P.S.: Did you know you can stick hashes to your own gists into the mypy-play link and it just kinda works(!) :DD

Upvotes: 1

FHTMitchell
FHTMitchell

Reputation: 12147

So this is a metaclass conflict since in python 3.6 the typing NamedTuple and Generic use different metaclasses (typing.NamedTupleMeta and typing.GenericMeta), which python can't handle. I'm afraid there is no solution to this, other than to subclass from tuple and manually initialise the values:

T1 = TypeVar("T1")
T2 = TypeVar("T2")

class Group(tuple, Generic[T1, T2]):

    key: T1
    group: List[T2]

    def __new__(cls, key: T1, group: List[T2]):
        self = tuple.__new__(cls, (key, group))
        self.key = key
        self.group = group
        return self            

    def __repr__(self) -> str:
        return f'Group(key={self.key}, group={self.group})'

Group(1, [""])  # --> Group(key=1, group=[""])

Due to PEPs 560 and 563 this is fixed in python 3.7:

Python 3.7.0b2 (v3.7.0b2:b0ef5c979b, Feb 28 2018, 02:24:20) [MSC v.1912 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import annotations
>>> from typing import *
>>> T1 = TypeVar("T1")
>>> T2 = TypeVar("T2")
>>> class Group(NamedTuple, Generic[T1, T2]):
...     key: T1
...     group: List[T2]
...
>>> g: Group[int, str] = Group(1, [""])
>>> g
Group(key=1, group=[''])

Of course in python 3.7 you can just use a dataclass which are less lightweight (and mutable) but serve similar purposes.

from dataclasses import dataclass, astuple
from typing import Generic, TypeVar, List

T1 = TypeVar('T1')
T2 = TypeVar('T2')

@dataclass
class Group(Generic[T1, T2]):

     # this stores the data like a tuple, but isn't required
     __slots__ = ("key", "group")

     key: T1
     group: List[T2]

     # if you want to be able to unpack like a tuple...
     def __iter__(self):
          yield from astuple(self)


g: Group[int, str] = Group(1, ['hello', 'world'])
k, v = g
print(g)

How well type checkers handle my solution / yours in python 3.7 though I haven't checked. I suspect it may not be seamless.

Edit

I found another solution -- make a new metaclass

import typing
from typing import *

class NamedTupleGenericMeta(typing.NamedTupleMeta, typing.GenericMeta):
    pass


class Group(NamedTuple, Generic[T1,T2], metaclass=NamedTupleGenericMeta):

    key: T1
    group: List[T2]


Group(1, ['']) # --> Group(key=1, group=[''])

Upvotes: 19

Related Questions