CODEWITHSUNDEEP
pythonpydantic
Lionel Hamayon
Lionel Hamayon

Reputation: 1330

How to build a self-referencing model in Pydantic with dataclasses?

I am building an API using FastAPI and pydantic.

As I follow DDD / clean architecture, which separates the definition of the model from the definition of the persistence layer, I use standard lib dataclasses in my model and then map them to SQLAlchemy tables using imperative mapping (ie. classical mapping).

This works perfectly :

@dataclass
class User:
    name: str
    age: int

@pydantic.dataclasses.dataclass
class PydanticUser(User):
     ...

However, I encounter a problem when defining a class with self-reference.

✅ class inheriting from Pydantic’s BaseModel can self-reference

Inheriting from pydantic’s BaseModel works, however this is not compatible with SQLAlchemy imperative mapping, which I would like to use to stick to clean architecture / DDD principles.

class BaseModelPerson(BaseModel):
     name: str
     age: int
     parent_person: BaseModelPerson = None

BaseModelPerson.update_forward_refs()
john = BaseModelPerson(name="John", age=49, parent_person=None)
tim = BaseModelPerson(name="Tim", age=14, parent_person=john)

print(john)
# BaseModelPerson(name='John', age=49, parent_person=None)
print(tim)
# BaseModelPerson(name='Tim', age=14, parent_person=BaseModelPerson(name='John', age=49, parent_person=None))

✅ Standard lib dataclasses can also self-reference

from __future__ import annotations
from dataclasses import dataclass

@dataclass
class StdlibPerson:
    name: str
    age: int
    parent: StdlibPerson


john = StdlibPerson(name="John", age=49, parent=None)
tim = StdlibPerson(name="Tim", age=14, parent=john)

print(john)
# StdlibPerson(name='John', age=49, parent=None)

print(tim)
# StdlibPerson(name='Tim', age=14, parent=StdlibPerson(name='John', age=49, parent=None))

❌ Pydantic dataclass conversion causes recursion error

The problem occurs when I try to convert the standard library dataclass into a pydantic dataclass.

Defining a Pydantic dataclass like this:

PydanticPerson = pydantic.dataclasses.dataclass(StdlibPerson)

returns an error:

# output (hundreds of lines - that is recursive indeed)

    # The name of an attribute on the class where we store the Field
  File "pydantic/main.py", line 990, in pydantic.main.create_model
  File "pydantic/main.py", line 299, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 411, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 342, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 456, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 673, in pydantic.fields.ModelField.populate_validators
  File "pydantic/class_validators.py", line 255, in pydantic.class_validators.prep_validators
  File "pydantic/class_validators.py", line 238, in pydantic.class_validators.make_generic_validator
  File "/usr/lib/python3.9/inspect.py", line 3111, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/lib/python3.9/inspect.py", line 2860, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
  File "/usr/lib/python3.9/inspect.py", line 2323, in _signature_from_callable
    return _signature_from_function(sigcls, obj,
  File "/usr/lib/python3.9/inspect.py", line 2155, in _signature_from_function
    if _signature_is_functionlike(func):
  File "/usr/lib/python3.9/inspect.py", line 1883, in _signature_is_functionlike
    if not callable(obj) or isclass(obj):
  File "/usr/lib/python3.9/inspect.py", line 79, in isclass
    return isinstance(object, type)
RecursionError: maximum recursion depth exceeded while calling a Python object

Defining StdlibPerson like this does not solve the problem:

@dataclass
class StdlibPerson
    name: str
    age: int
    parent: "Person" = None

nor does using the second way provided by pydantic documentation:

@pydantic.dataclasses.dataclass
class PydanticPerson(StdlibPerson)
    ...

❌ using Pydantic dataclasses directly

from __future__ import annotations
from pydantic.dataclasses import dataclass
from typing import Optional

@pydantic.dataclasses.dataclass
class PydanticDataclassPerson:
     name: str
     age: int
     parent: Optional[PydanticDataclassPerson] = None

john = PydanticDataclassPerson(name="John", age=49, parent=None)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 6, in __init__
  File "pydantic/dataclasses.py", line 97, in pydantic.dataclasses._generate_pydantic_post_init._pydantic_post_init
    # | False |       |       |
  File "pydantic/main.py", line 1040, in pydantic.main.validate_model
  File "pydantic/fields.py", line 699, in pydantic.fields.ModelField.validate
pydantic.errors.ConfigError: field "parent" not yet prepared so type is still a ForwardRef, you might need to call PydanticDataclassPerson.update_forward_refs().

>>> PydanticDataclassPerson.update_forward_refs()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'PydanticDataclassPerson' has no attribute 'update_forward_refs'

Question

how can I define a pydantic model with self-referencing objects so that it is compatible with SQLAlchemy imperative mapping ?

Upvotes: 12

Views: 13646

Answers (3)

Lionel Hamayon
Lionel Hamayon

Reputation: 1330

It seems there are no easy solution to build a REST API with FastAPI, self-referencing objects and SQLAlchemy imperative mapping.

I have decided to switch to the FastAPI / GraphQL stack, with the Strawberry library which is explicitly recommended in the FastAPI documentation.

No problem so far, Strawberry makes it easy to build a GraphQL server and it handles self-referencing object in a breeze.

#!/usr/bin/env python3.10
# src/my_app/entrypoints/api/schema.py

import typing
import strawberry

@strawberry.type
class Person:
    name: str
    age: int
    parent: Person | None

Upvotes: 4

vaibhav jain
vaibhav jain

Reputation: 1

Inheriting also works

class Person(BaseModel):
    name: str
    age: int


class StdlibPerson(Person):
    parent: Person = None

Upvotes: 0

Ibrahim
Ibrahim

Reputation: 23

from typing import ForwardRef
from pydantic import BaseModel

Foo = ForwardRef('Foo')


class Foo(BaseModel):
    a: int = 123
    b: Foo = None


Foo.update_forward_refs()

print(Foo())
#> a=123 b=None
print(Foo(b={'a': '321'}))
#> a=123 b=Foo(a=321, b=None)

You can use it like this. This is called as Postponed annotations. Here is the link to read about it more- https://pydantic-docs.helpmanual.io/usage/postponed_annotations/

Upvotes: 2

Related Questions