FastAPI / Pydantic circular references in separate files

Question

I would love to use a schema that looks something like the following in FastAPI:

from __future__ import annotations
from typing import List
from pydantic import BaseModel


class Project(BaseModel):
    members: List[User]


class User(BaseModel):
    projects: List[Project]


Project.update_forward_refs()

but in order to keep my project structure clean, I would ofc. like to define these in separate files. How could I do this without creating a circular reference?

With the code above the schema generation in FastAPI works fine, I just dont know how to separate it out into separate files. In a later step I would then instead of using attributes use @propertys to define the getters for these objects in subclasses of them. But for the OpenAPI doc generation, I need this combined - I think.

Answer 1

As stated in a previous comment updating typing-extensions helped. I found the error using aws-sam-cli.

poetry add typing-extensions@4.5.0 or pip install typing-extensions==4.5.0

Answer 2

If I want to split the models and schemas into separate files, I will create extra files for the ProjectBase model and UserBase model so the Project model and User model could inherit from them. I will do like this:

#project_base.py
from pydantic import BaseModel

class ProjectBase(BaseModel):
    id: int
    title: str
    
    class Config:
        orm_mode=True

 

#user_base.py
from pydantic import BaseModel

class UserBase(BaseModel):
    id: int
    title: str
    
    class Config:
        orm_mode=True

 

#project.py
from typing import List
from .project_base import ProjectBase
from .user_base import UserBase

class Project(ProjectBase):
    members: List[UserBase] = []

 

#user.py
from typing import List
from .project_base import ProjectBase
from .user_base import UserBase

class User(UserBase):
    projects: List[ProjectBase] = []

note: for this method the orm_mode must be put in the ProjectBase and UserBase, so it can read by Project and User even if it is not a dict

Answer 3

Just place all your schema imports to the bottom of the file, after all classes, and call update_forward_refs().

#1/4
from __future__ import annotations # this is important to have at the top
from pydantic import BaseModel

#2/4
class A(BaseModel):
    my_x: X   # a pydantic schema from another file

class B(BaseModel):
    my_y: Y   # a pydantic schema from another file

class C(BaseModel):
    my_z: int

#3/4
from myapp.schemas.x import X   # related schemas we import after all classes
from myapp.schemas.y import Y

#4/4
A.update_forward_refs()   # tell the system that A has a related pydantic schema
B.update_forward_refs()   # tell the system that B has a related pydantic schema
                          # for C we don't need it, because C has just an integer field.

NOTE: Do this in every file that has schema imports. That will enable you make any combination without circular import problems.

NOTE 2: People usually put the imports and update_forward_refs() after every class, and then report that it doesn't work. That is usually because if an app is complex, you do not know what import is calling which class and when. Therefore, if you put it at the bottom, you are sure that every class will be 'scanned' and visible for others.

Answer 4

To me, the other answers don't seem to solve this on a satisfactory level due to ignoring the locals in modules. Here is a straightforward way to make that work on separate files:

user.py

from typing import TYPE_CHECKING, List
from pydantic import BaseModel

if TYPE_CHECKING:
    from project import Project

class User(BaseModel):
    projects: List['Project']

project.py

from typing import TYPE_CHECKING, List
from pydantic import BaseModel

if TYPE_CHECKING:
    from user import User

class Project(BaseModel):
    members: List['User']

main.py

from project import Project
from user import User

# Update the references that are as strings
Project.update_forward_refs(User=User)
User.update_forward_refs(Project=Project)

# Example: Projects into User and Users into Project
Project(
    members=[
        User(
            projects=[
                Project(members=[])
            ]
        )
    ]
)

This works if you run the main.py. If you are building a package, you may put that content to an __init__.py file that is high enough in the structure to not have circular import problem.

Note how we passed the User=User and Project=Project to update_forward_refs. This is because the module scopes where these classes are don't have references to each other (as if they did, there would be circular import). Therefore we pass them in main.py when updating the references as there we don't have the circular import problem.

NOTE: About type checking

If if TYPE_CHECKING: patterns are unfamiliar, they are basically if blocks that are never True on runtime (running your code) but they are used by code analysis (IDEs) to highlight the types. Those blocks are not needed for the example to work but are highly recommended as otherwise, it's hard to read the code, find out where these classes actually are defined and fully utilize code analysis tools.

Answer 5

There are three cases when circular dependency may work in Python:

• Top of module: import package.module
• Bottom of module: from package.module import attribute
• Top of function: works both

In your situation, the second case "bottom of module" will help. Because you need to use update_forward_refs function to resolve pydantic postponed annotations like this:

# project.py
from typing import List
from pydantic import BaseModel


class Project(BaseModel):
    members: "List[User]"


from user import User
Project.update_forward_refs()

# user.py
from typing import List
from pydantic import BaseModel


class User(BaseModel):
    projects: "List[Project]"


from project import Project
User.update_forward_refs()

Nonetheless, I would strongly discourage you from intentionally introducing circular dependencies