Defining Fastapi Pydantic many to Many relationships

Question

What's a proper way to define many-to-many relationships in a pydantic model without getting circular import error. I have two files supplier_schema.py and category_schema.py.

supplier_schema.py

from pydantic import BaseModel, HttpUrl, EmailStr
from typing import Optional, List
from datetime import datetime
from schemas.category_schema import Category
from schemas.membership_schema import Membership


class SupplierBase(BaseModel):
    contact_person: str
    email_address: EmailStr
    company_name: str
    company_email: str
    company_logo: str
    country: str
    state_province: str
    city_area: Optional[str] = None
    location: str
    phone: str
    fax: Optional[str] = None
    tagline: str
    company_bio: Optional[str]
    year_established: int
    employees_count: str
    certificates: Optional[List[str]] = None
    cover_image: Optional[HttpUrl] = None
    gallery: Optional[List[HttpUrl]] = None
    company_license: Optional[HttpUrl] = None
    company_website: Optional[HttpUrl] = None
    whatsapp_number: Optional[str] = None
    facebook_url: Optional[HttpUrl] = None
    twitter_url: Optional[HttpUrl] = None
    linkedin_url: Optional[HttpUrl] = None
    pinterest_url: Optional[HttpUrl] = None
    instagram_url: Optional[HttpUrl] = None
    youtube_url: Optional[HttpUrl] = None
    annual_revenue: Optional[str] = None
    challenges: List[str]
    status: str = None

    class Config:
        orm_mode = True


class SupplierCreate(SupplierBase):
    password: str
    membership_id: int
    categories_id: Optional[List[int]] = []


class Supplier(SupplierBase):
    id: int
    created_at: datetime
    updated_at: datetime
    membership: Membership
    categories: List[Category]

category_schema.py

from pydantic import BaseModel, HttpUrl, EmailStr
from typing import Optional, List
from datetime import datetime
from schemas.category_schema import Category
from schemas.membership_schema import Membership


class SupplierBase(BaseModel):
    contact_person: str
    email_address: EmailStr
    company_name: str
    company_email: str
    company_logo: str
    country: str
    state_province: str
    city_area: Optional[str] = None
    location: str
    phone: str
    fax: Optional[str] = None
    tagline: str
    company_bio: Optional[str]
    year_established: int
    employees_count: str
    certificates: Optional[List[str]] = None
    cover_image: Optional[HttpUrl] = None
    gallery: Optional[List[HttpUrl]] = None
    company_license: Optional[HttpUrl] = None
    company_website: Optional[HttpUrl] = None
    whatsapp_number: Optional[str] = None
    facebook_url: Optional[HttpUrl] = None
    twitter_url: Optional[HttpUrl] = None
    linkedin_url: Optional[HttpUrl] = None
    pinterest_url: Optional[HttpUrl] = None
    instagram_url: Optional[HttpUrl] = None
    youtube_url: Optional[HttpUrl] = None
    annual_revenue: Optional[str] = None
    challenges: List[str]
    status: str = None

    class Config:
        orm_mode = True


class SupplierCreate(SupplierBase):
    password: str
    membership_id: int
    categories_id: Optional[List[int]] = []


class Supplier(SupplierBase):
    id: int
    created_at: datetime
    updated_at: datetime
    membership: Membership
    categories: List[Category]

But I get this error

Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/phat/projects/crdle2-api/venv/lib/python3.9/site-packages/uvicorn/subprocess.py", line 61, in subprocess_started
    target(sockets=sockets)
  File "/home/phat/projects/crdle2-api/venv/lib/python3.9/site-packages/uvicorn/server.py", line 49, in run
    loop.run_until_complete(self.serve(sockets=sockets))
  File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
  File "/home/phat/projects/crdle2-api/venv/lib/python3.9/site-packages/uvicorn/server.py", line 56, in serve
    config.load()
  File "/home/phat/projects/crdle2-api/venv/lib/python3.9/site-packages/uvicorn/config.py", line 308, in load
    self.loaded_app = import_from_string(self.app)
  File "/home/phat/projects/crdle2-api/venv/lib/python3.9/site-packages/uvicorn/importer.py", line 23, in import_from_string
    raise exc from None
  File "/home/phat/projects/crdle2-api/venv/lib/python3.9/site-packages/uvicorn/importer.py", line 20, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/phat/projects/crdle2-api/./main.py", line 3, in <module>
    from routers import supplier_router, membership_router, category_router
  File "/home/phat/projects/crdle2-api/./routers/supplier_router.py", line 4, in <module>
    from schemas.supplier_schema import Supplier, SupplierCreate
  File "/home/phat/projects/crdle2-api/./schemas/supplier_schema.py", line 4, in <module>
    from schemas.category_schema import Category
  File "/home/phat/projects/crdle2-api/./schemas/category_schema.py", line 4, in <module>
    from schemas.supplier_schema import Supplier
ImportError: cannot import name 'Supplier' from partially initialized module 'schemas.supplier_schema' (most likely due to a circular import) (/home/phat/projects/crdle2-api/./schemas/supplier_schema.py)

What am I doing wrong? How can I fix this?

Answer 1

So, I solved the issue by using annotations from __future__ import annotations and using pydantic's update_forward_refs

You can refer to this and this for further info.

supplier_schema.py

from __future__ import annotations
from pydantic import BaseModel
from typing import Optional, List
from datetime import datetime
# import schemas


class SupplierBase(BaseModel):
    company_email: Optional[str] = None
    logo: Optional[str] = None
    state_province: str = None
    city_area: Optional[str] = None
    location: Optional[str] = None
    company_phone: str = None
    fax: Optional[str] = None
    tagline: Optional[str] = None
    company_bio: Optional[str] = None
    postal_code: Optional[str] = None
    year_established: str = None
    employees_count: Optional[str] = None
    certificates: Optional[List[dict]] = []
    cover_image: Optional[str] = None
    gallery: Optional[List[dict]] = None
    license: Optional[str] = None
    website: Optional[str] = None
    whatsapp_number: Optional[str] = None
    facebook_url: Optional[str] = None
    twitter_url: Optional[str] = None
    linkedin_url: Optional[str] = None
    pinterest_url: Optional[str] = None
    instagram_url: Optional[str] = None
    youtube_url: Optional[str] = None
    annual_revenue: Optional[str] = None
    payment_methods: Optional[List[dict]] = None
    challenges: Optional[List[str]] = []
    status: str = None
    user_id: int

    class Config:
        orm_mode = True


class SupplierCreate(SupplierBase):
    pass


class Supplier(SupplierBase):
    id: int
    categories: Optional[List[Category]] = []
    created_at: datetime
    updated_at: datetime


from .category_schema import Category  # nopep8
Category.update_forward_refs()

category_schema.py

from __future__ import annotations
from typing import List
from pydantic import BaseModel
from datetime import datetime


class CategoryBase(BaseModel):
    label: str

    class Config:
        orm_mode = True


class CategoryCreate(CategoryBase):
    pass


class Category(CategoryBase):
    id: int
    created_at: datetime
    updated_at: datetime
    suppliers: List[Supplier]


from .supplier_schema import Supplier  # nopep8
Category.update_forward_refs()


class CategorySkeleton(CategoryBase):
    pass


class CategoryPatch(CategoryBase):
    pass

Answer 2

This import error is owing to the following process:

1. supplier_schema.py is defined
2. supplier_schema module imports Category from schemas.category_schema
3. Since schemas.category_schema hasn't yet been defined, it is now executed.
4. But wait! category_schema requires importing Supplier from supplier_schema.
5. supplier_schema cannot proceed execution without importing from category_schema and vice versa.
6. This is known as a circular import.

How can you solve circular imports?

I'll be addressing this specific case.
For more general use cases, read: Circular import dependency in Python

Option 1: Combine everything into one module.

• This is the fastest and simplest solution.
• Can get pretty out of hand once your ORM code increases to many models.

Option 2: Import models in the initialization of the root module and change import language:

• Robust solution
• Requires more careful planning and architecting of your project
• Requires testing (additional imports create opportunity for breakage)

Here's how to execute this:
Ensure your schemas module contains an init file:

-- schemas
|-- __init__.py
|-- category_schema.py
|-- membership_schema.py

Your __init__.py file should contain the following:

from . import category_schema
from . import membership_schema

Now, your imports should be revised to the following:

# Old way: ⬇
# from schemas.category_schema import Category

# New way: ⬇
import schemas

...

class Supplier(SupplierBase):    
    categories: List[schemas.category_schema.Category]
...

Repeat for other submodules and imports.

---

A word of advice on choosing a solution:

Circular imports are very common errors when working with ORMs. Encountering one means you've greatly extended the properties of your application – congratulations! Choosing the best solution for your project depends on your goals.

• If your code is a proof of concept and you need to move quickly, I'd recommend choosing option 1. (Move fast and break things.)
• If your code is part of a more robust project, even if you're in the early stages, definitely choose option 2. (Solve tomorrow's problems today.)