CODEWITHSUNDEEP
pythonjsonpython-3.xfastapipydantic
Arpad Horvath -- Слава Україні
Arpad Horvath -- Слава Україні

Reputation: 1882

In JSON created from a pydantic.BaseModel exclude Optional if not set

I want to exclude all the Optional values that are not set when I create JSON. In this example:

from pydantic import BaseModel
from typing import Optional


class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]


print(Foo(x=3).json())

I get {"x": 3, "y": 42, "z": null}. But I would like to exclude z. Not because its value is None, but because it is Optional and there was no keyword argument for z. In the two cases below I would like to have z in the JSON.

Foo(x=1, z=None)
Foo(x=1, z=77)

If there is any other solution to set z to optional in this sense, I would like to see it.

Upvotes: 21

Views: 34578

Answers (2)

Roshin Jay
Roshin Jay

Reputation: 939

If you're using FastAPI then using exclude_none doesn't seem to work when a response_model is mentioned in the route decorator.

@app.post("/items/", response_model=Item)
async def create_item(item: Item):
    return item.dict(exclude_none=True)

Fast api seems to reprocess the dict with the pydantic model

So overriding the dict method in the model itself should work

def Item(BaseModel):
   name: str
   description: Optional[str]
   ...
   def dict(self, *args, **kwargs) -> Dict[str, Any]:
        kwargs.pop('exclude_none', None)
        return super().dict(*args, exclude_none=True, **kwargs)

(an actual solution would put this definition in separate subclass of BaseModel for reuse)

Note: just changing the default value of the exclude_none keyword argument is not enough: it seems FastAPI always sends exclude_none=False as an argument.

Source:
https://github.com/tiangolo/fastapi/issues/3314#issuecomment-962932368

Upvotes: 6

alex_noname
alex_noname

Reputation: 32133

You could exclude only optional model fields that unset by making of union of model fields that are set and those that are not None.

Pydantic provides the following arguments for exporting method model.dict(...):

exclude_unset: whether fields which were not explicitly set when creating the model should be excluded from the returned dictionary; default False.

exclude_none: whether fields which are equal to None should be excluded from the returned dictionary; default False

To make union of two dicts we can use the expression a = {**b, **c} (values from c overwrites values from b). Note that since Python 3.9 it could be done just as a = b | c.

from pydantic import BaseModel
from typing import Optional
from pydantic.json import pydantic_encoder
import json


class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]

def exclude_optional_dict(model: BaseModel):
    return {**model.dict(exclude_unset=True), **model.dict(exclude_none=True)}

def exclude_optional_json(model: BaseModel):
    return json.dumps(exclude_optional_dict(model), default=pydantic_encoder)
    


print(exclude_optional_json(Foo(x=3)))  # {"x": 3, "y": 42}
print(exclude_optional_json(Foo(x=3, z=None)))  # {"x": 3, "z": null, "y": 42}
print(exclude_optional_json(Foo(x=3, z=77)))  # {"x": 3, "z": 77, "y": 42}

Update

In order for the approach to work with nested models, we need to do a deep union( or merge) of two dictionaries, like so:

def union(source, destination):
    for key, value in source.items():
        if isinstance(value, dict):
            node = destination.setdefault(key, {})
            union(value, node)
        else:
            destination[key] = value

    return destination

def exclude_optional_dict(model: BaseModel):
    return union(model.dict(exclude_unset=True), model.dict(exclude_none=True))

class Foo(BaseModel):
    x: int
    y: int = 42
    z: Optional[int]

class Bar(BaseModel):
    a: int
    b: int = 52
    c: Optional[int]
    d: Foo


print(exclude_optional_json(Bar(a=4, d=Foo(x=3))))
print(exclude_optional_json(Bar(a=4, c=None, d=Foo(x=3, z=None))))
print(exclude_optional_json(Bar(a=4, c=78, d=Foo(x=3, z=77))))

{"a": 4, "b": 52, "d": {"x": 3, "y": 42}}
{"a": 4, "b": 52, "d": {"x": 3, "y": 42, "z": null}, "c": null}
{"a": 4, "b": 52, "c": 78, "d": {"x": 3, "y": 42, "z": 77}}

Upvotes: 25

Related Questions