CODEWITHSUNDEEP
pythonpython-3.xpydantic
Jyotirmay
Jyotirmay

Reputation: 1825

Can I get incoming extra fields from Pydantic?

I have defined a pydantic Schema with extra = Extra.allow in Pydantic Config.

Is it possible to get a list or set of extra fields passed to the Schema separately.

For ex:

from pydantic import BaseModel as pydanticBaseModel
class BaseModel(pydanticBaseModel):
    name: str

    class Config:
        allow_population_by_field_name = True
        extra = Extra.allow

I pass below JSON:

   {
    "name": "Name",
    "address": "bla bla",
    "post": "post"
   }

I need a function from pydantic, if available which will return all extra fields passed. like: {"address", "post"}

Upvotes: 35

Views: 54305

Answers (3)

flakes
flakes

Reputation: 23634

NOTE: This answer applies to version Pydantic 1.x. See robcxyz's answer for a 2.x solution.

From the pydantic docs:

extra

whether to ignore, allow, or forbid extra attributes during model initialization. Accepts the string values of 'ignore', 'allow', or 'forbid', or values of the Extra enum (default: Extra.ignore). 'forbid' will cause validation to fail if extra attributes are included, 'ignore' will silently ignore any extra attributes, and 'allow' will assign the attributes to the model.

This can either be included in the model Config class, or as arguments when inheriting BaseModel.

from pydantic import BaseModel, Extra

class BaseModel(BaseModel, extra=Extra.allow):
    name: str


model = Model.parse_obj(
   {"name": "Name", "address": "bla bla", "post": "post"}
)

print(model)
# name='Name' post='post' address='bla bla'

To get the extra values you could do something simple, like comparing the set of __fields__ defined on the class to the values in __dict__ on an instance:

class Model(BaseModel, extra=Extra.allow):
    python_name: str = Field(alias="name")

    @property
    def extra_fields(self) -> set[str]:
        return set(self.__dict__) - set(self.__fields__)

>>> Model.parse_obj({"name": "Name", "address": "bla bla", "post": "post"}).extra_fields
{'address', 'post'}
>>> Model.parse_obj({"name": "Name", "foobar": "fizz"}).extra_fields
{'foobar'}
>>> Model.parse_obj({"name": "Name"}).extra_fields
set()

Upvotes: 37

alex_noname
alex_noname

Reputation: 32183

As far as I know, there is no out-of-the-box solution for this. But since you have access to all "raw" passed data, including extra fields, in the pre root validator, you can put them there in a separate dictionary.

An example is taken from here. Thanks to @PrettyWood for providing it:

from typing import Any, Dict, Optional

from pydantic import BaseModel, Field, root_validator


class ComposeProject(BaseModel):
    version: Optional[str] = Field(alias='version')  # just to show that it supports alias too
    extra: Dict[str, Any]

    @root_validator(pre=True)
    def build_extra(cls, values: Dict[str, Any]) -> Dict[str, Any]:
        all_required_field_names = {field.alias for field in cls.__fields__.values() if field.alias != 'extra'}  # to support alias

        extra: Dict[str, Any] = {}
        for field_name in list(values):
            if field_name not in all_required_field_names:
                extra[field_name] = values.pop(field_name)
        values['extra'] = extra
        return values


project = ComposeProject(**{
  "version": "3",
  "services": ...
})
print(project.version)  # => '3'
print(project.extra)  # => {'services': Ellipsis}

Upvotes: 18

robcxyz
robcxyz

Reputation: 641

Pydantic extra fields behaviour was updated in their 2.0 release. Per their docs, you now don't need to do anything but set the model_config extra field to allow and then can use the model_extra field or __pydantic_extra__ instance attribute to get a dict of extra fields.

from pydantic import BaseModel, Field, ConfigDict


class Model(BaseModel):
    python_name: str = Field(alias="name")

    model_config = ConfigDict(
        extra='allow',
    )

m = Model(**{"name": "Name", "address": "bla bla", "post": "post"}).model_extra
assert m == {'address': 'bla bla', 'post': 'post'}

m = Model(**{"name": "Name", "foobar": "fizz"}).__pydantic_extra__
assert m == {'foobar': 'fizz'}

m = Model(**{"name": "Name"}).__pydantic_extra__
assert m == {}

Upvotes: 35

Related Questions