pydantic / pydantic Goto Github PK

View Code? Open in Web Editor NEW

18.6K 114.0 1.6K 105.47 MB

Data validation using Python type hints

Home Page: https://docs.pydantic.dev

License: MIT License

Makefile 0.13% Python 99.80% Shell 0.07%

validation parsing json-schema python37 python38 pydantic python39 python hints python310

pydantic's People

Contributors

Stargazers

Watchers

Forkers

svisser graingert rmunoz12 john-sandall mariku gustavo-ferreira mgu petroswork koxudaxi nathanielobrown drpoggi bendemaree anentropic gr1n robusgauli gitter-badger jaheba niconorsk oldpadavan cfkanesan nabla-c0d3 virobidou dmfigol aahancoc gangefors shellcat-zero nphyatt aalexandrov funk66 jasonkuhrt tiangolo pydevd hugoduncan vppuzakov nkonin optiz0r liiight uditjuneja dgasmith je-l lblonk erosennin mselee pilosus zupo kataev tyrylu potykion atheuz tigerwings jekirl sean1708 paul-ilyin noanylove brittx centryplan theenglishway j1king kinneyjb mussard heylenz theendsofinvention sebastianmika jamesob dmontagu euri10 cjw296 sevaho oucb i8enn lepercon5000 hmvp mdaffern pfrederiks somada141 josepaiva94 hanaasagi wongpat dand-oss vlcinsky blockchain-bobby bobronium beesitech lnaden alexxmagpie guenp mohi7solanki imvvvvvip loparevdv brakedust hbeemster volker48 peter-boers wozniakty getmr siloam bohea derridda enkoder amitbl

pydantic's Issues

parse config from non-BaseModel class

read the annotations recursively from a class and generate a validator at runtime:

# pydantic
from typing import Callable, Any, TypeVar

_T = TypeVar('_T')


def parse(cls: Callable[..., _T], config: Optional[BaseConfig] = None, **kwargs: Any) -> _T:
    """
    Process annotations on cls, run verification on kwargs and return a new
    instance of cls or raise a validation error.
    """
    ...

# model.py
import pydantic
import typing

class OtherModel(typing.NamedTuple):
    foo: int
    bar: int

class Model(typing.NamedTuple):
    foo: str
    bar: int
    baz: OtherModel


m = pydantic.parse(Model, foo='three', bar='3', baz={'foo': '1', 'bar': '2'})
print(m)

ignore_extra = False for settings

by default

Changing date format

What is the right way to do it? I used some monkeypatch:

from pydantic import datetime_parse

datetime_parse.date_re = re.compile(r'(?P<day>\d{1,2}).(?P<month>\d{1,2}).(?P<year>\d{4})$')

but it feels rather hacky. Looks like I can do it with validators but when I return something It still tries to create date from string after that when it's not a string anymore.

class CheckUserRequest(pydantic.BaseModel):
    cardNumber: str
    lastName: str
    birthday: date

    @validator("birthday", pre=True)
    def parse_birthday(cls, value):
        return datetime.strptime(
            value,
            "%d.%m.%Y"
        ).date()

I get:

pydantic.exceptions.ValidationError: error validating input
birthday:
  int() argument must be a string, a bytes-like object or a number, not 'datetime.date' (error_type=TypeError track=date)

Check for invalid field names in validators

ref ed2b3f2

check in MetaModel.

List[T] validation is not strict enough

with this kind of model :

class Foo(BaseModel):
    items: List[int]

Foo(items='') does not throw any error. Looking at the code, we do not check the list is a list, but we iterate over it and check the elements. Any empty iterable can be used for List[T]

way to force validators to be run even if value is missing

eg. to generate a default or validate the default value.

support typing.Tuple

allow typed tuples. Note: variable length tuples use the ellipsis - need to make sure this doesn't interfere with required fields.

strip white space from strings

by default. Allow as argument to constr

support NewType

Pydantic does not seem to support NewType.

Given this code:

import typing as t
import pydantic

BossId = t.NewType('BossId', str)
EmployeeId = t.NewType('EmployeeId', str)
CustomerId = t.NewType('CustomerId', str)

class DirectReport(pydantic.BaseModel):
    boss: BossId
    employee: EmployeeId

The code will hit an error as it's being parsed:

my_code.py:8: in <module>
    class DirectReport(pydantic.BaseModel):
.tox/py36/lib/python3.6/site-packages/pydantic/main.py:82: in __new__
    config=config,
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:86: in infer
    description=field_config and field_config.get('description'),
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:72: in __init__
    self._prepare(class_validators)
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:111: in _prepare
    self._populate_validators(class_validators)
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:190: in _populate_validators
    *(get_validators() if get_validators else find_validators(self.type_)),
.tox/py36/lib/python3.6/site-packages/pydantic/validators.py:156: in find_validators
    raise TypeError(f'error checking inheritance of {type_!r} (type: {display_as_type(type_)})') from e
E   TypeError: error checking inheritance of <function NewType.<locals>.new_type at 0x407b26388> (type: function)

I currently am using Pydantic with this work around which involves monkey-patching:

import types
import pydantic

def find_validators(type_):
    if type_ is t.Any:
        return []
    for val_type, validators in pydantic.validators._VALIDATORS:
        try:
            if issubclass(type_, val_type):
                return validators
        except TypeError as e:
            if isinstance(type_, types.FunctionType):
                super_type = getattr(type_, '__supertype__')
                if super_type:
                    # Assume this is a NewType
                    if super_type != type_:
                        return find_validators(super_type)
            raise TypeError('error checking inheritance of '
                            f'{type_!r} (type: '
                            f'{pydantic.validators.display_as_type(type_)})'
                            ) from e
    raise pydantic.validators.ConfigError(f'no validator found for {type_}')


# Alters behavior of Pydantic so NewType will work (!!)
pydantic.validators.find_validators = find_validators
pydantic.fields.find_validators = find_validators

rename Module

Module doesn't create a module, should be renamed.

values include, exclude

make values a function and allow include and exclude arguments.

Problem with inheritance and Config.fields

from pydantic import BaseModel


class Foo(BaseModel):
    a: int


class Bar(Foo):
    b: str

    class Config:
        fields = {
            'a': 'aaa',
            'b': 'bbb',
        }


print(Bar(aaa=1, bbb='s'))

Doesn't work.

should give explicit error if fields which aren't defined are referenced in Config.fields
should allow aliases for fields in inherited models.

OPTIONS info

For OPTIONS requests:

alt names
descriptions
choices display text

validators being called twice

validators of the form @validator('last_updated', pre=True, always=True) seem to be being called twice.

Attribute cannot have the same name as its type

I am trying to make the class attribute having the same name as its type. The following code does not work and show the ConfigError

class A(BaseModel):
    pass

class B(BaseModel):
    A: A = None

ConfigError: unable to infer type for attribute "A"

However, if I make the attribute no default value then it works

class A(BaseModel):
    pass

class B(BaseModel):
    A: A

Is this behavior intended?

python2 compatibility?

Hi,

This may be a weird question for a project that's targetting type annotations in python3, but what's your opinion on Python 2 support?

I ask this because we're considering picking up a new validation / serialization library for a project we work on, and python2 is a requirement.

http://transmute-core.readthedocs.io/en/latest/

As you probably know, it's just a matter of a function which attaches a __annotations__ parameter to the object. In transmute, we just provide a decorator "@attributes" that does this.

If you're open to Python 2 support, but just don't want to do the work to enable it, I can give it a shot.

Error with inheritance

from pydantic.main import BaseModel

class Foo(BaseModel):
    a: float = ...

class Bar(Foo):
    x: float = 12.3
    a = 123

...
TypeError: issubclass() arg 1 must be a class

I tried out Pydantic to see if my data had the right type, but I was surprised that it cast the data to the right type. I would expect an error to be raised if the value is a list (say), instead I got the string representation of that list. For example, I found this string validator:

def str_validator(v) -> str:
    if isinstance(v, (str, NoneType)):
        return v
    elif isinstance(v, bytes):
        return v.decode()
    return str(v)  # <-- Should this raise a validation error, or cast to a string?

Maybe this is intended behaviour, but I think it should raise a validation error. Maybe a strict option can be added to the Config class?

Pickling Models

Add __getsetate__ and __setstate__ to make pickling models easy

validate method to validate entire result

so you can define def validate(self): which can be used to validate entire model.

inline model creation

class LabelModel(BaseModel):
    name: str
    machine_name: str

class XModel(BaseModel):
    labels1: List[LabelModel]
    labels2: List[create_model('LabelModel', name=(str, ...), machine_name=(str, ...))]

Here labels1 is the same as labels2

Validate on assignment to attribute

Currently pydantic does not validate when I assign to an attribute. Is it possible to perform validations in this case, too?

Length limit for lists and dicts

I guess through config

Add support for positional arguments

It seems Pydantic does not support positional arguments in the initializers it gives each BaseModel.

This is probably intentional and just a difference in philosophy. However it would be nice if it was supported.

In case this never changes, I'll share my current work around in case it helps anyone who Googles this. I'm extending BaseModel and adding an __init__ function which checks the fields property. This is probably not the fastest solution because it creates a list from the fields each time:

import pydantic

class BaseModel(pydantic.BaseModel):

    def __init__(self, *args, **kwargs):
        pos_args = list(self.fields.keys())

        for i in range(len(args)):
            if i >= len(pos_args):
                raise TypeError(f'__init__ takes {len(pos_args)} '
                                'positional arguments but 2 were given.')
            name = pos_args[i]
            if name in kwargs:
                raise TypeError(f'{name} cannot be both a keyword and '
                                'positional argument.')
            kwargs[name] = args[i]

        return super().__init__(**kwargs)

cannot infer type of bool False

class Settings(BaseSettings):
    DEBUG = False

pydantic.exceptions.ConfigError: unable to infer type for attribute "DEBUG"

Allow arbitrary content types when deserializing

My data is compressed and serialized to MsgPack.
I'd like be able to use parse_raw() with my choice of compression algorithm as well.
Should I simply override parse_raw? That works as a workaround.
Kombu has a registry for serialization methods and compression methods.
Should pydantic have one as well?
Another (and in my opinion, better) alternative is to allow to pass a Deserializer object through the model's constructor.

Assigning a pydantic type to an variable of a different type does not raise a mypy error

I have:

from pydantic import BaseModel

class Model(BaseModel):
    id: int


model = Model()
my_int: int
reveal_type(my_int)
reveal_type(model)
my_int = model
reveal_type(my_int)

and when I run mypy --strict-optional -s on it I get:

rpws/types.py:14: error: Revealed type is 'builtins.int'
rpws/types.py:15: error: Revealed type is 'rpws.types.Model'
rpws/types.py:17: error: Revealed type is '<nothing>'

I.e no type error. If I don't inherit from BaseModel I get the correct error:

class Model(object):
    id: int


model = Model()
my_int: int
reveal_type(my_int)
reveal_type(model)
my_int = model
reveal_type(my_int)

yields:

rpws/types.py:14: error: Revealed type is 'builtins.int'
rpws/types.py:15: error: Revealed type is 'rpws.types.Model'
rpws/types.py:16: error: Incompatible types in assignment (expression has type "Model", variable has type "int")
rpws/types.py:17: error: Revealed type is 'builtins.int'

Any advice? Am I doing something wrong? If this is a bug, how would I go about debugging it/where would I fix it? I think pydantic is a beautiful solution and I would love to help out.

Type erasure in parametrized sequences like List[Union[T, T1,…]]

Version: pydantic 0.3

from typing import Union, List
from pydantic import BaseModel


class First(BaseModel):
    a: int = 1


class Second(First):
    b: str = 2


class Container(BaseModel):
    c: List[Union[First, Second]] = []


first = First()
second = Second()

container = Container(c=[first, second])

assert type(container.c[0]) == First
assert type(container.c[1]) == Second, "Type erasure!"

typerror in example: Incompatible types in assignment (expression has type "ellipsis", variable has type "int")

with:

from datetime import datetime
from pydantic import BaseModel

class UserModel(BaseModel):
    id: int = ...
    name = 'John Doe'
    signup_ts: datetime = None

external_data = {'id': '123', 'signup_ts': '2017-06-01 12:22'}
user = UserModel(**external_data)
print(user)
# > UserModel id=123 name='John Doe' signup_ts=datetime.datetime(2017, 6, 1, 12, 22)
print(user.id)
# > 123

I get:

error: Incompatible types in assignment (expression has type "ellipsis", variable has type "int")
error: Incompatible types in assignment (expression has type None, variable has type "datetime")

class validators for entire objects

allow validation of entire list/dict rather than each element.

NamedTuple style Models (immutable instances, _replace method, etc)

To support immutable data structures there will need to be some slight tweaks to the BaseModel:

you cannot override the constructor, and would not want to, as there should be a type safe way to construct instances. Will need a staticmethod: eg parse.

from datetime import datetime
from typing import NamedTuple
from pydantic import ImmutableModel

class UserModel(ImmutableModel, NamedTuple):
    id: int = ...
    name = 'John Doe'
    signup_ts: datetime = None

external_data = {'id': '123', 'signup_ts': '2017-06-01 12:22'}
user = UserModel.parse(**external_data)
print(user)
# > UserModel id=123 name='John Doe' signup_ts=datetime.datetime(2017, 6, 1, 12, 22)
print(user.id)
# > 123
print(user[0])
# > 123
print(user._replace(name='foo'))
# > UserModel id=123 name='foo' signup_ts=datetime.datetime(2017, 6, 1, 12, 22)

No validation for UUIDs

There is not validator for UUIDs.
Can we add one?

depreciate values() and replace with dict()

values() is confusing as it can be confused with the method of the same name on dicts.

dict() (with same signature) would make sense.

Better error for None passed to constr field

No missing attribute error by mypy

mypy does not warn missing attribute when inheriting BaseModel.

from pydantic import BaseModel

class A:
    x: int

class B(BaseModel):
    x: int

A().y
B(x=1).y

main.py:9: error: "A" has no attribute "y"

Can I check missing attribute for B().y?

typing.Set

Add support for Set like List

override set and modify values

can also throw exception if the attribute doesn't exist.

Multiple inheritance with BaseModel

I'm trying to implement multiple inheritance with BaseModel, but every way I've tried failed. A minimal example:

class A():                     
    def __init__(self, a1, a2, *args, **kwargs):
        print('In A.')                        
        print(a1, a2, args, kwargs)          
        super().__init__()                   
        print('Leaving A.')

class B(pydantic.BaseModel):
    desc: pydantic.constr(min_length=3)
    def __init__(self, desc, *args, **kwargs):
        print('In B')
        super().__init__(*args, **kwargs)
        print('Leaving B.')

class C(B, A):
    def __init__(self, tmp, *args, **kwargs):
        print('In C')
        super().__init__(*args, **kwargs)
        print('Leaving C')


class B():                  
    desc: str                                
    def __init__(self, desc, *args, **kwargs):
        print('In B')
        self.desc = desc
        super().__init__(*args, **kwargs)
        print('Leaving B.')

If I use B inheriting from BaseModel, instantiation of C fails:
c = C(1,2,3,4)

In C
In B

TypeError Traceback (most recent call last)
in ()
----> 1 c = C(1,2,3,4)

in init(self, tmp, *args, **kwargs)
2 def init(self, tmp, *args, **kwargs):
3 print('In C')
----> 4 super().init(*args, **kwargs)
5 print('Leaving C')

in init(self, desc, *args, **kwargs)
3 def init(self, desc, *args, **kwargs):
4 print('In B')
----> 5 super().init(*args, **kwargs)
6 print('Leaving B.')

TypeError: init() takes 1 positional argument but 3 were given

However, if I use B only inheriting from object, everything works fine. Is there a way to make this work?

mypy does not run on pydantic

pydantic/datetime_parse.py:78: error: Result type of / incompatible in assignment
pydantic/datetime_parse.py:94: error: Argument 1 to "match" of "Pattern" has incompatible type "Union[str, int, float]"; expected "str"
pydantic/datetime_parse.py:111: error: Argument 1 to "match" of "Pattern" has incompatible type "Union[str, int, float]"; expected "str"
pydantic/datetime_parse.py:118: error: Value expression in dictionary comprehension has incompatible type "int"; expected type "str"
pydantic/datetime_parse.py:119: error: Argument 1 to "time" has incompatible type **Dict[str, str]; expected "int"
pydantic/datetime_parse.py:119: error: Argument 1 to "time" has incompatible type **Dict[str, str]; expected "tzinfo"
pydantic/datetime_parse.py:136: error: Argument 1 to "match" of "Pattern" has incompatible type "Union[str, int, float]"; expected "str"
pydantic/datetime_parse.py:152: error: Value expression in dictionary comprehension has incompatible type "int"; expected type "str"
pydantic/datetime_parse.py:154: error: Argument 1 to "datetime" has incompatible type **Dict[str, str]; expected "int"
pydantic/datetime_parse.py:154: error: Argument 1 to "datetime" has incompatible type **Dict[str, str]; expected "tzinfo"
pydantic/datetime_parse.py:179: error: Value expression in dictionary comprehension has incompatible type "float"; expected type "str"
pydantic/datetime_parse.py:180: error: Argument 1 to "timedelta" has incompatible type **Dict[str, str]; expected "float"
pydantic/utils.py:70: error: Incompatible types in assignment (expression has type "Union[str, Dict[<nothing>, <nothing>]]", variable has type "str")
pydantic/utils.py:74: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"
pydantic/fields.py:27: error: The return type of "__init__" must be None
pydantic/fields.py:45: error: Need type annotation for variable
pydantic/fields.py:52: error: Need type annotation for variable
pydantic/main.py:80: error: Need type annotation for variable

EMAIL_REGEX does not match email with internationalized domain name.

For example, a@vidéotron.ca is considered an invalid email address, while [email protected] is considered valid. Note that the validate_email module accepts both as valid, which is the correct behaviour.

Missing HISTORY.rst

pydantic installs fine from pip but via a dependency in setup.py it fails with a missing HISTORY.rst due to your long_description in setup.py. Basically, you need a MANIFEST.in that includes that file.

Processing pydantic-0.6.2.tar.gz
Writing /var/folders/4j/00jv8sg138n61hsj6ppf60pm0000gn/T/easy_install-o7rp3h7o/pydantic-0.6.2/setup.cfg
Running pydantic-0.6.2/setup.py -q bdist_egg --dist-dir /var/folders/4j/00jv8sg138n61hsj6ppf60pm0000gn/T/easy_install-o7rp3h7o/pydantic-0.6.2/egg-dist-tmp-7bd8a1a8
error: [Errno 2] No such file or directory: '/private/var/folders/4j/00jv8sg138n61hsj6ppf60pm0000gn/T/easy_install-o7rp3h7o/pydantic-0.6.2/HISTORY.rst'

error using Any

Dict[str, Any] as a type gives

    class RecipientModel(BaseModel):
  File "/home/samuel/code/morpheus/env/lib/python3.6/site-packages/pydantic/main.py", line 57, in __new__
    class_validators=class_validators,
  File "/home/samuel/code/morpheus/env/lib/python3.6/site-packages/pydantic/fields.py", line 60, in infer
    name=name
  File "/home/samuel/code/morpheus/env/lib/python3.6/site-packages/pydantic/fields.py", line 50, in __init__
    self._prepare(class_validators or {})
  File "/home/samuel/code/morpheus/env/lib/python3.6/site-packages/pydantic/fields.py", line 75, in _prepare
    self._populate_validators(class_validators)
  File "/home/samuel/code/morpheus/env/lib/python3.6/site-packages/pydantic/fields.py", line 148, in _populate_validators
    *(get_validators() if get_validators else find_validators(self.type_)),
  File "/home/samuel/code/morpheus/env/lib/python3.6/site-packages/pydantic/validators.py", line 117, in find_validators
    if issubclass(type_, val_type):
TypeError: issubclass() arg 1 must be a class

improve the error returned if a list is "validated" as a dict

currently gets

dictionary update sequence element #0 has length 9; 2 is required (error_type=ValueError track=_Action)

Which is very unclear.

parse_obj(obj) is just an aliases to __init__ which gives a validation error if the argument is not a dict
parse_json(data) parses a json string using json or ujson if it exists
parse_msgpack(data) as above but with msgpack
parse_content_type(data, content_type) parses data based on content type: json, msgpack perhaps xml if there's an elegant equivilence.

remove track and index from error if they're null

Should make error responses less verbose.

pydantic / pydantic Goto Github PK

pydantic's People

Contributors

Stargazers

Watchers

Forkers

pydantic's Issues

In C In B

Recommend Projects

Recommend Topics

Recommend Org

In C
In B