Giter VIP home page Giter VIP logo

middle's Introduction

middle

PyPI - Status PyPI Package latest release Supported versions Travis-CI Build Status AppVeyor Build Status Documentation Status Coverage Status Codacy Grade Packages status

Flexible, extensible Python data structures for general usage. Get data in and out, reliably, without boilerplate and with speed!

middle stands on the shoulders of attrs and aims to be as simple as possible to get data from complex objects to Python primitives and vice-versa, with validators, converters, a lot of sugar and other utilities! middle can be used with your preferred web framework, background job application, configuration parser and more!

Sneak peek

The most simple example of middle and some of its features (using Python 3.6+ syntax):

>>> import typing
>>> import middle

>>> class Address(middle.Model):
...     street_name: str
...     number: typing.Optional[int]
...     city: str

>>> class Person(middle.Model):
...     name: str
...     age: int
...     address: typing.Dict[str, Address]

>>> data = {
...     "name": "John Doe",
...     "age": 42,
...     "address": {
...         "home": {
...             "street_name": "Foo St",
...             "number": None,
...             "city": "Python Park"
...         },
...         "work": {
...             "street_name": "Bar Blvd",
...             "number": "1337",
...             "city": "Park City"
...         }
...     }
... }

>>> person = Person(data)

>>> person
Person(name='John Doe', age=42, address={'home': Address(street_name='Foo St', number=None, city='Python Park'), 'work': Address(street_name='Bar Blvd', number=1337, city='Park City')})

>>> middle.asdict(person)
{'name': 'John Doe', 'age': 42, 'address': {'home': {'street_name': 'Foo St', 'number': None, 'city': 'Python Park'}, 'work': {'street_name': 'Bar Blvd', 'number': 1337, 'city': 'Park City'}}}

Wanted a more complex example, with Python 3.5 compatible syntax? For sure!

>>> from typing import Dict, List
>>> import middle

>>> class Game(middle.Model):
...     name: str = middle.field()
...     score: float = middle.field(minimum=0, maximum=10)
...     resolution_tested: str = middle.field(pattern="^\d+x\d+$")
...     genre: List[str] = middle.field(unique_items=True)
...     rating: Dict[str, float] = middle.field(max_properties=5)

>>> data = {
...     "name": "Cities: Skylines",
...     "score": 9.0,
...     "resolution_tested": "1920x1200",
...     "genre": ["Simulators", "City Building"],
...     "rating": {
...         "IGN": 8.5,
...         "Gamespot": 8.0,
...         "Steam": 4.5
...     }
... }

>>> game = Game(**data)

>>> game
Game(name='Cities: Skylines', score=9.0, resolution_tested='1920x1200', genre=['Simulators', 'City Building'], rating={'IGN': 8.5, 'Gamespot': 8.0, 'Steam': 4.5})

>>> middle.asdict(game)
{'name': 'Cities: Skylines', 'score': 9.0, 'resolution_tested': '1920x1200', 'genre': ['Simulators', 'City Building'], 'rating': {'IGN': 8.5, 'Gamespot': 8.0, 'Steam': 4.5}}

middle is flexible enough to understand Enum, nested models and a large variety of types declared on the typing module out of the box. Also, you can extend it to your own classes!

Warning

IMPORTANT: middle is in very early stages of development. There are some requirements (like python-dateutil) that would not be required in future releases; as there's a lot of functionalities that needs to be implemented and some known misbehaviors to be addressed, not to mention it needs a lot of testing before moving to any other status rather than alpha.

TODO

  • Alias options (keys) to populate classes;
  • Read-only and write-only fields;
  • Better error handling (almost everywhere);
  • Create a benchmark suite against other solutions;
  • Formatters are still missing;
  • Possibility to "cast" an instance to another instance where the original object is a subclass of it;

Done

  • If possible, fine grain the converters, so a str input value of {} doesn't end up as str({});
  • Get date and datetime converters to be customizable, instead of an if isinstance statement;
  • Implement more validators and a registerable for more metadata options;
  • Implement a better "type dispatcher" based on more complex rules (other than type(field.type) delivered by functools.singledispatch) because the typing module has changed a bit between Python 3.6 and 3.7;
  • Support more types (typing.Tuple, decimal.Decimal);
  • Get 100% (or closer) in code coverage;
  • Lots of documentation;
  • Python 3.5 support (with the exception of Windows platforms, see warning for Windows developers below);

Future discussions

  • In Python 3.7, a neat feature was added: dataclasses. I know it sounds really awesome to not depend on a 3rd-party library - such as attrs, but the latest provides a lot of functionalities that can't be found on Python 3.7 dataclasses (for now), so I'll leave this open for further discussion.

Warning for Windows developers

If you're using Windows and Python 3.5, I think middle would not work well for you. CI in AppVeyor was disabled for Python 3.5 because of this issue. If Guido doesn't care, why should I (or you) ?

Documentation

https://middle.readthedocs.io/en/latest/

Useful links

Inspirations and thanks

Some libs that inspired the creation of middle:

  • attrs: how such a simple library can be such flexible, extendable and fast?
  • cattrs: for its speed on creating attrs instances from dict and to instances again;
  • pydantic: for such pythonic and beautiful approach on creating classes using typing hints;
  • mashmallow: it is one of the most feature rich modelling APIs I've seen;
  • apistar: it's almost magical!
  • Sanic: "Gotta go fast!"
  • ionelmc/cookiecutter-pylibrary: The most complete (or interesting) cookiecutter template I found so far (make sure to read this article too);

License

middle is a free software distributed under the MIT license.

middle's People

Contributors

pyup-bot avatar vltr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

mortbauer

middle's Issues

Support for postponed evaluation of annotations

Pep 563 defines the ability to lazily define type annotations.

Say I have something like this:

class Foo:
    name: str

class Bar:
    foos: List[Foo]

This would work fine. But what if I am importing from another library, or have them in defined in a different order?

class Bar:
    foos: List[Foo]

class Foo:
    name: str

This will break.

Pep 563 addresses this with from __future__ import annotations. Basically it passes the type annotation as a string to then be resolved at runtime with: typing.get_type_hints().

Therefore, rather than grabbing self.__annotations__, you do something like what I did in pydiggy:

annotations = get_type_hints(self, globalns=globals(), localns=localns)

I think for middle it would be a relatively simple change, but it is a 3.7+ change. Just a thought.

Drop Python 3.5 support

Drop Python 3.5 support for middle - only 4 downloads since December, so there's a lot of code just to support this version whereas the project would have a huge benefit using Python 3.6+ - like f strings.

Proper benchmarking

Create a proper benchmark suite for middle against other similar solutions

Respect `mro` order

middle should respect and work with proper class inheritance and mro, like being able to "cast" a child object to its parent type (or extract from it) and so on.

Problem with `Optional` in more complex definitions

middle/tests/test_types.py

Lines 953 to 969 in d3fc3f9

@pytest.mark.xfail # ! TODO - this shoud not fail on pytest.raises
def test_tuple_no_optional():
opt_tup = t.Tuple[float, int]
class TestModel(middle.Model):
value = middle.field(
type=t.Dict[int, t.Tuple[opt_tup, opt_tup, opt_tup, opt_tup]]
)
inst = TestModel(value={1: [[0.9, 1], [0.9, 1], [2.1, 2], [2.1, 2]]})
assert inst.value[1][0] == (0.9, 1)
assert inst.value[1][1] == (0.9, 1)
assert inst.value[1][2] == (2.1, 2)
assert inst.value[1][3] == (2.1, 2)
with pytest.raises(ValidationError):
TestModel(value={5: [[0.9, 1], None, [2.1, 2], None]})

The definition above should produce a ValidationError, since it's not Optional, hence the values on given for Tuples must not be null or None.

Read-only and write-only fields

Implement some sort of filters to display or hide field values depending the situation (input or output), much like marshmallow does

Setting fixed length to a string

  • middle version: 0.2.2
  • Python version: 3.7.3
  • Operating System: Alpine Linux v3.9

Description

Set a length requirement for a str

What I Did

class Foo(middle.Model):
    bar: str = middle.field(min_length=4, max_length=4)
ValueError: The 'min_length' keyword value must not be equal or greater than the 'max_length' keyword value. 

This seems like it should be possible. Or, perhaps:

bar: str = middle.field(length=4)

Does not work with EnumMeta

  • middle version: 0.2.2
  • Python version: 3.7
  • Operating System: Linux

Description

I tried to implement middle with an Enum that uses EnumMeta to implement a custom metaclass.

What I Did

Traceback (most recent call last):
  File "/tmp/p.py", line 28, in <module>
    class Item(middle.Model):
  File "/home/adam/.virtualenvs/TEMP37/lib/python3.7/site-packages/middle/model.py", line 95, in __new__
    _implement_converters(f, k, annotations)
  File "/home/adam/.virtualenvs/TEMP37/lib/python3.7/site-packages/middle/model.py", line 200, in _implement_converters
    converter_fn = converter(annotations.get(key))
  File "/home/adam/.virtualenvs/TEMP37/lib/python3.7/site-packages/middle/dispatch.py", line 25, in __call__
    return fn(*args)
  File "/home/adam/.virtualenvs/TEMP37/lib/python3.7/site-packages/middle/converters.py", line 154, in converter
    raise InvalidType()
middle.exceptions.InvalidType: The required type is not supported by ``middle``. Please, see https://middle.readthedocs.io/en/latest/extending.html for more information on how to add your own types.

Snippets

Here is the code that produced the exception.

from enum import Enum
from enum import EnumMeta
from enum import auto

import random
import middle


class RandomMetaclass(EnumMeta):
    @property
    def RANDOM(self):
        member_map = [x for x in self._member_map_.values()]
        return random.choice(member_map)


class BaseEnum(Enum, metaclass=RandomMetaclass):
    pass


class Status(BaseEnum):
    DRAFT = auto()
    PROPOSED = auto()
    ACCEPTED = auto()
    REJECTED = auto()
    ABANDONED = auto()


class Item(middle.Model):
    title: str
    status: Status

item = Item(title="FooBar", status=Status.RANDOM)
print(item)

I confirmed that it works if I do not use EnumMeta.

class Status(Enum):
    ...

item = Item(title="FooBar", status=Status.DRAFT)

The snippet provided (if any) can be used to create unit tests for this issue.

Types support

Support more types (provided in the Python standard library), if possible, like:

  • Any
  • Optional
  • Union
  • Decimal

Aliases and alternate keys

Add some kind of alias options to populate classes or read data (like author from name, as one example)

Drop attrs depepdency

This looks like a huge task, so there's no way to "minimize" words without saying that there's a need to drop attrs as a dependency from middle to be able to generate more fined grained classes, aiming Python 3.6+ only and allowing some enhancements, like a better error control system, as described and commented in #4.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.