Hi all!

I'm convinced this project is the future, should we start a donation campaign and recruit a hacker like 3 months to make a release of this ?

Would this help make this an actual ongoing project ?

Best regards

Top level API should be ideally similar to form/serializer, but it should be informed by a "second level" API which is purely functional.

I have something like this in my head:

# Read mode
a = Adapter()
data = a.adapt(initial={'instance': model_instance}) 

# Write mode
a = Adapter()
data = a.adapt(initial={'instance': model_instance}, data=request.data)
processed_data, errors = a.validate(data)
assert not errors
output = a.process(processed_data)
assert output['instance'] == model_instance

Note that no step needs to use a.blah for any blah except the structure of a.

Hi all !

I think it would be cool to have a weekly hangout session where we can discuss live and synchronize our brains to help going in the same direction.

I know time zones want to make it impossible, but if you want to try anyway please leave your timezone and usual availabilities in this issue ;)

@ian-foote

This is linked to #13.

Naming is the key thing here, I believe we want to provide toolsets that make providing flattened representations of things to the view layer easy to understand. "forms" and "serializers" don't always provide this context so well.

This could be a way where we're able to define the difference between the asymmetrical use cases.

https://github.com/schematics/schematics

http://schematics.readthedocs.io/en/latest/index.html

Looks like this gets a lot of things right that we're looking at. The API is nice, the to_primitive is a pretty good naming convention, and the validation/coercion/conversion logic seems clean. It lacks introspectability and good complex validation support as far as I can see, but it's pretty good!

• Construction
• Existing data input
• New data input
• Validation (also does data processing)
• Rendering
• Data output (to models)

Do you want to have an honest discussion about how the previous development cycle went ?

I'll start this with an anecdote: when I rewrote django-dynamic-fields into facond, I still had Form/Rule/Action/Conditions. When the code got to the point where i was satisfied with it i wrote the tutorial. This is how I figured this lib had just no need for the "Rule" component. Hopefuly, both python and js had serious unit test coverage, so I just removed the Rule stuff from Python and JS, changed all function signatures, and kept on fixing stuff until tests passed again. This strikes me: if i had started by writing the tutorial, I would not have wasted my time TDDing a component that I actually didn't need (the kind of things that would make me look silly in a retrospective right ? haha).

Currently, there's a lot of research documentation about low level components, a bit how "how it could be done", but isn't a lot of the fun about designing how we're going to use this package at a high level before digging into lower level components ?

Perhaps someone with experience about what this package is about (ie. has coded this kind of patterns in private or open source projects in the past) could write a tutorial for using this package ? yes, that means inventing the high level API from the top of their head for using the components which don't even exist yet ?

Sincerely hope this helps but if this doesn't make sense to you feel free to close ;)

FTR: How I failed at documentation driven design

https://github.com/WiserTogether/django-remote-forms (Django project)
https://github.com/insin/newforms (JS project which was heavily inspired by Django.forms)

Serialization is strictly the process of getting from a rich object to a bytestring (encoded in some format). Forms in particular make that bytestring rather complicated, and the deserialization step is funky. I'm not sure of the "right" terminology but I think we should think about two separate steps.

Firstly, we want to deconstruct and normalise our data set to a small subset of types. Some of these types are obvious - list, dict, str, int, float, bool, NoneType etc. Some are less clear whether they should be supported at this level - for example decimal.Decimal, collections.OrderedDict, datetime.datetime, set, or even more domain specific types like prices.Price. It's hard to know where to draw the line here - there's no perfect common set of data types which are supported by all possible encodings (renderers). Given that the HTMLForm/x-www-form-encoded renderer will need to do some custom transformations even for simple types as it depends on how things are being rendered by the widget (e.g. ['on', ''] -> [True, False] for checkboxes), I think it's clear we need to consider this as two steps. Perhaps we have a "core" set of data types which are valid return types from a serialization object and all renderers "must" support those types, but we allow renderers to become aware of other data types so they can support them.

Pseudocode:

s = MyNativeTypesSerializer(initial)
data = s.serialize()
json_blob = JSONRenderer().render(data)
html_form = FormRenderer(widgets).render(data)
JSONRenderer().can_render(data)  # True

s = MyRichSerializer
data = s.serialize()
assert isinstance(data['created'], datetime)
JSONRenderer().can_render(data)  # False

JSONRenderer.register_type_encoder(datetime, some_transformation_func)
JSONRenderer().can_render(data)  # True
# OR
JSONRenderer(type_encoders={datetime: some_transformation_func}).can_render(data)  # True

This obviously has some overlap with DjangoJSONEncoder which knows how to handle various data types such as UUID, datetime, lazy strings etc.

Whilst it would be possible, it should be strongly advised that you do not have type encoders for high level types such as instances of models.

Note that it's entirely possible here I'm just rehashing renderers/parsers from DRF, perhaps with some other extensibility ideas and we can pretty much use those for rendering. I think my main change is that perhaps I'd like to see .render(structure, data) or something so that the form renderer is aware of the underlying configuration of the form for example. I probably need to do more research into how DRF handles these layers, and the content negotiation for them.

When we eventually start writing code there should definitely be a license. And since this is intended to be merged into Django core eventually I think there is no other choice than the MIT license.

Some use cases for DRF serializers are different dependant on the use case.

For example:

User Input:

• Validating and cleaning input data

Service output:

• serialising model instances into external representations for particular views (without validation or cleaning)
• Providing modular representational "components" for use in various views and templates.

As a lot of people use DRF serializers to reduce richer objects to a simple set of key/values, often with different keys or computed values to their internal equivalent, this use case can bring a lot of value to serial-forms and replace the current use of DRF serializers, which arguably isn't quite the right tool for the job.

Another possibility is to make everything an adapter, which can have adapters who know about their parent, in which case steps just adapters methods or plain callbacks, which can orchestrate other steps of other adapters which are in a mapping structure, and defining a step is just defining a method or setting callbacks which may depend on methods priorly executed without errors because going through .steps.method() creates a safe clone (thanks @Ian-Foote !)

A step is a callback, but steps is a magic python object, it just calls a step callback(s) in an safe (clone) adapter context.

https://gist.github.com/jpic/9f9ef74e5ab739b4ffd04ecc373d1d55

In this case everything is an adapter, and executing a method through steps do a safe clone before actually calling the callback, that's why we need not to worry about it from the callbacks themselves. But if you want to code like this, jeez, there will be a lot of python magic ....

Is this more what it should look like ?

Hi all,

I stumbled accross the glom module which seems to do what we want, except that so far it supports Django models (and relations) and QuerySets, but not forms.

What do you think ?

Sorry if this has already been posted, but this is kind of a reference for a lot of people: https://react.rocks/example/react-jsonschema-form

"Form generation and validation from JSON schema. Includes nesting, arrays. Introduces UI-schema concept to customize widget selection, CSS classes."

class Validator(object):
    def validate(self, value):
        pass

    def clean(self, value):
        self.validate(value)
        return value

class IntegerCleaner(Validator):
    def clean(self, value):
        try:
            value = int(value)
        except ValueError:
            raise ValidationError(...)
        return value

class FunctionValidator(Validator):
    def __init__(self, func):
        self.func = func
    def validate(self, value):
        func(value)

Not 100% sure about the split between clean and validate but there is a difference between the two. Perhaps instead we have cleaners and validators, one of which changes and one of which doesn't, but they're chainable together.

Related but not necessarily belonging in this tree is idea of a shaper which is a multivalued cleaner which returns different data shape. Then again perhaps these should be in the validator tree so subsequent validators can use the changed shape. Shaping is also about input/output though so these lines are blurry.

I think we didn't explicitly mention how extra, non-ORM data should be handled.

On the construction step it says in the docs:

It should be easy to modify the set of fields on a per-request basis.

In "Existing Data" it also says:

This includes but is not limited to field defaults, initial data and model instances.

I still think we should explicitly document especially how the current request (and user) can be incorporated into the adapter since in my experience (reading #django) that is one the first things where new users get frustrated.

One goal of this project is to improve the interface to models and other data sources.

These were mentioned in #17:

• xmodelform - particularly form field factories
• django-betterforms - particularly MultiForm and MultiModelForm

Create a prototype of the validation tree, allowing similar chaining of validation/cleaning functions similar to what Naval can do (see #21).

I don't know what would be the proper title for this issue, but it's about something i've been doing manually over and over again most of my life.

Here goes: http://facond.readthedocs.io/en/master/tutorial.html#what-this-is

Do you think it's still time we had something like that to the specs ?

Surely, this will make this app more valuable than competiting libs, IMHO.

As soon as there is some meaningful documentation we should add it to readthedocs.

Considering my current understanding of the adapters pattern - waiting to stand corrected - we're going to define adapter mapping trees.

Will it exclusively go in one way, ie:

from django.db.models import Person
# add returns a clone of the payload, so we're instanciating 2 payloads here
# because the FormView adapter's post_add() will add(Form adapter, clone=False)
# if it's not already there ? 
p = Payload.factory(instance=Person()).add('django.views.ModelFormView')
# executing a step returns a clone of the payload, but we don't care:
# we have an adapter mapping on data, a request, and we want a response
assert p.steps.response(request=request).response

The tutorial demonstrates how the above would be possible, but it might look like this (not tested code, obviously clumsy):

• import Person model
• make a payload with an empty instance,
• because it's a model, factory will add the django model adapter, which has post_add():
  • introspect the payload,
  • map payload.instance._meta.fields to keys of the Payload corresponding to field names
  • with the appropriate adapters for each field
  • ie. payload.map.name.adapters == [StringAdapter(max_length=255)]
  • add validate and clean steps
• add the modelformview adapter on the person payload
  • add the form adapter on the personpayload,
  • introspect the payload, and map keys corresponding form fields to model fields
  • ie. payload.map.name.adapters == [StringAdapter(max_length=255), TextFieldAdapter(label="my verbose name")]
  • add validate and clean steps on the person payload
  • add a render step
  • add the template adapter on the person,
    • which adds a render step with a default template name
    • that will be able to see other adapter's render outputs
  • add a response step which to orchestrate other steps,
    • but needs payload.request, and sets payload.response
    • in a clone as usual when a step is executed
    • unless clone=False, for calling steps from within steps
• execute the response step by adding request to the payload
  • modelformview adapter response will try to execute all prior steps,
  • if no errors are added by clean() step then process() step will save
  • if no errors response() will if errors on validate step show the form again, otherwise redirect to the detail view, if errors were added during process who knows and honnestly i leave it up to you what the default behaviour will be since it should be so easy to override not only the method but the default adapter registered for ModelFormView !!

Or, will it allow to build a nested adapter map, and then be able to generate a model class with another adapter ?

class Hobby(adapters.Payload):
    name = Payload(adapters=[StringAdapter()])

class Person(adapters.Payload):
    hobbies = Payload(map=[HobbyAdapter()])
    class Meta:
        adapters = [OnlyAllowHobbiesToBe('archery', 'django', 'music')]

p = Person().add('django.db.models.Model')
# custom step by django model adapter, optinal, sets payload.model if not already present
p.steps.modelize().model

Another possibility is to make everything an adapter, which can have adapters who know about their parent, in which case steps also are adapters, just they orchestrate the adapters which are in a mapping structure, and defining a step is just defining a method which may depend on methods priorly executed.

Sorry if this doesn't make any sense please correct me ;)

If that makes sense to you then you probably understand why i consider this million $ worth, in terms of refactoring, and code reusability.

This is a proposal for a perhaps surely more verbose description for this repository:

intropectable and mutable isomorphic adapter interface for common python logical input and output steps from validating a string to rendering a response with recursive adapter mapping and function-supporting IOC micro framework to make classes based around data and mappings of data work together by creating new adapters for them

This makes it looks like it's definitely going to solve some developer problem which means help them solve more human problems ?

From what I understand of the specs, this lib will be completely python generic, expect for the part where we load or save data in Django models, or generate a schema based on a Django model.

If this is correct, should we consider a rename of the project and aim for loose coupling with Django rather than tight coupling ?