koorchik / livr Goto Github PK

Исторически сложилось так, что первая реализация LIVR была написана на Perl, и LIVR во многом унаследовал его систему типов, а именно:

• Нет разницы между строкой и числом
• Нет логического типа (true, false)

Это приводит к шоку разработчиков, которые не ожидают, например что правило "positive_integer" пропустит строку "1".

Нужно выработать универсальную концепцию для стандартных правил (и расширить набор тестов), и предложить best practices для пользователей реализаций, так как они работают с конкретным языком, и должны позаботиться о типах.

Еще пара кейсов:

• { flag: {one_of: [true, false]} }
• { flag: {one_of: 4} }

My form allows arbitrary string value for one of the fields. And this field is optional (not required).
How can I configure a LIVR rule for that?
I tried to pass 'fieldName' => NULL and 'fieldName' => [] - both trigger an error after calling validate():
Exception: Rule [] not registered
and
Undefined index:

Inconsistent behavior sample:

http://livr-multi-playground.webbylab.com/#%7B%22rules%22%3A%22%7B%5Cn%20%20link%3A%20'url'%5Cn%7D%22%2C%22input%22%3A%22%7B%5Cn%20%20link%3A%20'https%3A%2F%2Freciepts%2F12345678.pdf'%5Cn%7D%22%7D

Hi!

I forked the vlbaluk java implementation, and fix some rule, create javax annotation validator and publish the maven central repository.

Can you add it to docs and playground?

Link: https://github.com/gaborkolarovics/livr-validator

I have been struggling with this, is there not a way to export/import rules?

I mean, language independent is nice, you have a common set of rules to work with in all languages, but is there not a way to export rules and import them in all languages that need to validate the same models?

Like:

{ age: [ 'positive_integer', {default: 18} ] }

Mb add some tests also

Inconsistent behavior sample (erlang @Prots):
http://livr-multi-playground.webbylab.com/#%7B%22rules%22%3A%22%7B%5Cn%20%20%20%20product_ids%3A%20'trololo'%5Cn%7D%22%2C%22input%22%3A%22%7B%5Cn%20%20%20%20product_ids%3A%201%5Cn%7D%22%7D

LIVR defines a couple of 'list' rules: list_of, list_of_objects, list_of_different_objects.
However, there is no test case for the situations when in the input data the list is empty ([]).
Most of the implementations treat it consistently - they consider [] to be a valid value.
However, the PHP implementation generates FORMAT_ERROR error in this case (only list_of and list_of_objects rules).

This happens because the test suit doesn't explicitly cover such cases.

The related PR: #28

Currently test suite has "empty_field". But "missed_field" should be added to check that not passed fields will not be validatated.

Here is an example of complex JSON Schema. To describe such schema in LIVR additional rules should be registered. Does it make sense to add some of them to the core?

{
    "id": "http://some.site.somewhere/entry-schema#",
    "$schema": "http://json-schema.org/draft-04/schema#",
    "description": "schema for an fstab entry",
    "type": "object",
    "required": [ "storage" ],
    "properties": {
        "storage": {
            "type": "object",
            "oneOf": [
                { "$ref": "#/definitions/diskDevice" },
                { "$ref": "#/definitions/diskUUID" },
                { "$ref": "#/definitions/nfs" },
                { "$ref": "#/definitions/tmpfs" }
            ]
        },
        "fstype": {
            "enum": [ "ext3", "ext4", "btrfs" ]
        },
        "options": {
            "type": "array",
            "minItems": 1,
            "items": { "type": "string" },
            "uniqueItems": true
        },
        "readonly": { "type": "boolean" }
    },
    "definitions": {
        "diskDevice": {
            "properties": {
                "type": { "enum": [ "disk" ] },
                "device": {
                    "type": "string",
                    "pattern": "^/dev/[^/]+(/[^/]+)*$"
                }
            },
            "required": [ "type", "device" ],
            "additionalProperties": false
        },
        "diskUUID": {
            "properties": {
                "type": { "enum": [ "disk" ] },
                "label": {
                    "type": "string",
                    "pattern": "^[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}$"
                }
            },
            "required": [ "type", "label" ],
            "additionalProperties": false
        },
        "nfs": {
            "properties": {
                "type": { "enum": [ "nfs" ] },
                "remotePath": {
                    "type": "string",
                    "pattern": "^(/[^/]+)+$"
                },
                "server": {
                    "type": "string",
                    "oneOf": [
                        { "format": "host-name" },
                        { "format": "ipv4" },
                        { "format": "ipv6" }
                    ]
                }
            },
            "required": [ "type", "server", "remotePath" ],
            "additionalProperties": false
        },
        "tmpfs": {
            "properties": {
                "type": { "enum": [ "tmpfs" ] },
                "sizeInMB": {
                    "type": "integer",
                    "minimum": 16,
                    "maximum": 512
                }
            },
            "required": [ "type", "sizeInMB" ],
            "additionalProperties": false
        }
    }
}

I'm using LIVR for validation of user data. Said users are not necessarily tech-savvy, and to them this error would potentially imply that the date they have entered is incorrect, not that it isn't a date/can't be formatted as a date.
A few suggestions that me and my colleague have come up with:

• "NOT_A_DATE"
• "WRONG_FORMAT"
• "WRONG_DATE_FORMAT"

It is likely that the olifer (Erlang validator) will no longer be supported because the developer has changed the stack. I completely rewrote it: https://github.com/erlangbureau/oliver
Can you add it to docs and playground?

"in" - ключове слово JavaScript, тому, щоб не було випадкових помилок і ключи хеша не потрібно було би записувати в лапках, його варто замінити на аналог. Наприклад
set
oneof
enum
list
within
from
in_set
...

Часто нет возможности описать все возможные ключи объекта с пошью правила nested_object, тем не менее нужно проверить что пришел именно объект. Нужно добавить правило которое пропускает объект с любыми ключами. Было бы не плохо чтобы оно поддерживало ограничение на размер\длину\количество полей или вес объекта в целом, так как проверить поле с помошью max_length после ужесточения проверок типов уже нельзя, выдаст FORMAT_ERROR.

Example

Hello,
very interesting project, and concept.

We are thinking about creating a validation language to validate invoices
(https://github.com/konik-io/konik).

• Deep nested Data Structures or trees
• A rules can depend on multiple fields somewhere else in the tree.
• Rule should be able do do mathematical calculations.

I was wondering if this could be accomplished with LIVR? Are they any other similar projects?

Не понятно какое из полей валидатор пропускает и почему не выдает оба в резалтсете:
http://webbylab.github.io/livr-playground/#{%22rules%22%3A%22{\n%20%20%20%20name%3A%20%27required%27%2C\n%20%20%20%20email%3A%20[%27required%27%2C%20%27email%27]%2C\n%20%20%20%20gender%3A%20{%20one_of%3A%20[[%27male%27%2C%20%27female%27]]%20}%2C\n%20%20%20%20phone%3A%20{max_length%3A%2010}%2C\n%20%20%20%20password%3A%20[%27required%27%2C%20{min_length%3A%2010}%20]%2C\n%20%20%20%20password2%3A%20{%20equal_to_field%3A%20%27password%27%20}\n}%22%2C%22data%22%3A%22{\n%20%20%20%20name%3A%20%27John%27%2C\n%20%20%20%20email%3A%20%27john%40mail1.com%27%2C\n%20%20%20%20email%3A%20%27john%40mail.com%27%2C\n%20%20%20%20gender%3A%20%27male%27%2C\n%20%20%20%20gender%3A%20%27male%27%2C\n%20%20%20%20phone%3A%20%27%2B22221212%27%2C\n%20%20%20%20password%3A%20%27mypassword1%27%2C\n%20%20%20%20password2%3A%20%27mypassword1%27\n}%22}

Check here
http://livr-multi-playground.webbylab.com/#%7B%22rules%22%3A%22%7B%5Cn%20%20%20%20'test'%3A%20'decimal'%5Cn%7D%22%2C%22input%22%3A%22%7B%5Cn%20%20%20%20'test'%3A%20-3.00%5Cn%7D%22%7D

This issue is for discussion of the features that should be covered by the next spec release.

Here are some of the proposals:
#26
#22
#12
#11
#7
#6
#9

Adding all to the discussion: @pterolex, @asholok, @maktwin, @SDSWanderer, @Prots, @WebbyLab, @YanaSavchenko, @antonfin, @siavolt

It would be useful to have a validation rule for modulo 10 (and quite likely other check-digit verification algorithms, though this issue is limited to modulo 10).

My specific use case would be for validation of Swedish national identification numbers, which are (for validation purposes) two part, the first specifying the century of birth and the second part specifying the date of birth, some additional digits and a mod10/luhn check digit. It might look something like this:

{
  "swedish_national_identification_number": {
    "nested_object": {
      "century": [
        "required",
        {
          "number_between": [
            19,
            20
          ]
        }
      ],
      "number": [
        "required",
        "modulo10"
      ]
    }
  }
}

Another case that comes to mind is the credit card validation in js-livr-extra-rules. It could perhaps be expressed along the lines of an alias like:

{
  "name": "credit_card",
  "rules": [
    "string",
    {
      "length_between": [
        14,
        16
      ]
    },
    "modulo10"
  ],
  "error": "WRONG_CREDIT_CARD_NUMBER"
}

This would not be strictly equivalent, but with the addition of the experimental

{ "like": "^\\d*$" }

it would be functionally equivalent to the js-livr-extra-rules's code, but using only core rules.

As noted earlier modulo10 is only one among a number of check-digit verification algorithms, so it might make sense to use a prefix to signal that it belongs to a family of rules.

Hello,

Just saw the link to this on HN - and from the syntax description, it seems that this project is only intended to validate JSON?

The documentation does not make it clear at all. I suggest updating the it to state this explicitly.

An "Empty" rule, meaning the value must be empty. Useful in combination with "or" and "Not empty" ?

Examples

"id" should be email or positive_integer:

{
    id: { or: ['email', 'positive_integer' ] }
}

Combining with other rules:

{
    id: [{ or: ['email', 'positive_integer' ] }, 'to_lc']
}

Set of rules in "or":

{
    id: { or: [['email', 'to_lc'], 'positive_integer' ] }
}

Emulate list_of_different_objects:

{
    products: {list_of: { or: [
        {nested_object: {
            product_type: ['required', {eq: 'material'}],
            material_id: ['required', 'positive_integer'],
            quantity: ['required', {'min_number': 1} ],
            warehouse_id: 'positive_integer'
        }},
        {nested_object: {
            product_type: ['required', {eq: 'service'}],
            name: ['required', {'max_length': 20} ]
        }}
    ]}}
}

is the same as

{
    order_id: ['required', 'positive_integer'],
    products: ['required', { 'list_of_different_objects': [
        product_type, {
            material: {
                product_type: 'required',
                material_id: ['required', 'positive_integer'],
                quantity: ['required', {'min_number': 1} ],
                warehouse_id: 'positive_integer'
            },
            service: {
                product_type: 'required',
                name: ['required', {'max_length': 20} ]
            }
        }
    ]}]
}

Is there a way to validate keys in an object?

To put it into a theorycraft example, something like this, borrowing to_entries and from_entries from jq's filters of the same name:

Rule:

{
  author: ['required', 'string'],
  commits: ['to_entries', {
    'nested_object': {
      'key': ['required', {'like': '^[0-9a-f]+$'}],
      'value': ['required', 'string']
    }
  }, 'from_entries']
}

Valid data:

{
  author: 'chtseac',
  commits: {
    '6d12ba9': 'Add foo to bar',
    'e32d134': 'Initial commit'
  }
}

Needs a Java implementation ideally that could easily plug into Spring

http://joxi.ru/YxAeaVgFyxxVAy
нет зон с одной буквой

http://webbylab.github.io/livr-playground/#%7B%22rules%22%3A%22%7B%5Cnorder_id%3A%20%5B%20'required'%2C%20%7B'number_between'%3A%20%5B600%2C%2018000%5D%7D%20%5D%5Cn%7D%22%2C%22data%22%3A%22%7B%5Cn%20order_id%3A%20'forever'%5Cn%7D%22%7D

For example "yyyy-mm-dd hh-MM-ss"

Hi there! Quite interesting project, but it would be great if you'll
enhance README with few keystrokes how it is differs from existing
schema definition languages (e.g. http://json-schema.org/)?

At some point I was interested in json-schema, and although it is
well thought project with variety of libs, it still have undefined
behaviours (google groups of json schema will clarify what I mean).

Hello!
Could you, please, describe or provide some example of how can I validate nested data structure (parent->children->children and so on)? Can't figure the best way without repeating the same validation rule for field.
For example, we have the next nested structure:

{
    id
    title
    children {
       id
       title
       children {....}
    }
}

Is it possible to validate structure with infinite/finite nesting without repeating validation rules for each children?
Thanks!

у Вас есть http://koorchik.blogspot.ru/2012/01/mojolicious.html , пользовался.
Логично увидеть и для LIVR

Допустим есть правило на поле to:{'required', ....} и такое же на поле reg_id:{'required', ....}, но в запросе должно быть только одно из этих полей. Если есть поле to, то не должно біть reg_id, и наоборот.

http://webbylab.github.io/livr-playground/#%7B%22rules%22%3A%22%7B%5Cn%20%20%20%20age%3A%20integer%5Cn%7D%22%2C%22data%22%3A%22%7B%5Cn%20%20%20%20age%3A%20'04'%5Cn%7D%22%7D

Expected result: throws exception 'NOT_INTEGER"

Текущая версия LIVR имеет один недостаток, первая реализация была сделана на базе JavaScript в следствии этого правила допускают неявное преобразование типов.

Предлагаю дополнить правила строгим набором, в котором были бы правила которые не приводят тип к нужному, а сверяют соответствует ли он ожидаемому или нет.

Например вместо правила integer реализовать два правила is_integer и to_integer, аналогично для других типов данных.

Можно упростить описание вложенных объектов, если при парсинге правил интерпретировать хеш как вызов правила nested_object.

Т.е вместо

var rules = {
    address: {nested_object: {
        city: 'required',
        zip: ['required', 'positive_integer']
    }}
}

Позволить писать

var rules = {
    address: {
        city: 'required',
        zip: ['required', 'positive_integer']
    }
}

Это довольно очевидное, однозначное и удобное сокращение, ведь поля на верхнем уровне (address) мы не оборачиваем в nested_object так:

var rules = {nested_object: {
    address: {nested_object: {
        city: 'required',
        zip: ['required', 'positive_integer']
    }}
}}

Use case:
value: [{min_length: 10}]

When this validation fails it just returns 'TOO_LONG' error code. However, quite often we need to show the user how many characters are actually expected.

Can you think of a good way to include this data (actual length limit in that case) into the error output?

Income:

• 2 not required fields must be equal

Desired outcome:

• Not equal error for empty second field
• Equal for both empty or both field and equal

Actual outcome:

• Allow to live second field empty and validator return fields values

Core members: @fuksito @asholok @Prots @erlangbureau @vlbaluk @maktwin @fperrad @k33nice @DanielHreben @pterolex

Here are several ideas:

1. I am going to tests to core test suite for checking the correctness of DSL - koorchik/Validator-LIVR#11 . It is not the first situation of misuse of the syntax.

2. Add a page to http://livr-spec.org/ with a list of companies which use LIVR. There are a lot of them already. Does it make sense?

3. Check that every implementation has LICENSE files. I've changed JS implementation license to MIT as the most permissive and understandable.

4. For JS implementation, we have a module with extra rules - https://github.com/koorchik/js-livr-extra-rules . It is a good example of LIVR's extensibility.

5. I've thught about parameterized aliases. Now we can define only names, but it would be great to have an ability to define parameters.
   The final idea (possibly somewhere in future) is to create some sort of "LIVR galaxy" (like ansible galaxy) with aliases described as data structures and to make these aliases compatible with any LIVR implementations. The main issue here is that we do not unified regex support (or at least regex checker, which will guarantee cross language compatibility of regexes). So, it will create great network effect cross all implementations
   But, at first, we need to make aliases parameterized)

Guys, what do you think? What does LIVR still lack?

Currently rule not_empty_list behaves as required rule and not contains if ( util.isNoValue(value) ) return; line. Is this an expected behavior?
BTW: it concerns all implementations.

Например, приходит ошибка TOO_LONG, для неё бы отобразить "Слишком длинное значение: значение должно быть не длиннее N символов", а этот самый N взять неоткуда.
К этому нужен какой-то интерфейс как получить соответствующее ошибке правило. Или возможность получить ошибки валидации в паре с правилами, тогда можно было бы прогнать какой-то цикл типа (псевдокод):

var errors = validator.getDetailedErrors();
foreach(error : errors){
   switch(error.code) {
      case "TOO_LONG": print("Слишком длинное значение: значение должно быть не длиннее "+
         error.rule.max_length +" символов")
   }
}

Можно, конечно, хардкодить тексты ошибок в местах реализации, мол, написали правило
phone: {max_length: 10},
написали и сообщение об ошибке "телефон не может быть длиннее 10 символов"
Но тут проблемы:

1. Это приводит к дублированию значений в нескольких местах. Потом правило меняется, сообщение становится неактуальным, его забывают обновить, получается ошибка говорит "поле не длиннее 10 символов", а на самом деле оно должно быть не длиннее 8 уже.
2. Т.к. не для всех полей может быть задано специфичное сообщение, хотелось бы реализовать нормальное дефолтное: показать пользователю TOO_LONG это вообще плохо, а вот показать адекватное общее значение было бы норм ("поле дожно быть не длиннее 15 символов").

Один из кейсов который приводит к не одинаковому поведению. Не уверен что это баг, но возможно стоит упомянуть об этом в спецификации:

{ flag: {one_of: 4.00000000000001} } - PHP сдался
{ flag: {one_of: 4.000000000000001} } - Perl тоже
{ flag: {one_of: 4.0000000000000001} } - JavaScript и Python

• а в erlng вообще не так, но это уже кейс для #22

Не совсем понятно почему спецификация допускает наличие в исходном пакете полей, для которых не заданы правила валидации? И в резалтсете мы получаем обрезанный набор исходных данных вместо ошибки валидации.

Are there plans or suggestions on how to handle conditional validation?

For example, "if field 4 is true, then fields 4a and 4b are required, otherwise not".

Было бы удобно иметь базовом функционале преобразование для примитивов (integer, boolean, string, ...).
В моем кейсе, прилетают данные в POST запросе в виде строки Пример формы
Приходиться дополнительно после валидации делать parseInt(value, 10)

Добрый день.

А может ли данное описание использоваться для динамической валидации?

Например у нас есть два поля:
type_of_data - тип данных (цифра, строка)
value - значение, если цифра то от 1 до 100
если строка то от 1 до 3 символов

По типу данных все понятно:
type_of_data : { one_of('digital', 'string') },

Но вот как описать правила для значений через условие?
Типа:
Если type_of_data eq 'digital' то value : {number_between(1,100)}
Если type_of_data eq 'string' то value : {length_between(1,3)}

I really appreciate the idea of semantic error codes as opposed to numeric codes or messages.

However, specifically in the case of like, exclusively returning the error code can lead to bad user experience. In the example,

https://github.com/koorchik/LIVR#like

displaying to the user "First name must contain only letters, numbers, or underscores" is a lot more helpful than saying "First name is in the wrong format."

Regexes can get complicated, and not having a way to communicate the constraints of the check back to the consuming code (whatever language is doing the validating) means that either the consuming code has to be aware of what the validation does and couple it to the LIVR rules, defeating the purpose of using LIVR, or it has to settle for degraded UX and un-informative validation messages.

Would you be open to considering optional human-friendly validation messages to be returned alongside the error codes? Consumers could check for a human-friendly message if they wanted, then fall back to the code when none is provided.