Unknown format: float64 about python-fastjsonschema HOT 8 CLOSED

Format float64 does not exist in JSON schema specification. If you want it, you have to provide own validation, see formats parameter in documentation.

from python-fastjsonschema.

The JSON Schema reference manual makes it clear that format is extensible and should only be validated if recognised.

By default, format is just an annotation and does not effect validation.
...
Implementations may provide validation for only a subset of the built-in formats or do partial validation for a given format.

https://json-schema.org/understanding-json-schema/reference/string#format

A schema should not be considered invalid when a format is not recognised by the validator. In my case, I want to use uuid (which, btw, is "New in draft 2019-09") but the schema is considered invalid and won't parse. My desired behaviour would be that I set "format": "uuid" now, and in future when fastjsonschema adds support for that format, my strings would begin being validated against it. Instead, I am forbidden from specifying the format until I either add explicit support myself (making my schema not portable) or wait for built-in support.

See santhosh-tekuri/jsonschema#21 for more discussion in similar project.

from python-fastjsonschema.

@johndavidpearce Thank you for pointing out that by the specs, formats shouldn't do any assertions. I published a new version that accepts use_formats parameter that can be used to disable it. I made the decision early to make assertions that would now be incompatible if turned off, the same way as defaults should also be just annotation, which I solved the same way. Hope it helps.

from python-fastjsonschema.

Note that you can use custom_formats and pass uuid even in previous versions, or even with the new one and keeping assertions for all formats.

from python-fastjsonschema.

Thanks @horejsek . If I am reading the updated docs correctly, then use_formats completely disables any and all validation based on format? I don't think this quite fits with what I understand the spec to expect. My understanding is specifically that unrecognised formats should be allowed/ignored, but recognised formats can/should be used for validation (format violations should still be disallowed).

While disabling formats entirely is useful, I would still prefer to have the option of leaving the formats open for extension, such that any formats which are indeed recognised are still validated against, but new/unrecognised formats are ignored. Such a feature would allow schema-developers to be free to add more format specifiers to their schemas without worrying about breaking the schema validator. After all, the goal is to validate the JSON document in question against the schema, not validate the schema itself. If the schema is valid w.r.t. the official spec, then why should fastjsonschema refuse to parse and use it?

Would this flag be better?

# Allow unrecognised formats to exist in the schema without raising an error.
# Such formats will be ignored, while recognised formats will still be validated against.
any_format=True

from python-fastjsonschema.

I don't think its unrecognised formats only. If I read this paragraph, it clearly says format is just an annotation.

The format keyword allows for basic semantic identification of certain kinds of string values that are commonly used. For example, because JSON doesn't have a "DateTime" type, dates need to be encoded as strings. format allows the schema author to indicate that the string value should be interpreted as a date. By default, format is just an annotation and does not effect validation.

If you want to keep validation but allow any unknown format, you can always achieve it by using formats parameter by implementing own special dictionary-like object to have all possible formats on the world and always returning True. You could also use this object to log from your app what you are ignoring and do alerts etc. No need for another parameter.

class AllFormats:
    def __contains__(self, key):
        return True

    def __getitem__(self, key):
        return lambda data: True

fastjsonschema.validate({"format": "any"}, "abc", formats=AllFormats())

from python-fastjsonschema.

Thanks @horejsek for your suggestion. I can certainly understand the desire to avoid another parameter. I think that suggestion is workable, but I would be more comfortable using it if that class was part of fastjsonschema itself -- that way users would be confident that they hadn't missed any required methods on the AllFormats class and that it will remain compatible with future versions of fastjsonschema (after all, the full dict interface is huge).

from python-fastjsonschema.

At this moment, I believe that disabling formats altogether (to keep them as annotations only) is an option that follows the specification properly. How to do validation, if any, is up to the library. I think it is better to fail for the unknown format to avoid surprises. I don't want to provide any class like this because, as I suggested above, you might want to do some alerting to avoid surprises, which will be different for every project. The accepted object is a dictionary, and you can also inherit your object from a dictionary to ensure future compatibility (in case the library uses more methods from the dictionary). If the type changes completely, it will be a bump in a major version and properly documented. I will think about it for future releases, but I'm leaving it as is for now.

from python-fastjsonschema.