Comments (13)
Hi.
Can you provide an example of the data in yaml format that you are trying to validate? I think that if pyyaml do not automatically convert the data on load then i am not sure if it is currently possible or not. If you build a datastructure in python and use the Core
object directly, your data should already be casted to a float as is.
About multiple types for mappings, it was not part (to my knowledge) of the original pykwalify standard. I have tried to not include to much new stuff into pykwalify that extends the original framework. But there might be 1 possibility that you could explore. You could possibly set required=False
for the key and that would silence the errors some and None could be accepted. This however is not perfect becuase you might disable/silence some errors that should really be raised.
from pykwalify.
Hm, yes, you are right, yaml doesn't parse the scientific notation as a float but as a string.
example:
from pykwalify.core import Core
import yaml
validated = yaml.load("{'fluxrange': 1e-06, 'radius': null}")
validator = yaml.load("{'mapping': {'fluxrange': {'type': 'float'}, 'radius': {'type': 'float'}, }, 'type': 'map'}")
print(type(validated['fluxrange']))
print(type(validated['radius']))
c = Core(source_data=validated, schema_data=validator)
c.validate(raise_exception=True)
output:
<class 'str'>
validation.invalid
<class 'NoneType'>
--- All found errors ---
["Value '1e-06' is not of type 'float'. Path: '/fluxrange'", "Value 'None' is not of type 'float'. Path: '/radius'"]
Traceback (most recent call last):
File "/Users/gijs/Work/kliko/test.py", line 11, in <module>
c.validate(raise_exception=True)
File "/Users/gijs/Work/rodrigues/.virtualenv/lib/python3.5/site-packages/pykwalify/core.py", line 157, in validate
error_msg=u'.\n - '.join(self.validation_errors)))
pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
- Value '1e-06' is not of type 'float'. Path: '/fluxrange'.
- Value 'None' is not of type 'float'. Path: '/radius'.: Path: '/'>
from pykwalify.
Sorry, i've been mixing up parsing yaml and json in my example. here an other example
from pykwalify.core import Core
import json
validated = json.loads('{"fluxrange": 1e-06, "radius": null}')
validator = json.loads("""{"mapping": {"fluxrange": {"type": "float"}, "radius": {"type": "float"}}, "type": "map"}""")
print(type(validated['fluxrange']))
print(type(validated['radius']))
c = Core(source_data=validated, schema_data=validator)
c.validate(raise_exception=True)
output:
<class 'float'>
<class 'NoneType'>
validation.invalid
--- All found errors ---
["Value 'None' is not of type 'float'. Path: '/radius'"]
Traceback (most recent call last):
File "/Users/gijs/Work/kliko/test.py", line 15, in <module>
c.validate(raise_exception=True)
File "/Users/gijs/Work/rodrigues/.virtualenv/lib/python3.5/site-packages/pykwalify/core.py", line 157, in validate
error_msg=u'.\n - '.join(self.validation_errors)))
pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
- Value 'None' is not of type 'float'. Path: '/radius'.: Path: '/'>
so the problem is indeed in the yaml parsing (shouldn't parse json with jaml parser anyway). When parsed with the json parser it works. Sorry for the confusion.
Still having support for multiple types for a value in a mapping would be very useful!
from pykwalify.
Related to your suggestion to setting the required filed to silence the None error, that doesn't seem to work:
from pykwalify.core import Core
validated = {"radius": None}
validator = {"mapping": {"radius": {"type": "float"}}, "type": "map", "required": False}
Core(source_data=validated, schema_data=validator).validate(raise_exception=True)
output:
validation.invalid
--- All found errors ---
["Value 'None' is not of type 'float'. Path: '/radius'"]
Traceback (most recent call last):
File "/Users/gijs/Work/kliko/test.py", line 4, in <module>
Core(source_data=validated, schema_data=validator).validate(raise_exception=True)
File "/Users/gijs/Work/rodrigues/.virtualenv/lib/python3.5/site-packages/pykwalify/core.py", line 157, in validate
error_msg=u'.\n - '.join(self.validation_errors)))
pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
- Value 'None' is not of type 'float'. Path: '/radius'.: Path: '/'>
from pykwalify.
Hmm that is true that it should not silence them, the type validation will still fail, required for mappings is that data can be allowed to be missing.
The main problem with adding some kind of support for multiple type validations for a key is that how the synxat should be changed to still preserv backwards compatibility with kwalify but at the same time add this new feature and also at the same time not making the syntax super horrible.
from pykwalify.
so currently syntax for a mapping is:
type: map
mapping:
key_one:
type: str
How about adding a keyword types
, which accepts a list of types:
type: map
mapping:
key_one:
types:
- str
- none
I don't see any backward compatibility issues there and it makes sense.
from pykwalify.
I like the idea, but i do think that problems could apear that makes it abit more complicated to code and work with.
Depending on what type
you specify, you need to have some fields to support validation the specefied types. For example, if you want to specify both map
, seq
and str
as valid types for a data structure (there might be several cases where this is valid) you need to have mapping
and sequence
added to the same level to continue to validation of the content inside that type. And if you take this one step further, there might be cases where values that is required to validate one time, can't be included or you want to have different values depending on the sub type. This would be even more problematic if you only want to validate scalar values and you want different values for range
or unique
.
from pykwalify.
The following commit 4372ade fixes the scientific notation part of the problem. The multiple types in mapping is still a open problem, but i have been thinking about it some and i might have some ideas on how to possibly support it, but it will take some time to test it out and possible implement it.
from pykwalify.
awesome, thanks!
from pykwalify.
let me know if you need some input or to share some thoughts.
from pykwalify.
@gijzelaerr Would you mind if i close this one as fixed because of the scientific notation fixes that is included in 1.5.1
and create a new one that investigates the support for multiple types in the same level?
from pykwalify.
sure!
2016-03-07 14:10 GMT+02:00 Grokzen [email protected]:
@gijzelaerr https://github.com/gijzelaerr Would you mind if i close
this one as fixed because of the scientific notation fixes that is included
in 1.5.1 and create a new one that investigates the support for multiple
types in the same level?—
Reply to this email directly or view it on GitHub
#39 (comment).
Gijs Molenaar
http://pythonic.nl
from pykwalify.
Ticket created. Closing this because scientific notation was fixed in 1.5.1
from pykwalify.
Related Issues (20)
- Union of schemas HOT 4
- rule.py is missing a comma in defined_keywords list
- Deprecation warning due to invalid escape sequences in Python 3.8 HOT 1
- Fails to regex match a mapping with integer keys HOT 1
- AttributeError: 'int' object has no attribute 'startswith'
- Support multiple data type HOT 1
- New release? Maintainers needed or? HOT 2
- Deprecation warning "the imp module is deprecated in favour of importlib" HOT 1
- pattern rule not working with all scalar types HOT 1
- <RuleError: error code 4
- Using fulmatch for regex matching. HOT 1
- Schema Validation Against Multiple Files HOT 1
- Outputting Scan Results in SARIF Format HOT 1
- How can I check key only? HOT 5
- anchors to inherit properties are falsely reported as Duplicate HOT 2
- ReadTheDocs does not seem to be updating
- Some error message for length violations are wrong
- issue while using "unique: true" for the sequence data type HOT 3
- Tests fail with ruamel.yaml 0.18.0
- Core Validation Fails - Multi Threading & Python 3.11 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pykwalify.