Comments (10)
@sitaktif When you say timestamp do you mean a string or integer like "1426059442" || 1426059442 or do you mean that your object is a python datetime object? Is it possible that you can provide a sample schema file and data file so i can test the problem myself?
The reason behind throwing the validation error is simple and that is because i have to implement the check for a given type and that is what any is for. A improvment upon this problem might be to implement a lower type then any that basiclly do nothing and accepts anything perhaps?
from pykwalify.
@sitaktif Your question answers itself i think because the docs state how it works and that will not change any time soon. A simple solution could be to use str
or int
depending on your data files and let it validate through that one. I will add a new type in the future that is something more in the line off all
or whatever
or something that validates no matter what it says. To add datetime and timestamp support is something in the pipe but like yourself i have not got around to do it :]
from pykwalify.
@Grokzen Thanks for your answer! And to clarify, by timestamp I meant something that looks like 2015-03-27T21:07:46Z
. I believe this is what the vanilla Kwalify library calls it (I was myself confused by the naming).
The reason behind throwing the validation error is simple and that is because i have to implement the check for a given type and that is what any is for.
I am not sure I understand why you have to check for all the types that any
checks for. Is it for security reasons?
The current meaning for any
is a bit hard to handle at the moment because each time a new type gets supported, it is going to break backward compatibility in the API. Indeed, when the timestamp
(or let's call that datetime
rather) gets added, any
's behaviour will suddenly change (it will accept values that it used to deny).
Also, accepting any type (even the non-implemented ones) with any
will make it more compatible with the original Kwalify, and will allow people to use a given Kwalify schema in Python, Ruby and Java. Adding a new keyword such as all
or whatever
will unfortunately not fix that issue.
This is maybe not particularly relevant to the development of Pykwalify, but to give you one data point, the main reason I got interested in Pykwalify was that I could use the same Yaml schema in different languages. This is why compatibility between the different kwalify libraries is important to me -- but again I understand this might not be one of your goal maintaining this particular lib.
from pykwalify.
The original documentation is lacking with the defenition of the major difference between, time, date and timestamp.
I would say that here in the python version it could be defined as
time
- Format like HH-MM-SS
it should be configurable what formats that should be supported ofc but the default list should contain all the most used formats.
date
- Format like YYYY-MM-DD
or MM-DD-YYYY
Here also it should be configurable what formats should be supported.
timestamp
- I define this as a unix timestamp that you derive from
>>> import time
>>> time.time()
1427595892.498187
It could be valid with or without decimal numbers but it could be configurable.
datetime
- This should match against the normal datetime formats but should also be configurable what formats should be supported.
I understand your argumentation with the any
rule and i could agree to change it to your defenition if it brings the schema closer to the original kwalify schema.
from pykwalify.
The original documentation is lacking with the definition of the major difference between, time, date and timestamp.
I agree the Kwalify documentation is lacking on this but putting Kwalify (and Pykwalify) aside, we should follow standards and not come with our own recipes. Doing it in a Pythonic way is not necessarily the right way to go (and believe me, I love the pythonic way :-) ) because Yaml is a concept that is language agnostic.
You can see an example of timestamps in the Yaml official standard on [http://www.yaml.org/spec/1.2/spec.html#id2761573](see the "Example 2.22. Timestamps" section). It follows the ISO8601 format (see http://en.wikipedia.org/wiki/ISO_8601), which is one of the widely used standards for timestamps (or what we would call datetime).
Btw, the way I described above is the way Kwalify behaves. You can look at the code, but I built an example to be sure:
$ cat schema.yaml
type: timestamp
$ cat just_an_int.yaml
1427650980
$ cat full_datetime.yaml
2015-03-29T18:45:00+00:00
$ ./validate.rb schema.yaml just_an_int.yaml
[/] '1427650980': not a timestamp.
$ ./validate.rb schema.yaml full_datetime.yaml
2015-03-29 11:45:00 -0700
The code for validate.rb
is 90% taken for the Kwalify documentation example:
#!/usr/bin/ruby
require 'kwalify'
require 'yaml'
schema=ARGV[0]
document=ARGV[1]
schema = YAML.load_file(schema)
validator = Kwalify::Validator.new(schema)
document = YAML.load_file(document)
errors = validator.validate(document)
if errors && !errors.empty?
for e in errors
puts "[#{e.path}] #{e.message}"
end
else
puts document
end
i could agree to change it to your defenition if it brings the schema closer to the original kwalify schema.
That's good to know. It's a backward incompatible change so it's probably better to postpone it to a sort of "major" release, but I think it's good to keep this change in mind.
Thanks!
from pykwalify.
If i change the any
i would tag it with a new major release 1.1.0 and just document a upgrade notice that the defenition has changed. There is nothing stopping me from doing this right now if i want so i probably will push it sometime during the week :]
Your exampleis very good and i can easily try to implement something similar to that.
from pykwalify.
@sitaktif Do you know some good list or site or something that could help to determine what formats to support out of the box? I am thinking about adding ability to manually specify what formats that you want for a datetime but there must be a default set also.
from pykwalify.
Timestamp implemented in c5a4782 Will be included in next tag/release.
from pykwalify.
The any
type change is now implemented and new release 1.1.0
is released and uploaded.
If you have any comments or suggestions on the timestamp
implementation please open a new issue :]
from pykwalify.
@Grokzen Nice!
This is coming a bit late (sorry...) but I do have a suggestion about what formats to support: why don't we use yaml.dump(the_date_value)
to support exactly what the actual yaml library supports? That way there is no question of whether we accept invalid formats (or whether we throw an error on a yaml-compliant format).
Using the yaml library to parse the date delegates this kind of decision to the people maintaining the yaml lib and ensures the formats are aligned.
EDIT: I'll open a new issue to discuss this.
from pykwalify.
Related Issues (20)
- Union of schemas HOT 4
- rule.py is missing a comma in defined_keywords list
- Deprecation warning due to invalid escape sequences in Python 3.8 HOT 1
- Fails to regex match a mapping with integer keys HOT 1
- AttributeError: 'int' object has no attribute 'startswith'
- Support multiple data type HOT 1
- New release? Maintainers needed or? HOT 2
- Deprecation warning "the imp module is deprecated in favour of importlib" HOT 1
- pattern rule not working with all scalar types HOT 1
- <RuleError: error code 4
- Using fulmatch for regex matching. HOT 1
- Schema Validation Against Multiple Files HOT 1
- Outputting Scan Results in SARIF Format HOT 1
- How can I check key only? HOT 5
- anchors to inherit properties are falsely reported as Duplicate HOT 2
- ReadTheDocs does not seem to be updating
- Some error message for length violations are wrong
- issue while using "unique: true" for the sequence data type HOT 3
- Tests fail with ruamel.yaml 0.18.0
- Core Validation Fails - Multi Threading & Python 3.11 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pykwalify.