yangcatalog / yang-validator-extractor Goto Github PK
View Code? Open in Web Editor NEWA web application to extract and validate YANG modules. Built on the bottle web-framework for Python.
License: BSD 2-Clause "Simplified" License
A web application to extract and validate YANG modules. Built on the bottle web-framework for Python.
License: BSD 2-Clause "Simplified" License
After a certain amount of API calls to the API endpoint /api/rfc/< rfc > - which is used in health check functionality, YANG validator starts to respond with following error:
RecursionError: maximum recursion depth exceeded while calling a Python object
As in title
Hi again,
Aside from the previous issue, we are trying to investigate a validation error, and we are not fully understanding what is happening.
When validating draft-ietf-ccamp-flexigrid-tunnel-yang, we have the following yanglint error:
libyang err : Invalid default - value does not fit the type (Invalid identityref "flexi-ch-spc-6p25ghz" value - identity not found in module "ietf-flexi-grid-tunnel".). (Schema location /ietf-te:te/tunnels/tunnel/secondary-reverse-paths/secondary-reverse-path/path-out-segment/label-restrictions/label-restriction/label-step/technology/ietf-flexi-grid-tunnel:flexi-grid/flexi-grid-channel-spacing.)
But in the IETF tracker, also validated with yanglint (see the validation output), there is no error, and we are not really sure why validation outputs are different (and which we should trust :D).
Thank you in advance for your help!
as in title
Draft names are a well-known set of the file names that can be indexed into the ES index.
This can be used in YANG Validator UI for autocomplete functionality while validating draft by its name.
This task consists of following:
as in title
An open API must be created / offered / documented / monitored to submit YANG modules, RFC/IETF drafts to the same tools as via the web interface to YANG validator. The returned information should also have the same content (presented as JSON).
-éric
Now that the yangvalidator is under yangcatalog, add a "yangvalidator" entry under https://github.com/YangCatalog
It should point to https://github.com/YangCatalog/bottle-yang-extractor-validator/issues (this comes from https://yangvalidator.com/yangvalidator/about)
Hi,
how the yangvalidator is supposed to handle imports? I guess that yangvalidator have some internal storage where validators search for imported modules, but what if user submit also one of the imported modules? Should the submitted schema be prefered to the schema in the storage? What if the version in storage is newer and import is without revision? And what if the revision of the submitted schema is not specified in the filename?
Specific example:
Display the current pyang, yanglint, ... versions on the input/home yangvalidator page.
https://www.yangcatalog.org/yangvalidator/
I see pyang, confdc. yanglint. We need yumaworks as well.
On top of that, we need the same output as the compilation-result in the metadata.
That means: we need pyang --ietf
By the way I have bigger issues with YangValidator. When I load ietf-system-capabilities@2020-03-08 into it I get
RecursionError at /yangvalidator/validator
maximum recursion depth exceeded
Request Method: | POST |
---|---|
Request URL: | http://www.yangvalidator.com/yangvalidator/validator |
Django Version: | 3.0.4 |
Exception Type: | RecursionError |
Exception Value: | maximum recursion depth exceeded |
Exception Location: | /home/yang/yangvalidator/lib/python3.6/site-packages/pyang-2.1.1-py3.6.egg/pyang/statements.py in newf, line 42 |
Python Executable: | /usr/sbin/uwsgi |
Python Version: | 3.6.10 |
Python Path: | ['/home/yang/yangvalidator/lib/python3.6/site-packages/pyang-2.1.1-py3.6.egg/pyang/transforms', |
Hi,
Happy New Year!
As the title says, we had some issues with the online validator. When validating draft-ietf-ccamp-flexigrid-tunnel-yang, the online validator frontend just hangs and I can see in the browser DevTools that a request to the backend returned code 500.
Doing a further investigation, I just noticed that, when submitting the YANG code without the revision-date (see https://github.com/ietf-ccamp-wg/draft-ietf-ccamp-flexigrid-tunnel-yang/blob/master/ietf-flexi-grid-tunnel.yang#L9), the code gets validated. So my guess is that something is broken in the processing of the revision dates.
If we go to this page there is a message: "No modules were extracted using xym from /var/yang/ietf/my-id-archive-mirror/draft-claise-ipfix-information-model-rfc5102bis-01.txt"
It is better if we remove absolute path from this error message, so it will be only: "No modules were extracted using xym from draft-claise-ipfix-information-model-rfc5102bis-01.txt"
NOTE: Maybe we can display this on the UI side as info instead of error? Similar to this info
Following error is displayed after running the command docker logs yc-yangvalidator
:
rsyslogd: imklog: cannot open kernel log (/proc/kmsg): Operation not permitted.
rsyslogd: activation of module imklog failed [v8.32.0 try http://www.rsyslog.com/e/2145 ]
Remove sync.sh script from Docker image build. This script pull rfc/draft documents using rsync however it is used in SDO analysis and running daily.
This will speed up Docker image build.
Fetch, extract , and validate doesn't produce any output when Upload Internet Draft option used.
Appreciate your suggestions and help.
Regards,
Greg
Is it possible to upload a set of zipped yang modules using the rest API and get them validated ?
What is the first part of the URL
https://www.yangvalidator.com/yangvalidator/rest
or
https://www.yangvalidator.com/yangvalidator
Thanks Balazs
YANGvalidator doesn't produce any output when I upload my working version of the YANG model in IETF draft. Appreciate your help.
Regards,
Greg
I uploaded a zipped set of yang files that depend on each other (import each other). However imports between the uploaded files were not resolved. I got a lot of error messages like:
err : Importing "_3gpp-common-measurements" module into "_3gpp-common-managed-function" failed.
Is it possible to solve this? How?
upgrade and use pyang api instead of command line arguments
It seems that no YANG modules can be extracted from draft-ietf-lpwan-schc-yang-data-model-02.
Previously fine models fail with the following:
Traceback (most recent call last):
pyang", line 444, in
run()
pyang", line 219, in run
repos = pyang.FileRepository(path, no_path_recurse=o.no_path_recurse)
init.py", line 408, in init
location = pip.locations.distutils_scheme('pyang')
AttributeError: 'module' object has no attribute 'locations'
Much appreciate your help.
Regards,
Greg
Some steps in Dockerfile can be moved, so build will use caching more.
This can result in faster image builds.
#81 is marked as fixed but the home page of YANG validator has a link about the REST API leading to YC.O but not to the swagger documentation :-(
Tested on Chrome/Macos
when I click on the file listed under "Extracted 1 YANG modules" I just get taken to the same page I'm already on...
Yangvalidator is no longer based on the bottle framework, and the naming should be changed to reflect this.
ietf-ospf.yang
Pyang Validation
ietf-ospf.yang:992: error: type "bandwidth-ieee-float32" not found in module ietf-routing-types
ietf-ospf.yang:996: error: type "bandwidth-ieee-float32" not found in module ietf-routing-types
ietf-ospf.yang:1009: error: type "bandwidth-ieee-float32" not found in module ietf-routing-types
ietf-ospf.yang:1966: error: identity "crypto-algorithm" not found in module ietf-key-chain
These are clearly in the latest versions of the modules.
The online yangvalidator throws errors on augments. Taken separately the augment statement validates, but when both augment statements are included in the same file, the validator chokes on them.
I have created four test yang files. Two of them validate in the online validator and two do not.
The test-ok.yang and test-ok-2.yang files work.
The test-ok file has only one augment (for /nw:networks/nw:network/nw:network-types/tet:te-topology)
The test-ok-2.yang file has only one augment (for /nw:networks/nw:network/nw:node/nt:termination-point/tet:te)
Taken separately they augments work.
The test-1.yang file has both augments Taken together, the online yanglint spits out:
libyang err : Augment target node "/nw:networks/nw:network/nw:network-types/tet:te-topology" from module "test-1" was not found. (/test-1:{augment='/nw:networks/nw:network/nw:network-types/tet:te-topology'})
libyang err : Augment target node "/nw:networks/nw:network/nw:node/nt:termination-point/tet:te" from module "test-1" was not found. (/test-1:{augment='/nw:networks/nw:network/nw:node/nt:termination-point/tet:te'})
YANGLINT[E]: Processing schema module from /var/yang/tmp/yangvalidator/yangvalidator-v2-workdir-BYiAZxUm/test-1.yang failed.
When I swap the order of the augments, I get different errors – which is strange. (test-2.yang)
libyang err : Augment target node "/nw:networks/nw:network/nw:node/te/te-node-attributes/connectivity-matrices/connectivity-matrix" from module "ietf-te-topology" was not found. (/ietf-te-topology:{augment='/nw:networks/nw:network/nw:node/te/te-node-attributes/connectivity-matrices/connectivity-matrix'})
libyang err : Augment target node "/nw:networks/nw:network/nw:node/te/tunnel-termination-point/local-link-connectivities" from module "ietf-te-topology" was not found. (/ietf-te-topology:{augment='/nw:networks/nw:network/nw:node/te/tunnel-termination-point/local-link-connectivities'})
libyang err : Augment target node "/nw:networks/nw:network/nw:node" from module "ietf-te-topology" was not found. (/ietf-te-topology:{augment='/nw:networks/nw:network/nw:node'})
libyang err : Augment target node "/nw:networks/nw:network/nw:node/te/information-source-entry/connectivity-matrices/connectivity-matrix" from module "ietf-te-topology" was not found. (/ietf-te-topology:{augment='/nw:networks/nw:network/nw:node/te/information-source-entry/connectivity-matrices/connectivity-matrix'})
YANGLINT[E]: Processing schema module from /var/yang/tmp/yangvalidator/yangvalidator-v2-workdir-fXryozgz/test-2.yang failed.
I have also attached the the full module that I'm trying to validate ([email protected])
BTW... I have yanglint v1 and yanglint v2 running locally in linux and all the yang files I have shared validate (and I have instance data that works too)
augtest.zip
.
thanks
also use socket files rather then port
Transfer yangvalidator.[org/com] to yangcatalog.
RFC 6021 contains two YANG modules but only one is processed
as in title
Problem:
yangvalidator app is currently running as root, which results in permission error in removeUnused.py script while trying to delete old yangvalidator cache files.
Traceback (most recent call last):
File "/backend/utility/removeUnused.py", line 219, in <module>
raise e
File "/backend/utility/removeUnused.py", line 126, in <module>
shutil.rmtree(os.path.join(yang_validator_cache, dir))
File "/usr/local/lib/python3.9/shutil.py", line 718, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/local/lib/python3.9/shutil.py", line 675, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/local/lib/python3.9/shutil.py", line 673, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: 'nokia-conf.yang'
These are helper files which are copied into cache directories during validation.
Possible solution:
If application will be running under yang user/group, these files will be also created/copied under this user.
The web tool reports descriptors appearing before a type as an error.
For example in a leaf
leaf foo {
description "example";
type uint32;
}
one gets
'keyword "description" not in canonical order,expected "type", (See RFC 6020, Section 12)'
The leaf value, among other statements, clearly notes ";; these stmts can appear in any order" in RFC 6020, Section 12. The statement can mislead one to believe that the YANG is in error when in fact RFC 6020, Section 12 permits description appearing before type.
I tried to validate the following:
leaf iref {
type identityref {
base x;
base y ;
}
I got back:
keyword 'base' already given for the second base line.
YANG 1.1 supports multiple base statements for identityrefs, so why do I get the error? Is it just not yet supported or is there some other problem here?
Hi,
the YANG validator used to produce the tree diagram as one of its outputs. Now, as I've found, it does not. Can you please kindly bring back "-f tree"?
Regards,
Greg
GET request on URL: https://yangcatalog.org/yangvalidator/api/versions should return json with versions of yangvalidator used, but currently return Server Error (500).
Yangvalidator times out if validating takes too long. Also there are several failures in yangvalidator logs
This is what the xym extraction output of RFC document number 7223 looks like:
ERROR: '/var/yang/ietf/rfc/rfc7223.txt', Line 1584 - YANG module 'ex-ethernet' with no <CODE BEGINS> and not starting with 'example-'
ERROR: '/var/yang/ietf/rfc/rfc7223.txt', Line 1666 - YANG module 'ex-ethernet-bonding' with no <CODE BEGINS> and not starting with 'example-'
ERROR: '/var/yang/ietf/rfc/rfc7223.txt', Line 1713 - YANG module 'ex-vlan' with no <CODE BEGINS> and not starting with 'example-'
Among other things, it contains full path to cached file rfc7223.txt.
However, this information is not important for user, it only reveals where the files are stored on the server and also it also make response unnecessarily larger.
These paths are also found in the outputs of the individual validators and should be removed.
Hi, I'm not able to make the validator running locally. Probably due to missing some of the dependencies, but I'm out of ideas what next must be installed.
Running the main.py
I'm getting the following error:
$ python --version
Python 2.7.6
$ sudo python ./bottle-yang-extractor-validator/main.py
Bottle v0.12.13 server starting up (using CherryPyServer())...
Listening on http://0.0.0.0:8080/
Hit Ctrl-C to quit.
Traceback (most recent call last):
File "./main.py", line 253, in <module>
run(server='cherrypy', host='0.0.0.0', port=port)
File "/usr/local/lib/python2.7/dist-packages/bottle-0.12.13-py2.7.egg/bottle.py", line 3127, in run
server.run(app)
File "/usr/local/lib/python2.7/dist-packages/bottle-0.12.13-py2.7.egg/bottle.py", line 2787, in run
from cherrypy import wsgiserver
ImportError: cannot import name wsgiserver
I have tried it on Ubuntu 14.04 and 16.04 and on openSUSE with the same results. In all cases the cherrypy-wsgiserver
(which seems to be connected with the error according to its name) is installed in version 0.1dev.
as in title
Noticed that today validation fails on YANG module that worked before:
[email protected]:9: error: module "ietf-inet-types" not found in search path
[email protected]:13: error: module "ietf-yang-types" not found in search path
[email protected]:17: error: module "ietf-key-chain" not found in search path
I've checked with BFD YANG and got similar failures:
Confdc Output
[email protected]:22: error: module 'ietf-routing' not found
[email protected]:23: error: module 'ietf-routing' not found
[email protected]:28: error: module 'ietf-key-chain' not found
[email protected]:13: error: module 'ietf-routing' not found
yanglint Validation
yanglint error: Unable to use search path (/var/tmp/yangmodules/extracted) - No such file or directory.
As a part of refactoring, please remove usage of .format()
method for formatting strings.
Same is for joining strings using +
character.
Instead, use f-strings for formatting strings.
The Pyang Validation section return the error:
Traceback (most recent call last):
pyang", line 444, in
run()
pyang", line 219, in run
repos = pyang.FileRepository(path, no_path_recurse=o.no_path_recurse)
init.py", line 408, in init
location = pip.locations.distutils_scheme('pyang')
AttributeError: 'module' object has no attribute 'locations'
To fix the same problem on my own version of pyang, I have commented out the following section in init.py.
'''
import pkgutil
if not pkgutil.find_loader('pip'):
return # abort search if pip is not installed
import pip
location = pip.locations.distutils_scheme('pyang')
self.dirs.append(os.path.join(location['data'],
'share','yang','modules'))
'''
use documents that we have in yangcatalog instead of using ietf.org/id which might take quiet a long time in some cases. If we don t have that document then use this url
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.