an2deg / pyraml-parser Goto Github PK

View Code? Open in Web Editor NEW

81.0 11.0 32.0 295 KB

pyraml-parser - Python parser to RAML, the RESTful API Modeling Language.

Python 100.00%

pyraml-parser's Introduction

pyraml-parser - Python parser to RAML, the RESTful API Modeling Language.

Warning!

❗ This project is not maintained anymore! Please consider to fork project.

Introduction

Implementation of a RAML parser for Python based on PyYAML. It is compliant with RAML 0.8 http://raml.org/spec.html

Installation

The pyraml package can be installed with pip or easy_install from GIT repository or from tarball.

$ pip install https://github.com/an2deg/pyraml-parser/archive/master.zip

For Python 2.6 packages ordereddict and 'lxml' should be installed:

$ pip install ordereddict lxml

Developing pyraml-paser

You may need to install nose, tox and mock packages to run pyraml-parser tests.

Typical installation process for developing purposes:

$ git clone [email protected]:an2deg/pyraml-parser.git
$ cd pyraml-parser
$ pip install mock
$ python setup.py develop

To run tests on all supported python versions:

$ pip install tox
$ tox

Or to run tests with nose:

$ pip install nose
$ python -m nose

Or to run tests using unittest:

$ python setup.py test

Using pyraml-parser

An instance of the RamlRoot object can be obtained by calling the pyraml.parser.load function:

import pyraml.parser

p = pyraml.parser.load('schema.raml')

print p

MIT LICENSE

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

pyraml-parser's People

Contributors

Stargazers

Watchers

pyraml-parser's Issues

Update to latest https://pypi.org/project/PyYAML

Hi,
To fix the security warning using old pyyaml version, can you update to latest version? Let me know if you need help with that.
Ref: https://nvd.nist.gov/vuln/detail/CVE-2017-18342

Thanks.

Query parameters in HTTP methods

It seems that method.queryParameters is not implemented (although the queryParameters do appear in the traits). If this is not intentional, I suggest the following patch:

diff --git a/pyraml/entities.py b/pyraml/entities.py
index 930c301..84614f5 100644
--- a/pyraml/entities.py
+++ b/pyraml/entities.py
@@ -79,6 +79,7 @@ class RamlMethod(Model):
     description = String()
     body = Reference(RamlBody)
     responses = Map(Int(), Reference(RamlBody))
+    queryParameters = Map(String(), Reference(RamlQueryParameter))


 class RamlResource(Model):
diff --git a/pyraml/parser.py b/pyraml/parser.py
index e4d37d5..7b0cae3 100644
--- a/pyraml/parser.py
+++ b/pyraml/parser.py
@@ -10,7 +10,7 @@ from collections import OrderedDict

 from raml_elements import ParserRamlInclude
 from fields import String, Reference
-from entities import RamlRoot, RamlResource, RamlMethod, RamlBody, RamlResourceType, RamlTrait
+from entities import RamlRoot, RamlResource, RamlMethod, RamlBody, RamlResourceType, RamlTrait, RamlQueryParameter
 from constants import RAML_SUPPORTED_FORMAT_VERSION
 import bootstrap

@@ -263,6 +263,7 @@ def parse_method(c, parent_object):
     method.responses = c.get_property_with_schema("responses", RamlMethod.responses)
     method.description = c.get_string_property("description")
     method.body = parse_body(ParseContext(c.get("body"), c.relative_path), method)
+    method.queryParameters = c.get_property_with_schema("queryParameters", RamlMethod.queryParameters)
     return method

Global mediaType ignored in parsing of body

To see this happen run the parser with the github api: http://api-portal.anypoint.mulesoft.com/github/api/github-api-v3/github-api-v3.raml

This has a global mediaType defined as application/json, so it is not explicitly defined in each body...

 /members:
    type: collection
    get:
      description: |
        List team members.
        In order to list members in a team, the authenticated user must be a member
        of the team.
      responses:
        200:
          body:
            schema: |
              {
                  "$schema": "http://json-schema.org/draft-03/schema",
                  "type": "array",
                  "list": [
                    {
                        "properties": {
                            "login": {
                                "type": "string"
                            },
                            "id": {
                                "type": "integer"
                            },
                            "avatar_url": {
                                "type": "string"
                            },
                            "gravatar_id": {
                                "type": "string"
                            },
                            "url": {
                                "type": "string"
                            }
                        },
                        "type": "object"
                    }
                  ]
              }
            example: |
              [
                {
                  "login": "octocat",
                  "id": 1,
                  "avatar_url": "https://github.com/images/error/octocat_happy.gif",
                  "gravatar_id": "somehexcode",
                  "url": "https://api.github.com/users/octocat"
                }
              ]

This currently yields the error:

Traceback (most recent call last):
  File "parser.py", line 10, in <module>
    p = pyraml.parser.load('github.raml')
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 107, in load
    return parse(c, relative_path)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 139, in parse
    resources[property_name] = parse_resource(context, property_name, root)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 194, in parse_resource
    resources[property_name] = parse_resource(new_context, property_name, resource)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 179, in parse_resource
    _method = parse_method(ParseContext(_method, c.relative_path), resource)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 263, in parse_method
    method.responses = c.get_property_with_schema("responses", RamlMethod.responses)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 70, in get_property_with_schema
    return property_schema.to_python(property_value)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/fields.py", line 379, in to_python
    for key, val in value.items()])
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/fields.py", line 462, in to_python
    value = self.ref_class.from_json(value)
  File "/Users/tsmith5/dev/pyraml-parser/pyraml/model.py", line 122, in from_json
    raise ValidationError(errors)
pyraml.model.ValidationError

This can be fixed by specifying appplication/json in each body section, but ideally the parser should know that a global media type was specified

ValidationError using !include for response example

Hi, I'm having trouble including an example file for a response field. I've tried a lot of different things. Am I doing something wrong?

For example:

/me:
  get:
    description: Get detailed profile information about the current user (including the current user’s username)
    responses:
      200:
        body:
          application/json:
            example: !include me-example.json

It's breaking with this stack trace:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
<ipython-input-31-2eb03608abae> in <module>()
----> 1 p = pyraml.parser.load('my-spec.raml')

/Library/Python/2.7/site-packages/pyraml/parser.pyc in load(uri)
    105         c, _ = _load_local_file(uri)
    106
--> 107     return parse(c, relative_path)
    108
    109

/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse(c, relative_path)
    137     for property_name in context.__iter__():
    138         if property_name.startswith("/"):
--> 139             resources[property_name] = parse_resource(context, property_name, root)
    140
    141     if resources > 0:

/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse_resource(c, property_name, parent_object)
    192     for property_name in new_context.__iter__():
    193         if property_name.startswith("/"):
--> 194             resources[property_name] = parse_resource(new_context, property_name, resource)
    195
    196     if resources > 0:

/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse_resource(c, property_name, parent_object)
    177         _method = new_context.get(_http_method)
    178         if _method:
--> 179             _method = parse_method(ParseContext(_method, c.relative_path), resource)
    180             methods[_http_method] = _method
    181         elif _http_method in new_context.data:

/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse_method(c, parent_object)
    261
    262     method = RamlMethod()
--> 263     method.responses = c.get_property_with_schema("responses", RamlMethod.responses)
    264     method.description = c.get_string_property("description")
    265     method.body = parse_body(ParseContext(c.get("body"), c.relative_path), method)

/Library/Python/2.7/site-packages/pyraml/parser.pyc in get_property_with_schema(self, property_name, property_schema)
     68     def get_property_with_schema(self, property_name, property_schema):
     69         property_value = self.get(property_name)
---> 70         return property_schema.to_python(property_value)
     71
     72     def _load_include(self, file_name):

/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
    316                 value = OrderedDict([
    317                     (self._key_type.to_python(key), self._value_type.to_python(val))
--> 318                     for key, val in value.items()])
    319
    320         self.validate(value)

/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
    399         elif isinstance(value, dict):
    400             # Value is JSON object, convert it to `ref_class`
--> 401             value = self.ref_class.from_json(value)
    402         elif value is None:
    403             # Value empty, just instantiate empty `ref_class`

/Library/Python/2.7/site-packages/pyraml/model.pyc in from_json(cls, json_object)
     97             # Validate and process a field of JSON object
     98             try:
---> 99                 value = field_type.to_python(json_object.get(model_field_name, None))
    100                 setattr(rv, model_field_name, value)
    101             except ValueError as e:

/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
    316                 value = OrderedDict([
    317                     (self._key_type.to_python(key), self._value_type.to_python(val))
--> 318                     for key, val in value.items()])
    319
    320         self.validate(value)

/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
    399         elif isinstance(value, dict):
    400             # Value is JSON object, convert it to `ref_class`
--> 401             value = self.ref_class.from_json(value)
    402         elif value is None:
    403             # Value empty, just instantiate empty `ref_class`

/Library/Python/2.7/site-packages/pyraml/model.pyc in from_json(cls, json_object)
    113                             errors[model_field_name] = unicode(e)
    114         if errors:
--> 115             raise ValidationError(errors)
    116
    117         return rv

ValidationError:

side-effect on other modules using PyYAML

Following example script fails because yaml.add_representer function which is called in initialization of pyraml module (pyraml/__init__.py) changes behavior of PyYAML module. Modified PyYAML module will parse dictionary structure in YAML to UniqOrderedDict, so re-assignment to the data parsed by PyYAML will raise exception.

import pyraml
import yaml


if __name__ == '__main__':
    with open("test.yml") as f:
        data = yaml.load(f)

    print(data)
    data['key'] = "new_value"    # <= raise ValidationError
    print(data)

Result:

UniqOrderedDict([('key', 'value')])
Traceback (most recent call last):
  File "example.py", line 10, in <module>
    data['key'] = "new_value"
  File "/usr/local/lib/python2.7/site-packages/pyraml/__init__.py", line 24, in __setitem__
    raise ValidationError("Property already used: {0}".format(key))
pyraml.ValidationError: 'Property already used: key'

The exception is raised from the line below.

pyraml-parser/pyraml/__init__.py

Line 24 in 6d9f474

raise ValidationError("Property already used: {0}".format(key))

This problem is caused by design of yaml.add_representer in PyYAML module, but it may be better to modify pyraml-parser because PyYAML module is used widely and is hard to be changed.

Defining JSON body using native RAML?

I understand that RAML doesn't define a standard for JSON payloads and, instead, falls back to JSONSchema for that. However, having to switch between RAML and JSONSchema is disorienting and can lead to a few typos down the line. I would rather just use RAML throughout my schemas if I can. I understand that I should probably post this in the RAML forums but I'm curious to know what are your thoughts on this?

Validation shall fail on double entries (e.g. double `title`)

Following RAML:

#%RAML 0.8

---
title: Schema with double title entry in YAML
title: Schema with double title entry in YAML
version: v1
baseUri: https://sample.com/api
/media:
    get: !!null

has title defined twice.

raml-parser command line tool properly detects this as an error with a message:

while validating while validating root properties
root property already used: 'title'
  in "/home/javl/sandbox/pyraml-parser/tests/samples/invalid/invalid-double-title.yaml", line 4, column 1

pyraml-parser does not detect this as an error.

PyPI release

Please release this on the Python Package Index.

submit 0.1.1 to pypi

Numeric API versions crashing parser

According to the RAML documentation and samples the api version does not have to be restricted to strings.

uriParameters support

Hi there!

I noticed that this library does not support uriParameters per the RAML spec. Is there any intention to?

Thanks!

Basic Usage Scenario

Hello @an2deg,
I really like the library that you have created here.
I just wanted to ask, based on your code if you have any specific usage scenario in mind.
Are you using this library to export services (like mulestudio does for raml) or create a client?
Thank you very much in advance!

RAML parser failed with attribute error

p=pyraml.parser.load("C:\Python27\test.raml")
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\pyraml\parser.py", line 136, in load
c, _ = _load_network_resource(uri)
File "C:\Python27\lib\site-packages\pyraml\parser.py", line 451, in _load_netw
ork_resource
with contextlib.closing(urllib2.urlopen(url, timeout=60.0)) as f:
AttributeError: 'Module_six_moves_urllib' object has no attribute 'urlopen'

TypeError: 'RamlRoot' object is not callable

from flask import Flask
app = Flask(name)
import pyraml.parser

@app.route("/")
def hello():
p= pyraml.parser.load('schema.raml')
return p

if name == "main":
app.run(host='0.0.0.0',port=8087,debug=True,threaded=True)

it throws an exception, did I miss something?

TypeError
TypeError: 'RamlRoot' object is not callable

Traceback (most recent call last)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Library/Python/2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1478, in full_dispatch_request
response = self.make_response(rv)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1577, in make_response
rv = self.response_class.force_type(rv, request.environ)
File "/Library/Python/2.7/site-packages/werkzeug/wrappers.py", line 841, in force_type
response = BaseResponse(*_run_wsgi_app(response, environ))
File "/Library/Python/2.7/site-packages/werkzeug/test.py", line 867, in run_wsgi_app
app_rv = app(environ, start_response)
TypeError: 'RamlRoot' object is not callable

Grabbing raml over network fails in both python 2 & 3

The first issue is that urlopen() does not exist in urllib, but rather urllib_request.

from six.moves import urllib_request as urllib2

Further, the relative path doesn't work when using urlparse.urljoin(). For example, if we load http://example.com/dir/test.raml, the relative url is set to http://example.com/dir. When it comes across an include such as !include schema.json, we get

urlparse.urljoin(relativeurl, schama.json) = http://example.com/schema.raml

instead of the expected /dir/schema.json.

I've made the fixes in pull request #25. I think these will do, but I'm not a python export.

Support RAML 1.0

This library only supports RAML 0.8. Are there any plans to support RAML 1.0 at any point? Or is this project abandoned?

Docs

Is there a plan for adding docs or a tutorial for using this package? I am interested in seeing if it would fit some needs for parsing raml, and testing an API with it. I don't mind perusing code, but seems fairly easy to setup something like readthedocs.

parser does not include body in put request

example of an RAML put where the body of the put is not in the parse tree:

%RAML 0.8

title: putexample
version: v1.0

/putexample:
put:
body:
application/json:
schema: |
{ "$schema": "http://json-schema.org/schema",
"value" : { "type": "boolean" }
}
responses:
200:
body:
application/json:
schema: |
{ "$schema": "http://json-schema.org/schema",

              "value" : { "type": "boolean" }
            }
          example: |
            { 
              "value": "true"
            }

output from the parser:
(it sees the body, but does not fill an body object in the resulting tree, since the expected body at tree level is "None", Hence the parsing of the body fails ??).
parse_body: <pyraml.parser.ParseContext object at 0x02C088F0> {'application/json': {'schema': '{ "$schema": "http://json
-schema.org/schema",\n "value" : { "type": "boolean" }\n} \n'}}
None
parse_body: {'body': None, 'is_': None, 'formParameters': None, 'headers': None, 'notNull': None, 'example': None, 'sche
ma': None}

Kind Regards,
Wouter van der Beek

mediaType support

Hello.

Please add mediaType support.
It is allowed to specify mediaType only once at top level, and then omit it in responses body.

Currently this valid raml will fail with error:

pyraml.model.ValidationError: {'body': "test' expected to be <class 'pyraml.entities.RamlBody'> or dict"}

My valid raml:

#%RAML 0.8
title: Test API
baseUri: http://localhost:8001
mediaType: application/json

schemas:
  - test: !include schemas/test.json

/resources:
  get:
    description: Get all resources
    responses:
      200:
        body:
            schema: test

Can't export to new RAML/YAML files

I tried to use this today to flatten out some RAML files (so I could import them into postman) and discovered that the pyraml models don't re-encode cleanly. Using a simple script like this one:

import pyraml.parser
import yaml
p = pyraml.parser.load('main.raml')
print yaml.dump(p)

... elements encode using a variety of non-generic types like:

!!python/object:pyraml.entities.RamlTrait
!!python/object:pyraml.entities.RamlResource
!!python/object:pyraml.entities.RamlMethod
!!python/unicode (for embedded JSON schemas)

And so forth.

I haven't yet tested this but it looks like fixing some of these might be as easy as extending BaseModel from yaml.YAMLObject instead of plain object.

Is this something you'd be open to having fixed (even if it means creating a pyraml.parser.dump(p) type method)? If so, I can keep poking at it.

Order not conserved

Hi,

I have tryed you parser on one of my RAML project and I have noticed a pretty annoying issue: Even if the resources are provided in the form of an OrderedDict, the order of the items is not conserved between the RAML file and the resulting OrderedDict.
It seems to be linked to the YAML parser itself that does not keep the order...

Do you think there is any simple solution to fix this issue?

Thanks :)