Giter VIP home page Giter VIP logo

python-yamlable's Introduction

yamlable

Convert Python objects to YAML and back.

Python versions Build Status Tests Status Coverage Status Flake8 Status

Documentation PyPI Downloads Downloads per week GitHub stars codecov

This is the readme for developers. The documentation for users is available here: https://smarie.github.io/python-yamlable/

Want to contribute ?

Contributions are welcome ! Simply fork this project on github, commit your contributions, and create pull requests.

Here is a non-exhaustive list of interesting open topics: https://github.com/smarie/python-yamlable/issues

nox setup

This project uses nox to define all lifecycle tasks. In order to be able to run those tasks, you should create python 3.7 environment and install the requirements:

>>> conda create -n noxenv python="3.7"
>>> activate noxenv
(noxenv) >>> pip install -r noxfile-requirements.txt

You should then be able to list all available tasks using:

>>> nox --list
Sessions defined in <path>\noxfile.py:

* tests-2.7 -> Run the test suite, including test reports generation and coverage reports.
* tests-3.5 -> Run the test suite, including test reports generation and coverage reports.
* tests-3.6 -> Run the test suite, including test reports generation and coverage reports.
* tests-3.8 -> Run the test suite, including test reports generation and coverage reports.
* tests-3.7 -> Run the test suite, including test reports generation and coverage reports.
- docs-3.7 -> Generates the doc and serves it on a local http server. Pass '-- build' to build statically instead.
- publish-3.7 -> Deploy the docs+reports on github pages. Note: this rebuilds the docs
- release-3.7 -> Create a release on github corresponding to the latest tag

Running the tests and generating the reports

This project uses pytest so running pytest at the root folder will execute all tests on current environment. However it is a bit cumbersome to manage all requirements by hand ; it is easier to use nox to run pytest on all supported python environments with the correct package requirements:

nox

Tests and coverage reports are automatically generated under ./docs/reports for one of the sessions (tests-3.7).

If you wish to execute tests on a specific environment, use explicit session names, e.g. nox -s tests-3.6.

Editing the documentation

This project uses mkdocs to generate its documentation page. Therefore building a local copy of the doc page may be done using mkdocs build -f docs/mkdocs.yml. However once again things are easier with nox. You can easily build and serve locally a version of the documentation site using:

>>> nox -s docs
nox > Running session docs-3.7
nox > Creating conda env in .nox\docs-3-7 with python=3.7
nox > [docs] Installing requirements with pip: ['mkdocs-material', 'mkdocs', 'pymdown-extensions', 'pygments']
nox > python -m pip install mkdocs-material mkdocs pymdown-extensions pygments
nox > mkdocs serve -f ./docs/mkdocs.yml
INFO    -  Building documentation...
INFO    -  Cleaning site directory
INFO    -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
  - long_description.md
INFO    -  Documentation built in 1.07 seconds
INFO    -  Serving on http://127.0.0.1:8000
INFO    -  Start watching changes
...

While this is running, you can edit the files under ./docs/ and browse the automatically refreshed documentation at the local http://127.0.0.1:8000 page.

Once you are done, simply hit <CTRL+C> to stop the session.

Publishing the documentation (including tests and coverage reports) is done automatically by the continuous integration engine, using the nox -s publish session, this is not needed for local development.

Packaging

This project uses setuptools_scm to synchronise the version number. Therefore the following command should be used for development snapshots as well as official releases: python setup.py sdist bdist_wheel. However this is not generally needed since the continuous integration engine does it automatically for us on git tags. For reference, this is done in the nox -s release session.

Merging pull requests with edits - memo

Ax explained in github ('get commandline instructions'):

git checkout -b <git_name>-<feature_branch> master
git pull https://github.com/<git_name>/python-yamlable.git <feature_branch> --no-commit --ff-only

if the second step does not work, do a normal auto-merge (do not use rebase!):

git pull https://github.com/<git_name>/python-yamlable.git <feature_branch> --no-commit

Finally review the changes, possibly perform some modifications, and commit.

python-yamlable's People

Contributors

jfuruness avatar smarie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

python-yamlable's Issues

Add default implementation

It would be convenient that to_yaml_dict and from_yaml_dict have a default implementation, so that users could customize only one of them, or neither.

Rename the internal methods with dunder

to_yaml_dict should probably be renamed __to_yaml_dict__
and the same
from_yaml_dict > __from_yaml_dict__

This will help mark the difference between public api and internal one

Python3.10 breakage

Python3.10 is breaking some tests I have for other repos, it looks like yamlable is using collections.Mapping which does not exist anymore in Python3.10?

.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:118: in represent_mapping                                                                                                                  
    node_value = self.represent_data(item_value)                                                                                                                                                           
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:52: in represent_data                                                                                                                      
    node = self.yaml_multi_representers[data_type](self, data)                                                                                                                                             
.tox/python3.9/lib/python3.10/site-packages/yamlable/main.py:273: in encode_yamlable                                                                                                                       
    return dumper.represent_mapping(yaml_tag, new_data, flow_style=None)                                                                                                                                   
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:118: in represent_mapping                                                                                                                  
    node_value = self.represent_data(item_value)                                                                                                                                                           
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:52: in represent_data                                                                                                                      
    node = self.yaml_multi_representers[data_type](self, data)                                                                                                                                             
.tox/python3.9/lib/python3.10/site-packages/yamlable/main.py:273: in encode_yamlable                                                                                                                       
    return dumper.represent_mapping(yaml_tag, new_data, flow_style=None)                                                                                                                                   
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:118: in represent_mapping                                                                                                                  
    node_value = self.represent_data(item_value)                                                                                                                                                           
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:52: in represent_data                                                                                                                      
    node = self.yaml_multi_representers[data_type](self, data)                                                                                                                                             
.tox/python3.9/lib/python3.10/site-packages/yamlable/main.py:273: in encode_yamlable                                                                                                                       
    return dumper.represent_mapping(yaml_tag, new_data, flow_style=None)                                                                                                                                   
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:118: in represent_mapping                                                                                                                  
    node_value = self.represent_data(item_value)                                                                                                                                                           
.tox/python3.9/lib/python3.10/site-packages/yaml/representer.py:52: in represent_data                                                                                                                      
    node = self.yaml_multi_representers[data_type](self, data)                                                                                                                                             
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'lib_bgp_simulator.tests.engine_tests.utils.simulator_codec.simulator_codec.SimulatorCodec'>, dumper = <yaml.dumper.Dumper object at 0x7fb23089dd50>, obj = <Relationships.CUSTOMERS: 3>
without_custom_tag = False, kwargs = {}, yaml_tag_suffix = 'Relationships', obj_as_dict = {'name': 'CUSTOMERS', 'value': 3}

    @classmethod
    def encode(cls, dumper, obj,
               without_custom_tag=False,  # type: bool
               **kwargs):
        # type: (...) -> MappingNode
        """
        The method used to encode YamlAble object instances
     
        :param dumper:
        :param obj:
        :param without_custom_tag: if this is set to True, the yaml tag !yamlable/<yaml_tag_suffix> will not be written
            to the document. Warning: if you do so, decoding the object will not be easy!
        :param kwargs: keyword arguments coming from pyyaml, not sure what you will find here.
        :return:
        """
        # Convert objects to a dictionary of their representation
        yaml_tag_suffix, obj_as_dict = cls.to_yaml_dict(obj)
>       if not isinstance(obj_as_dict, collections.Mapping) or not isinstance(yaml_tag_suffix, str):
E       AttributeError: module 'collections' has no attribute 'Mapping'

.tox/python3.9/lib/python3.10/site-packages/yamlable/main.py:467: AttributeError

yaml_info decorator breaks mypy checks

Hi,

thanks for providing this library! This took a lot of headaches away from me. However, despite in the actual code everything works perfectly fine, there is a small issue woth my mypy validation.

Basically, this is my class:

@yaml_info(yaml_tag_ns='myns)
class MyClass(YamlAble):
    def __init__(self, **kwargs: Any) -> None:
      # ...
      self.__dict__.update(kwargs)

When I now run the mypy checks, I'm getting the following error which points to the line with class MyClass(YamlAble)::

mypackage/myfile.py:29: error: Too few arguments

When I remove the decorator like this, the error is gone:

# @yaml_info(yaml_tag_ns='myns)
class MyClass(YamlAble):
    def __init__(self, **kwargs: Any) -> None:
      # ...
      self.__dict__.update(kwargs)

Any idea how to solve the mypy error without losing the functionality of your library?

`YamlAble` should not provide a constructor implementation

Since there is already a way to declare the yaml tag (by setting __yaml_tag_suffix__ on the class or on the instance), and that there is a helper to do that with @yaml_info decorator, it is more than enough. YamlAble should not provide an implementation of __init__ in order to avoid disturbing other class behaviour especially in multiple-inheritance context.

`@yaml_info` should not be used on subclasses of `YamlObject2`

Indeed, if it manages to be used, that means that the metaclass has already been called. Therefore it means that yaml_tag has already been read on the class and used to register the codec with pyyaml.

Therefore any modification of yaml_tag made by the decorator is anyway too late and will not change the tag that is associated with the class.

Consider Sequence Support?

# coding: utf-8
from yamlable import YamlObject2
import yaml

class Base:
    def __init__(self, *args, **kwargs):
        print("here", args, kwargs)
        self.args = args
        self.kwargs = kwargs

    def __repr__(self):
        return f"{self.__dict__}"


class Ice(YamlObject2, Base):
    yaml_tag = "![*ice]"


class Fire(YamlObject2, Base):
    yaml_tag = "![*fire]"


file = """
dict:
  - &ice ~
  - &fire ~
data:
  - ![*ice] {k: 1, v: 2}
  - ![*fire] [1, 2]
"""

data = yaml.safe_load(file)

print(data)

When parsing Ice (mapping, to dict), everything works well. However YamlAble doesn't have a handler to handle sequence yet, it will cause
`
E:\Python36\python.exe C:/Users/Administrator/AppData/Roaming/JetBrains/PyCharm2020.1/scratches/test_scratch.py
here () {'k': 1, 'v': 2}

Traceback (most recent call last):
File "C:/Users/Administrator/AppData/Roaming/JetBrains/PyCharm2020.1/scratches/test_scratch.py", line 31, in
data = yaml.safe_load(file)
File "E:\Python36\lib\site-packages\yaml_init_.py", line 162, in safe_load
return load(stream, SafeLoader)
File "E:\Python36\lib\site-packages\yaml_init_.py", line 114, in load
return loader.get_single_data()
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 51, in get_single_data
return self.construct_document(node)
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 60, in construct_document
for dummy in generator:
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 408, in construct_yaml_seq
data.extend(self.construct_sequence(node))
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 130, in construct_sequence
for child in node.value]
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 130, in
for child in node.value]
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 100, in construct_object
data = constructor(self, node)
File "E:\Python36\lib\site-packages\yamlable\yaml_objects.py", line 115, in from_yaml
constructor_args = read_yaml_node_as_dict(loader, node)
File "E:\Python36\lib\site-packages\yamlable\base.py", line 210, in read_yaml_node_as_dict
loader.flatten_mapping(node)
File "E:\Python36\lib\site-packages\yaml\constructor.py", line 184, in flatten_mapping
key_node, value_node = node.value[index]
TypeError: 'ScalarNode' object is not iterable
`

Maybe a type router to call constructor in different ways is better?

Simplify abstract yaml-able class definition by removing `NONE_IGNORE_CHECKS`

As of today all classes are checked for an explicit __yaml_tag_suffix__ (if inheriting from YamlAble) or a yaml_tag (if using YamlObject2). If this field is not explicitly present, exceptions are raised (when dumping/loading for YamlAble and at class definition time for YamlObject2).

Because of the above behaviour, it was not possible to define an abstract parent class for several yaml-able types, with common yaml-ability. Therefore a NONE_IGNORE_CHECKS symbol was introduced so that abstract classes could use it in place of their yaml_tag or yaml_tag_suffix. This symbol is redundant with a simple usage of None. This should be simplified, we can probably safely remove NONE_IGNORE_CHECKS (keeping it for legacy, but it will be equal to None).

Improve packaging

  • py.typed
  • setup.cfg universal_wheel
  • setup.py remove six dependency
  • setup.py zip_safe=False
  • setup.py remove tests folder from package.

publish the package to conda-forge

I'm relying on this package in a conda-forge recipe and it breaks as it's not available there. Would you consider publishing it on conda-forge ?

I can do it for you, you will just need to agree to be set admin of the feedstock.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.