Expected Behavior

I would expect when loading a config from a dictionary, yaml, json, etc. that if a given value doesn't exist the default specified in the Config class would be used.

from file_config import config, var

@config
class TestConfig(object):
	name = var(str, default="MyName", required=False)

cfg = TestConfig.loads_json("""{}""")
assert cfg.name == "MyName"  # equals True

Current Behavior

Currently loading a config from a dictionary, yaml, json, etc. does not use the default if it doesn't exist. Instead it just sets the value to None...

from file_config import config, var

@config
class TestConfig(object):
	name = var(str, default="MyName", required=False)

cfg = TestConfig.loads_json("""{}""")
assert cfg.name == "MyConfig" # raises AssertionError
# currently cfg.name == None

Possible Solution

Luckly the solution looks pretty straight forward.
At https://github.com/stephen-bunn/file-config/blob/master/src/file_config/_file_config.py#L328

If we just replace None with arg_default the specified default value will be used. Note that arg_default is set to None if no value is given as the default (so prior implementations shouldn't break).

Your Environment

• File Config Version: 0.3.4
• Python Version: 3.7+
• Operating System and Version: Ubuntu 18.10

❤️ Thank you!

Expected Behavior

I would expect the built schema of this config class...

from typing import Dict, List
from file_config import config, var, build_schema

@config
class Config(object):
  hooks = var(Dict[str, List[str]])

build_schema(Config)

to return

{ '$id': 'Config.json',
  '$schema': 'http://json-schema.org/draft-07/schema#',
  'properties': { 'hooks': { '$id': '#/properties/hooks',
                             'patternProperties': { '^(.*)$': { 'items': { '$id': '#/properties/hooks/items',
                                                                           'type': 'string'},
                                                                'type': 'array'}},
                             'type': 'object'}},
  'required': ['hooks'],
  'type': 'object'}

Current Behavior

Currently the generated schema comes out looking like this...

{ '$id': 'Config.json',
  '$schema': 'http://json-schema.org/draft-07/schema#',
  'properties': { 'hooks': { '$id': '#/properties/hooks',
                             'patternProperties': { '^(.*)$': {},
                             'type': 'object'}},
  'required': ['hooks'],
  'type': 'object'}

Possible Solution

This is due to the fact that schema_builder@_build is not handling typing types any differently than bulitin types. So when it tries to build the nested typing type (typing.List[str]) it is instead (somewhat silently) failing and returning an empty schema ({})

Solution would be to have a special handler in _build which deals with nested typings. This may also require some changes in the other _build_{type}_type handlers with out modifiers are built (they currently expect config vars if the given type is not a builtin).

Steps to Reproduce (for bugs)

1. Try and run the code show in "Expected Behavior"

Your Environment

• File Config Version: 0.3.1
• Python Version: 3.7
• Operating System and Version: Ubuntu 18.04

❤️ Thank you!

Expected Behavior

It would be wonderful if I can pass a path to load_json and dump_json methods, and they take care of checking/opening the file before reading/writing to it.

Current Behavior

As of now, I have to pass a file object to these methods.

I need to manually add two new methods (let's call them save and load) to take the path string, open a file for reading/writing and then call load_json or dump_json.

This is a cleanup task I need to address to better standardize the logic for checking the given types of variables. This includes things like properly checking the __origin__ property of typing variables (vs. checking if the given typing var is in a list).

Essentially I need something similar to six's string_types but in an abstract way that can work for various modules such as typing, collections, and any other future things.

I need to get some real documentation into Sphinx for readthedocs since it provides better categorization and structure.

Also I hate that GitHub and PyPi treat rst differently.

• Setup sphinx doc structure
• Setup user documentation sections (getting started, usage, changelog, etc...)
• Setup automodule documentation

Expected Behavior

defusedxml has been raising deprecation warnings as a result of it not being needed(?). However, there is currently no migration guide to switching back and no real confirmation on if defusedxml is actually no longer required.

The relevant issue is open and I'll keep on it until something comes about to switch off of defusedxml.

Current Behavior

Possible Solution

Context

Your Environment

• File Config Version: 0.3.9
• Python Version: 3.7

❤️ Thank you!

Currently the instance loading logic is very naive regarding any kind of types outside of bulitins and nested configs. Need to also determine which types to load to given a defined attribute type that is part of the typing module.

This logic should probably exist in the _build method of file_config._file_config

Expected Behavior

Configs should also be able to dump and load to and from XML.

Current Behavior

There should be an XMLHandler added which provides the dynamic functions dump(s)_xml and load(s)_xml to a config instance.

Possible Solution

There several libraries which go from dict to xml...

• dict2xml
• dicttoxml
• xmltodict

But from what I can tell, they are poorly implemented and not reflective.

TODO:

• Add functionality for dict -> xml -> dict serialization
• Add documentation to docs/source/handlers.rst
• Add news about xml support

❤️ Thank you!

Expected Behavior

When a subsection of a configuration is missing from a serialized representation I would expect the deserialization to conclude without errors as it's the role of validation to check the fitting of a configuration instance.
For example, given the following configuration definition:

    @file_config.config
    class TestConfig:

        @file_config.config
        class InnerConfig:
            foo = file_config.var(str, default="Default", required=False)

        inner = file_config.var(InnerConfig, required=False)
        bar = file_config.var(str, default="Default", required=False)

and the following YAML file

bar: goofy

I expect the load() operation to complete without errors and to return an instance with bar == "goofy" and inner == None

Current Behavior

Raises a generic error trying to do None.get()

Possible Solution

Take care of the case when the subsection isn't present in the containing section dict

Your Environment

• File Config Version: 0.3.5
• Python Version: 3.7
• Operating System and Version: NixOS GNU/Linux
• Link to your branch (if applicable):

❤️ Thank you!

Expected Behavior

I want to also add support for dumping to and loading from .ini files.
Should add a dumps_ini and loads_ini methods.

Current Behavior

There is no support 😞

Possible Solution

Most likely I can just use configparser.
Particularly the read_dict method will be helpful.

❤️ Thank you!

Codacy's recent breakage caused a lot of errors to occur with the webhook process from this repo. Best thing to do is remove and re-add the project to Codacy.

So Codacy links and badges will most likely be broken for a while until they are replaced.

• Fix Codacy badges
• Fix Codacy testing options

Expected Behavior

@file_config.config
class ProjectConfig(object):
    name = file_config.var(str)
    dependencies = file_config.var(Dict[str, file_config.Regex(r"^v\d+$")])

config = ProjectConfig(name="My Project", dependencies={"A Dependency": "12"})
print(file_config.validate(config))

Should raise an error as 12 does not match the provided file_config.Regex (^v\d+$).

Current Behavior

Raises a warning...

/home/stephen-bunn/Git/file-config/file_config/schema_builder.py:186: UserWarning: unhandled translation for type <class 'function'>
  warnings.warn(f"unhandled translation for type {type_!r}")

But succeeds validation (when it shouldn't)

Possible Solution

The result of file_config.build_schema(ProjectConfig) is...

{'$id': 'ProjectConfig.json',
 '$schema': 'http://json-schema.org/draft-07/schema#',
 'properties': {'dependencies': {'$id': '#/properties/dependencies',
                                 'patternProperties': {'^(.*)$': {}},
                                 'type': 'object'},
                'name': {'$id': '#/properties/name', 'type': 'string'}},
 'required': ['name', 'dependencies'],
 'type': 'object'}

So it appears to be an issue with the embedding of the correct types in patternProperties for the #/properties/dependencies id.

Your Environment

• File Config Version: 0.0.4
• Python Version: Python 3.6
• Operating System and Version: Linux Ubuntu 18.10
• Link to your branch (if applicable):

❤️ Thank you!

Add testing for Windows and Linux pipelines for multiple python versions.
Perhaps using Azure Pipelines will make this simpler to implement.

We need to support automated testing for python 3.6 and 3.7 on both Windows and Linux.

I'm kind of getting tired of dealing with my default project setup. So I need to take some time and restructure the project to follow a better project structure.

Relevant changes are WIP in feature/restructure.

Expected Behavior

Both dump_{x} and dumps_{x} should allow the same kwargs to be passed...

from file_config import config, var

@config
class MyConfig(object):
    name = var(str)

c = MyConfig(name="Name")
c.dumps_json(indent=2)
# or...
with open("/some/file.json", "w") as fp:
    c.dump_json(fp, indent=2)

Current Behavior

dump_{x} handlers do not accept any kwargs while dumps_{x} handlers do.

Your Environment

• File Config Version: 0.3.2
• Python Version: 3.7
• Operating System and Version: Ubuntu 18.04

❤️ Thank you!

Expected Behavior

Defined config classes should have the ability to define the generated JSONschema's $id and $schema properties. Most likely the expected usage would look something like the following:

from file_config import config, var

@config(schema_id="Resource.json", schema_draft="http://json-schema.org/draft-07/schema#")
class MyResource(object):

    name: str = var()

This would build the following JSONSchema:

{
    "type": "object",
    "required":
    [
        "name"
    ],
    "properties":
    {
        "name":
        {
            "$id": "#/properties/name",
            "type": "string"
        }
    },
    "$id": "Resource.json",
    "$schema": "http://json-schema.org/draft-07/schema#"
}

Current Behavior

Currently, the default behavior to generate the id is just to take the name of the class and set that as the JSONschema $id. By default, the $id of the above config class would be MyResource.json without any method to modify this value.

The document $schema is hard-set to draft-07 as that draft is the minimum required to support several features such as pattern matching for the file_config.Regex fields. Unsure how to allow this to be mutable as of right now.

Possible Solution

The code that is utilized to set these properties is found at https://github.com/stephen-bunn/file-config/blob/master/src/file_config/schema_builder.py#L447-L451. Changing this to read properties from the config's metadata store should be simple.

I propose that we add the following optional kwargs to the config() decorator to allow the dynamic setting of generated JSONschema $id and $schema fields.

• schema_id - controls the $id property in JSONschema
• schema_draft - controls the $schema property in JSONschema

As we hard-depend on draft-07 for our schema building, there will probably either need to be some kind of validation or warning issued when a draft < 07 is specified for schema_draft.

Steps to Reproduce (for bugs)

N/A

Context

N/A

Your Environment

• File Config Version: 0.3.10
• Python Version: 3.7
• Operating System and Version:
• Link to your branch (if applicable):

❤️ Thank you!

Expected Behavior

Given the following:

from textwrap import dedent
from file_config import config, var, build_schema, utils, validate, to_dict

@config
class TestConfig:
    @config
    class InnerConfig:
        foo = var(str, default="Default", required=False)

    inner = var(InnerConfig, required=False)
    bar = var(bool, default="Default", required=False)


json_ = dedent(
    """\
    {
        "bar": false
    }
"""
)
c = TestConfig.loads_json(json_)
print(c)

I would expect the value of the parsed config instance would be TestConfig(inner=TestConfig.InnerConfig(foo="Default"), bar=False).

Current Behavior

Given the above example, the loaded TestConfig instance is TestConfig(inner=None, bar=False).

Possible Solution

The solution that would solve this case is by changing the handler for building config types in _file_config._build...

elif is_config_type(entry.type):
    if arg_key not in dictionary:
-        kwargs[var.name] = arg_default
+        kwargs[var.name] = (
+            _build(entry.type, {}) if arg_default is None else arg_default
+        )
    else:
        kwargs[var.name] = _build(
            entry.type, dictionary.get(arg_key, arg_default)
        )

@azazel75 this is related to #24. You mentioned you expect that the build config instance to be None for missing nested configs. My initial/desired implementation is to have the "default state" of the inner config to be defined.

The proposed diff would ensure this as long as default is not None for the inner = var(InnerConfig, required=False) then the nested config will be built with its "empty" state. This will cause issues if you ever truly expect the built inner config to be None.

The other solution is to internally default config vars to some kind of custom type which is interpreted like javascript's undefined.

Your Environment

• File Config Version: 0.3.6
• Python Version: 3.7.1
• Operating System and Version: LUbuntu 18.10
• Link to your branch (if applicable):

❤️ Thank you!

Current Behavior

Building a schema for a config which mixes both file_config.Regex types and typing.List types can sometimes cause AttributeError on type_.__name__ checks for is_regex_type.

Possible Solution

This is an edge case which can be fixed by just having a statement which enforces that all types sent to is_regex_type cannot be typing types.

Steps to Reproduce (for bugs)

1. Create a config which uses var(List[str]) and var(file_config.Regex)
2. Edge case only occurs when Regex is evaluated before List which casues lru_cache to try and use is_regex_type again

Your Environment

• File Config Version: 0.3.2
• Python Version: 3.7
• Operating System and Version: Ubuntu 18.04

❤️ Thank you!

In some use cases typing types are considered to be typing._GenericAlias subclasses instead of typing.GenericMeta. In these cases the type.__extra__ attribute is not defined as it is not considered to be part of the typing module. However the type.__origin__ attribute contains the same needed data.

(type.__args__ is the same no matter what)

This can be mitigated by trying __extra__ first (as that is the more likely scenario) and catching AttributeError in order to then try and load the base type from type.__origin__ this bug appears on https://github.com/stephen-bunn/file-config/blob/master/src/file_config/utils.py#L370.

Your Environment

• File Config Version: 0.3.3
• Python Version: 3.7
• Operating System and Version: Ubuntu 18.04

❤️ Thank you!

Expected Behavior

If I either use typehints or pass in the type keyword argument to the var() definitions; the loading / dumping logic should be the same.

Current Behavior

Usage of type-hints vs. explicit type parameter is in-consistent. See the below example. In test case 1, we utilize the documented use of passing in the type to the var(). However, if the alternative attrs method of defining types (via typehints) is used (as displayed in test case 2), the loading of the config does not jump through the nested config correctly.

from typing import List
from file_config import config, var


@config
class TestCase1(object):
    @config
    class Nested(object):
        name = var(str)

    nests = var(List[Nested])


@config
class TestCase2(object):
    @config
    class Nested(object):
        name: str = var()

    nests: List[Nested] = var()


LOADABLE_JSON = """{"nests": [{"name": "hello"}, {"name": "world"}]}"""

print(TestCase1.loads_json(LOADABLE_JSON))
print(TestCase2.loads_json(LOADABLE_JSON))

TestCase1(nests=[TestCase1.Nested(name='hello'), TestCase1.Nested(name='world')])
TestCase2(nests=[{'name': 'hello'}, {'name': 'world'}])

Possible Solution

Most likely, I'm just not completely considering the available attributes in attrs._make.Attribute unless they implicitly override the type-hint value or mutate it in some way.

Steps to Reproduce (for bugs)

See the above example...

Context

Your Environment

• File Config Version: 0.3.9
• Python Version: 3.7
• Operating System and Version: Windows 10 is where this bug was encountered
• Link to your branch (if applicable):

❤️ Thank you!

We should have continuous testing for all of our handlers and supported modules. Hopefully this will allow us to detect if changes to those libraries will break our logic.

The kind of testing we should probably focus on is:

• Handler is importable
• Handler is reflective
• Handler dumping / loading options
• Config preferred handler keyword

Codacy raises an issue with missing defusedxml in order to prevent xml bombs. So we should add this as the default for the XMLParser which currently only depends on lxml.

I believe that defusedxml just works as a etree replacement so it shouldn't be too hard to replace some of the current logic.

• Replace etree.fromstring with defusedxml alternative

We should have better test coverage for schema_builder.py and _file_config.py as those files contain 90% of our logic. Currently we do basic testing for building basic configs and their schemas. We don't deal with edge cases for unnormal config instances.

Tests should focus on:

• Testing all config to schema types when building schemas
• Testing schema building for nested configs
• Testing / validating the generated jsonschema metadata properites

Expected Behavior

When using the prefer keyword, I expect that the module I specify should be used for a given dump / load call. So for example:

from file_config import config, var

@config
class A:
  foo = var(str)

c = A(foo="test")
c.dumps_json(prefer="json") # will result in content dumped by the `json` module
c.dumps_json(prefer="rapidjson") # will result in content dumped by the `rapidjson` module

Current Behavior

Currently if you define a config and want to dump that config to json with two different json serialization handlers (using the prefer keyword). The first handler you prefer will be set forever and will not change on the second prefer.

For example:

from file_config import config, var

@config
class A:
	foo = var(str)

c = A(foo="test")
c.dumps_json(prefer="json") # will result in content dumped by the `json` module
c.dumps_json(prefer="rapidjson") # will result in the content dumped by the `json` (not the `rapidjson`) module

Possible Solution

Will need to take a look at how our handlers/_common.py logic for _prefer_package works. Since we try to optimize the handling of what modules get imported dynamically, we will most likely have to play around with that logic.

Your Environment

• File Config Version: 0.3.7
• Python Version: 3.7+
• Operating System and Version: Linux 18.04

❤️ Thank you!