We still lack test cases for the cli.

Currently we are basing our CLI on a library which is not maintained actively enough. Thus we need to directly depend on prompt_toolkit.

Sadly this means more code and more tests but via this we can better respond to changes and prompt_toolkit is very good maintained what is good for us.

Since cobbler is adding Ubuntu 23.10 to the supported distros we should include it here as well

Currently for new users it is hard to catch what this project does. Thus we need a quickstart which explains what to do with this project.

We want to do this in ReadTheDocs and shrink the README.md.

We need four chapters:

1. Quickstart
2. Python Bindings usage explanation
3. CLI explanation
4. JSON-File specification

Off Doc Tasks: Asciinema presentation for the README and the docs.

The schema we have at the moment is only valid for the main branch of Cobbler. This is a great step forward but we need to care about the older versions of the schema that we possibly created accidentally in the past.

This issue is a precondition for issue #57

We want to achieve that schema version 1 is mapping to the schema that was used in Cobbler 2.8.5. Older versions of the schema we don't care about. After that, we want to increment the schema every time we change it. If we didn't change the schema since 2.8.5 then subsequently we could use the current schema as version 1. If not I would like to have a version of the schema for every backwards-compatible change we made in between.

Lastly in the docs, we should have a table that lists what version of the JSON schema is applicable to what Cobbler version.

So after reasoning with @agraul in this thread here, we have decided to get rid of properties and use a dataclass for Osversion.

Since we do want to do type validation on the members, we need a library who supports that. I see three options currently:

• dataclass-type-validator:
  • GH: https://github.com/levii/dataclass-type-validator
  • PyPi: https://pypi.org/project/dataclass-type-validator/
  • Py Version: 3.8+
• enforce-typing
  • GH: https://github.com/matchawine/python-enforce-typing
  • PyPi: https://pypi.org/project/enforce-typing/
  • Py Version: 3.7+
• gata
  • GH: https://github.com/kodemore/gata
  • PyPi: https://pypi.org/project/gata/
  • Py Version: 3.5+

I would vote for the gata library because it seems the most used one but I see the downside that this means that we can't run it with Python 3.6 which we have on SLE15 (no SP).

CC @nodeg @meaksh @agraul @vzhestkov we need to maintain this library in the future so the choice is also up to you. I did not look up what we have in the Buildservice packaged or not already.

When an error is thrown it is not handled in a user-friendly way during an import.

Currently we have 3 instances for distro_signatures.json:

1 in libcobblersignatures,
1 in cobbler and
1 in cobbler.github.io

We want to unify these and thus reduce the effort for maintaining an example signatures file. The idea is to have a static site in cobbler.github.io that serves the examples for all the different schema versions

As a project maintainer
I want not to invest time to make a release
so that I can focus on developing with the code

This is related to cobbler/cobbler#3123

The release workflow should be started by pushing a tag with semantic versioning.

TODO:

• Copy release_main.yml to enable releasing on PyPi
• Create automation that build artifacts generated by #47 are attached to a release
• Create changelog automation

Since we use Black on the main codebase, we should also enable it here.

Currently, several OS versions under a breed replicate the same fields (eg default_autoinstall). This change will minimize information duplication and decrease the signature file size.

As a user
I want to have it simple to install this project
so that I can use it more easily.

This project is a library that contains a stand-alone CLI. Thus I would like to have native DEB and RPMs. They should be built on PRs to main, for releases and for every commit of main itself.

Currently supported_repo_breeds accepts a list of the following breeds:

• rsync
• rhn
• yum
• apt

All other breeds would result in an error. This is not indicated by either the V2 schema or the description of the property.

For later: The property decides what repositories Cobbler should try to automatically detect inside the ISO and makes them available to the user as a mirror.

Before we can release the first version of this library we need to confirm that according to our spec the produced JSON files are valid. This can be achieved through a JSON Schema validator. For this we need a JSON Schema file which we currently don't have.

Suggestion: In theory we should be able to check the validity of our JSONs in the library directly with a JSON Schema validator.

Website for JSON-Schema: https://json-schema.org/

Currenty we have an empty table when trying to describe what purpose the keys in our JSON model have. i now found with reading the cobbler/cobbler code at least the purpose of the signatures key. Additionally we have the description of some of the keys already in the docstring of the code of our llibrary.

Purpose of the signatures is the following: It represents the portion of the path from the root of the repository which may indentify the os version. This must be always one or more paths relative from the root of the ISO. If no values in this array are present the attribute should be considered invalid.

New distros got added to the signatures in cobbler and we should migrate these changes to this repo as well

This is a Thread which will collect all usability issues, as well as code issues, I am able to find during my session at the 2021-06-18. Every Box will be discussed with @nodeg.

This issue will be closed when all points are discussed and splitout issues are created in case we decide to act upon it.

• There is no option to pretty print the output. - #34
• There is no warning when adding a breed with the same name a second time. - #35
• Current master signatures are not importable. - #37
• #74
• The properties in the library are not documented - #33
• The Code Autodoc could be pulled one level up. --> Autodoc generates the modules.rst file automatically
• The enums should be split out of the models. - #36
• Default and missing values should be skipped during im and export. #37
• Add typehints for the docs - https://github.com/agronholm/sphinx-autodoc-typehints --> This is done automatically by the next regeneration. Sphinx now supports type annotations.
• Move code from signatures.py into __init__.py since it makes the imports easier for consumers. - #36
• Autoconversion of null values to skipped values. (see above)
• Publish the package to the testing PyPi - #32
• Publish the package to PyPi after this issue was closed as Version 0.2.0
• Add Wiki Page with Information on how to regenerate the Autodoc etc. - https://github.com/cobbler/libcobblersignatures/wiki/Documentation
• Create and integrate JSON Schema in this library - #39
• Fix the stacktrace about editing array values in the library - #40

Our GitHub action to automatically publish to test.pypi is currently broken. This should be fixed by updating the versions in the GitHub actions we are using to publish our current build

As a developer
I want to be able to have less work with the json file maintenance
so that I can help my users to easier contribute to the project

At the moment we maintain distro_signatures.json coupled to the server version of Cobbler. Thus we have a version of the file for every branch in Cobbler plus a file on cobbler.github.io. This is a lot of maintenance. After #39 is done, we could integrate a step in the CI that checks the format of the JSON.

This opens up the possibility for us to change to a format based versioning, instead of the current Cobbler-version based approach.

As a developer
I want be able to not maintain duplicated code in relation to distro_signatures.json
so that I can take advantage of the advanced functionality of this library in the main server.

Some remarks for this issue:

• If there are changes required to this library to make access for consumers of the library easier, let's discuss it and do it, we haven't tagged anything yet
• The CLI of this library needs to be mature enough to work with existing distro_signatures.json files
• The serialization for the server items we need to adjust since that is at the moment storing the strings for the versions. If we switch to the DAO (data access objects) here, then we need to be careful with serialization and the XML-RPC API.

My idea for the timeline is the following:

• This project is being integrated into the main server codebase in a pre-release (let's say something alike 0.1.0)
• We test everything before a Cobbler release
• We iterate the first two points before this one until we are happy with the results
• We tag 1.0.0 for this library
• We upgrade the Cobbler dependency to 1.0.0
• We release Cobbler with the integrated library

Code usages I was able to identify:

• https://github.com/cobbler/cobbler/blob/0ea84602160b0216d0bd641789dc42885ffb352d/cobbler/api.py#L174-L189
• https://github.com/cobbler/cobbler/blob/0ea84602160b0216d0bd641789dc42885ffb352d/cobbler/api.py#L1548-L1587
• https://github.com/cobbler/cobbler/blob/0ea84602160b0216d0bd641789dc42885ffb352d/cobbler/remote.py#L2729-L2739
• https://github.com/cobbler/cobbler/blob/0ea84602160b0216d0bd641789dc42885ffb352d/cobbler/modules/managers/import_signatures.py#L311-L370
• https://github.com/cobbler/cobbler/blob/0ea84602160b0216d0bd641789dc42885ffb352d/cobbler/utils/signatures.py
• https://github.com/cobbler/cobbler/blob/0ea84602160b0216d0bd641789dc42885ffb352d/cobbler/validate.py#L233-L288

Additionally the CLI has some usages but we ignore those as we will not put effort into a deprecated CLI. The effort needs to be put into the Golang CLI. This CLI can be progressed as soon as cobbler/cobbler#3164 (with PR cobbler/cobbler#3244) is fixed (merged) and we have merged cobbler/cobblerclient#10 on the CLI.

Currently PRs are failing sometimes, even when they are good. This is due to the reason that the CI providers are not correctly configured.

To close this issue we have the following AC:

• Take a look at the Codacy Linters and decide which we will use.
• Configure choosen Codacy Linters
• Configure Pyup via a configuration file

In order to keep our workflows as similar as possible to cobbler we are switching our code-coverage tool to codacy

The related issue in cobbler: cobbler/cobbler#3365

As a user
I want to have a starting point for my signatures.json file
so that I can spend less time modifying the defaults.

The questionary dialogue defined in import_menu_questions should receive a new option that states Built-In.

When this option is chosen then the built-in v2 schema (data/v2/distro_signatures.json) should be loaded via importlib resources. The code should utilize either the backport (already defined in setup.py) or the standard library.

Under no circumstance, it should be possible to overwrite the built-in config file.

Hint: The method to react correctly for the menu can be found here.

When opening PRs at both the website (https://github.com/cobbler/cobbler.github.io) and the server repo it is at the moment not possible to verify if the JSON is handed in the correct format. It would be nice if we (and others) can verify this with a GitHub Action.

This is dependant on #39.