Giter VIP home page Giter VIP logo

configlet's Introduction

Configlet

The official tool for managing Exercism language track repositories.

Download

Each track should have a bin/fetch-configlet script, and might have a bin/fetch-configlet.ps1 script too. The first is a bash script, and the second is a PowerShell script.

Running one of these scripts downloads the latest version of configlet to the bin directory. You can then use configlet by running bin/configlet, or bin/configlet.exe on Windows.

Verifying

Exercism signs each configlet release archive with minisign.

For now, if you want to verify the signature of a configlet release, you need to do it manually. The fetch-configlet script may support checking the release signature in the future, but it won't require checking it: Exercism doesn't want to require every fetch-configlet user to install minisign.

To verify a release archive, first download (from the assets section of a release) the archive and its corresponding .minisig file. Write them to the same directory. For example, to verify the configlet 4.0.0-beta.13 Linux x86-64 release, download these files to the same directory:

configlet_4.0.0-beta.13_linux_x86-64.tar.gz
configlet_4.0.0-beta.13_linux_x86-64.tar.gz.minisig

Then run a minisign command in that directory:

minisign -Vm configlet_4.0.0-beta.13_linux_x86-64.tar.gz -P RWQGj6DTXgYLhKvWJMGtbDUrZerawUcyWnti9MGuWMx7VDW9DqZn2tMZ

where the argument to -P is the configlet public key.

The preceding command has verified the release archive if (and only if) the command's output begins with Signature and comment signature verified. For example:

Signature and comment signature verified
Trusted comment: timestamp:2023-08-09T10:27:15Z  file:configlet_4.0.0-beta.13_linux_x86-64.tar.gz  hashed

Then extract the archive to obtain the (now-verified) configlet executable. You may delete the archive and the .minisig file.

Usage

The application is a single binary, and you can use it as follows:

Usage:
  configlet [global-options] <command> [command-options]

Commands:
  completion  Output a completion script for a given shell
  create      Add a new exercise, approach or article
  fmt         Format the exercise 'config.json' files
  generate    Generate Concept Exercise 'introduction.md' files from 'introduction.md.tpl' files
  info        Print some information about the track
  lint        Check the track configuration for correctness
  sync        Check or update Practice Exercise docs, metadata, and tests from 'problem-specifications'.
              Check or populate missing 'files' values for Concept/Practice Exercises from the track 'config.json'.
  uuid        Output new (version 4) UUIDs, suitable for the value of a 'uuid' key

Options for completion:
  -s, --shell <shell>             Choose the shell type (required)
                                  Allowed values: b[ash], f[ish], z[sh]

Options for create:
      --approach <slug>           The slug of the approach
      --article <slug>            The slug of the article
      --practice-exercise <slug>  The slug of the practice exercise
      --concept-exercise <slug>   The slug of the concept exercise
  -e, --exercise <slug>           Only operate on this exercise
  -o, --offline                   Do not update the cached 'problem-specifications' data
  -a, --author                    The author of the exercise, approach or article
  -d, --difficulty                The difficulty of the exercise (default: 1)

Options for fmt:
  -e, --exercise <slug>           Only operate on this exercise
  -u, --update                    Prompt to write formatted files
  -y, --yes                       Auto-confirm the prompt from --update

Options for info:
  -o, --offline                   Do not update the cached 'problem-specifications' data

Options for sync:
  -e, --exercise <slug>           Only operate on this exercise
  -o, --offline                   Do not update the cached 'problem-specifications' data
  -u, --update                    Prompt to update the unsynced track data
  -y, --yes                       Auto-confirm prompts from --update for updating docs, filepaths, and metadata
      --docs                      Sync Practice Exercise '.docs/introduction.md' and '.docs/instructions.md' files
      --filepaths                 Populate empty 'files' values in Concept/Practice exercise '.meta/config.json' files
      --metadata                  Sync Practice Exercise '.meta/config.json' metadata values
      --tests [mode]              Sync Practice Exercise '.meta/tests.toml' files.
                                  The mode value specifies how missing tests are handled when using --update.
                                  Allowed values: c[hoose], i[nclude], e[xclude] (default: choose)

Options for uuid:
  -n, --num <int>                 Number of UUIDs to output

Global options:
  -h, --help                      Show this help message and exit
      --version                   Show this tool's version information and exit
  -t, --track-dir <dir>           Specify a track directory to use instead of the current directory
  -v, --verbosity <verbosity>     The verbosity of output.
                                  Allowed values: q[uiet], n[ormal], d[etailed] (default: normal)

configlet lint

The primary function of configlet is to do linting: checking if a track's configuration files are correctly structured - both syntactically and semantically. Misconfigured tracks may not sync correctly, may look wrong on the website, or may present a suboptimal user experience. Configlet's checks are therefore important for maintaining the integrity of Exercism.

The configlet lint command is still under development. The list of currently implemented checks is here.

configlet sync

A Practice Exercise on an Exercism track is often implemented from a specification in the exercism/problem-specifications repository.

Exercism intentionally requires that every exercise has its own copy of certain files (such as .docs/instructions.md), even when that exercise exists in problem-specifications. Therefore configlet has a sync command, which can check that such Practice Exercises on a track are in sync with that upstream source, and can update them when updates are available.

There are three kinds of data that configlet can update from problem-specifications: documentation, metadata, and tests. There is also one kind of data that configlet can populate from the track-level config.json file: filepaths in exercise config files.

Later sections describe the checking and updating of these data kinds, but as a quick summary:

  • configlet sync only operates on exercises that exist in the track-level config.json file. Therefore if you are implementing a new exercise on a track and want to add the initial files with configlet sync, add the exercise to the track-level config.json file first. If the exercise isn't yet ready to be user-facing, set its status value to wip.
  • A plain configlet sync makes no changes to the track, and checks every data kind for every exercise.
  • To operate on a subset of data kinds, use some combination of the --docs, --filepaths, --metadata, and --tests options.
  • To interactively update data on the track, use the --update option.
  • To non-interactively update docs, filepaths, and metadata on the track, use --update --yes.
  • To non-interactively include every unseen test for a given exercise, use (for example) --update --tests include --exercise prime-factors.
  • To skip downloading the problem-specifications repository, add --offline --prob-specs-dir /path/to/local/problem-specifications
  • Note that configlet sync tries to preserve the key order in exercise .meta/config.json files when updating. To write these files in a canonical form without syncing, use the configlet fmt command. However, configlet sync does add (possibly empty) required keys (authors, files, blurb) when they're missing. This is less like syncing, but more ergonomic: when implementing a new exercise, you can use sync to create a starter .meta/config.json file.
  • configlet sync removes keys that aren't in the spec. Custom key/value pairs are still supported, but you must write them inside a JSON object named custom.
  • Configlet exits with an exit code of 0 when all the seen data are up to date, and 1 otherwise.

Note that in configlet releases 4.0.0-alpha.34 and earlier, the sync command operated only on tests.

Docs

A Practice Exercise that originates from the problem-specifications repository must have a .docs/instructions.md file (and possibly a .docs/introduction.md file too) containing the exercise documentation from problem-specifications.

To check every Practice Exercise on the track for available documentation updates (exiting with a nonzero exit code if at least one update is available):

configlet sync --docs

To interactively update the docs for every Practice Exercise, add the --update option (or -u for short):

configlet sync --docs --update

To non-interactively update the docs for every Practice Exercise, add the --yes option (or -y for short):

configlet sync --docs --update --yes

To operate on a single Practice Exercise, use the --exercise option (or -e for short). For example, to non-interactively update the docs for the prime-factors exercise:

configlet sync --docs -uy -e prime-factors

Metadata

Every exercise on a track must have a .meta/config.json file. For a Practice Exercise that originates from the problem-specifications repository, this file should contain the blurb, source and source_url key/value pairs that exist in the corresponding upstream metadata.toml file.

To check every Practice Exercise for available metadata updates (exiting with a nonzero exit code if at least one update is available):

configlet sync --metadata

To interactively update the metadata for every Practice Exercise, add the --update option (or -u for short):

configlet sync --metadata --update

To non-interactively update the metadata for every Practice Exercise, add the --yes option (or -y for short):

configlet sync --metadata --update --yes

To operate on a single Practice Exercise, use the --exercise option (or -e for short). For example, to non-interactively update the metadata for the prime-factors exercise:

configlet sync --metadata -uy -e prime-factors

Tests

If a track implements an exercise for which test data exists in the problem-specifications repository, the exercise must contain a .meta/tests.toml file. The goal of the tests.toml file is to track which tests the exercise implements. Each test in this file has a UUID to identify it, and may have an include key to specify whether the test is implemented.

A tests.toml file has this format:

# This is an auto-generated file.
#
# Regenerating this file via `configlet sync` will:
# - Recreate every `description` key/value pair
# - Recreate every `reimplements` key/value pair, where they exist in problem-specifications
# - Remove any `include = true` key/value pair (an omitted `include` key implies inclusion)
# - Preserve any other key/value pair
#
# As user-added comments (using the # character) will be removed when this file
# is regenerated, comments can be added via a `comment` key.

[1e22cceb-c5e4-4562-9afe-aef07ad1eaf4]
description = "basic"

[79ae3889-a5c0-4b01-baf0-232d31180c08]
description = "lowercase words"

[ec7000a7-3931-4a17-890e-33ca2073a548]
description = "invalid input"
include = false
comment = "excluded because we don't want to add error handling to the exercise"

In this case, the track has chosen to implement two of the three available tests. If a track uses a test generator to generate an exercise's test suite, it must use the contents of the tests.toml file to determine which tests to include in the generated test suite.

To check every Practice Exercise tests.toml file for available tests updates (exiting with a nonzero exit code if there is at least one test case that appears in the exercise's canonical data, but not in the tests.toml):

configlet sync --tests

To interactively update the tests.toml file for every Practice Exercise, add the --update option:

configlet sync --tests --update

For each missing test, this prompts the user to choose whether to include/exclude/skip it, and updates the corresponding tests.toml file as appropriate. Configlet writes an exercise's tests.toml file when the user has finished making choices for that exercise. This means that you can exit configlet at a prompt (for example, by pressing Ctrl-C in the terminal) and only lose the syncing decisions for at most one exercise.

To non-interactively include every unseen test case, use --tests include. For example, to do so for an exercise named prime-factors:

configlet sync --tests include -u -e prime-factors

Remember to actually implement these tests on the track.

Filepaths

Finally, the sync command also handles "syncing" from a source that isn't problem-specifications - the track-level config.json file. Every Concept Exercise and Practice Exercise must have a .meta/config.json file with a files object that specifies the (relative) locations of the files that the exercise uses. Such filepaths usually follow a simple pattern, and so configlet can populate the exercise-level values from patterns in the files key of the track-level config.json file.

To check that every Concept Exercise and Practice Exercise on the track has a fully populated files key (or at least one that configlet can populate from the track-level files key):

configlet sync --filepaths

(Note that configlet lint also produces an error when an exercise has a missing or empty files key.)

To populate empty or missing values of the exercise-level files key for every Concept Exercise and Practice Exercise from the patterns in the track-level files key:

configlet sync --filepaths --update

To do this non-interactively and for a single exercise named prime-factors:

configlet sync --filepaths -uy -e prime-factors

Using sync when adding a new exercise to a track

The sync command is useful when adding a new exercise to a track. If you are adding a Practice Exercise named foo that exists in problem-specifications, one possible workflow is:

  1. Manually add an entry to the track-level config.json file for the exercise foo. This makes the exercise visible to configlet sync.
  2. Run configlet sync --docs --filepaths --metadata -uy -e foo to create the exercise's documentation, and a starter .meta/config.json file with populated files, blurb, and perhaps source and source_url values.
  3. Edit the exercise .meta/config.json file as needed. For example, add yourself to the authors array.
  4. Run configlet sync --tests include -u -e foo to create a .meta/tests.toml file with every test included.
  5. View that .meta/tests.toml file, and add include = false to any test case that the exercise won't implement.
  6. Implement the tests for the exercise to match those included in .meta/tests.toml.
  7. Add the other required files.

configlet fmt

An Exercism track repository has many JSON files, including:

  • The track config.json file.

  • For each concept, a .meta/config.json and links.json file.

  • For each Concept Exercise or Practice Exercise, a .meta/config.json file.

These files are more readable if they have a consistent formatting Exercism-wide, and so configlet has a fmt command for rewriting a track's JSON files in a canonical form.

The fmt command currently only operates on the exercise .meta/config.json files, but it's likely to operate on all the track JSON files in the future.

A plain configlet fmt makes no changes to the track, and checks the formatting of the .meta/config.json file for every Concept Exercise and Practice Exercise.

To print a list of paths for which there isn't already a formatted exercise .meta/config.json file (exiting with a nonzero exit code if at least one exercise lacks a formatted config file):

configlet fmt

To make configlet prompt to write formatted config files, add the --update option (or -u for short):

configlet fmt --update

To non-interactively write the formatted config files, add the --yes option (or -y for short):

configlet fmt --update --yes

To operate on a single exercise, use the --exercise option (or -e for short). For example, to non-interactively write the formatted config file for the prime-factors exercise:

configlet fmt -uy -e prime-factors

When writing JSON files, configlet fmt will:

  • Write the key/value pairs in the canonical order.

  • Use two spaces for indentation.

  • Use a separate line for each item in a JSON array, and each key in a JSON object.

  • Remove key/value pairs for keys that are optional and have empty values. For example, it removes "source": "".

  • Remove "test_runner": true from Practice Exercise config files. This is an optional key - the spec says that an omitted test_runner key implies the value true.

  • When a JSON object has more than one key/value pair with some key name, keep only the final one.

The canonical key order for an exercise .meta/config.json file is:

- authors
- [contributors]
- files
  - solution
  - test
  - exemplar           (Concept Exercises only)
  - example            (Practice Exercises only)
  - [editor]
  - [invalidator]
- [language_versions]
- [forked_from]        (Concept Exercises only)
- [test_runner]        (Practice Exercises only)
- [representer]
  - version
- [icon]
- blurb
- [source]
- [source_url]
- [custom]

where the square brackets indicate that the enclosed key is optional.

Note that configlet fmt only operates on exercises that exist in the track-level config.json file. Therefore if you are implementing a new exercise on a track and want to format its .meta/config.json file, add the exercise to the track-level config.json file first. If the exercise isn't yet ready to be user-facing, set its status value to wip.

The exit code is 0 when every seen exercise has a formatted .meta/config.json file when configlet exits, and 1 otherwise.

configlet uuid

Each exercise and concept has a UUID, which must only appear once across all of Exercism. It must be a valid version 4 UUID (compliant with RFC 4122) in the canonical textual representation, which means that it must match this regular expression:

^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

You can run configlet uuid to output a new, appropriate UUID. There is also the -n, --num option for outputting multiple new UUIDs:

$ configlet uuid --num 5
3823f890-be49-4700-baac-e19de8fda76f
c12309a2-8bd6-4b9c-a511-e1ee4083f492
26167ad5-fe20-43d4-8b1f-3bbb9618c36e
5df11ac0-e612-4223-b0f8-f6cd2cb15cb1
e42b94bb-9c90-47f2-aebb-03cdbc27bf3b

configlet generate

Each concept exercise and concept have an introduction.md file. If you want the exercise's introduction to include the concept's introduction verbatim, you can create a introduction.md.tpl file to achieve this. This file may use a placeholder to refer to the concept's introduction, so that the information isn't duplicated.

Concept placeholders must use the following format:

%{concept:<slug>}

For example, if the track has a concept named floating-point-numbers then an introduction.md.tpl file can contain:

%{concept:floating-point-numbers}

You can run configlet generate to generate the exercise's introduction.md for any exercise that has an introduction.md.tpl file. The generated introduction.md is identical to the introduction.md.tpl, except that configlet replaces concept placeholders with the contents of the concept's introduction.md file (minus its top-level heading). In the future, configlet generate will also increment the level of other headings by 1 (for example from ## My Heading to ### My Heading), but this isn't yet implemented.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/exercism/configlet.

configlet's People

Contributors

bobtfish avatar dependabot[bot] avatar ee7 avatar erikschierboom avatar exercism-bot avatar glennj avatar ihid avatar isaacg avatar jmrunkle avatar joshiraez avatar kytrinyx avatar sanderploegsma avatar saschamann avatar valentin-p avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

configlet's Issues

sync: show an error if `--exercise` specifies an exercise which is not found

$  bin/canonical_data_syncer --check --exercise sum-of-multiple
Fri 16 Oct 2020 11:42:09 AM CEST
Checking exercises...
Cloning into '/mnt/c/Users/coriolinus/Documents/Projects/exercism/xrust/.problem-specifications'...
remote: Enumerating objects: 540, done.
remote: Counting objects: 100% (540/540), done.
remote: Compressing objects: 100% (512/512), done.
remote: Total 540 (delta 27), reused 149 (delta 17), pack-reused 0
Receiving objects: 100% (540/540), 247.13 KiB | 1000.00 KiB/s, done.
Resolving deltas: 100% (27/27), done.
All exercises are up-to-date!

That looks good, right? There's a single-character typo, though:

$  bin/canonical_data_syncer --check --exercise sum-of-multiples
Fri 16 Oct 2020 11:42:00 AM CEST
Checking exercises...
Cloning into '/mnt/c/Users/coriolinus/Documents/Projects/exercism/xrust/.problem-specifications'...
remote: Enumerating objects: 540, done.
remote: Counting objects: 100% (540/540), done.
remote: Compressing objects: 100% (512/512), done.
remote: Total 540 (delta 27), reused 149 (delta 17), pack-reused 0
Receiving objects: 100% (540/540), 247.13 KiB | 1.00 MiB/s, done.
Resolving deltas: 100% (27/27), done.
[warn] sum-of-multiples: missing 1 test cases
[warn] some exercises are missing test cases

It would be better if an improper exercise chose raised an error.

fetch script: unclear error message with invalid `GITHUB_TOKEN`

The script was not downloading the binary and exiting with non-zero exit code. Some quick debuging showed that $URL was an empty string. Then I checked what curl --header "$HEADER" -s "$LATEST" (as on line 38) output and saw:

{
  "message": "Bad credentials",
  "documentation_url": "https://docs.github.com/rest"
}

Removing --header "$HEADER" fixes the immediate issue and now I have a binary.

However before putting in a PR for this I'd like to know if this is an actual fix or just a workaround and tha the real problem may be elsewhere.

sync: silence git when cloning

$  bin/canonical_data_syncer --check --exercise sum-of-multiples
Fri 16 Oct 2020 11:42:00 AM CEST
Checking exercises...
Cloning into '/mnt/c/Users/coriolinus/Documents/Projects/exercism/xrust/.problem-specifications'...
remote: Enumerating objects: 540, done.
remote: Counting objects: 100% (540/540), done.
remote: Compressing objects: 100% (512/512), done.
remote: Total 540 (delta 27), reused 149 (delta 17), pack-reused 0
Receiving objects: 100% (540/540), 247.13 KiB | 1.00 MiB/s, done.
Resolving deltas: 100% (27/27), done.
[warn] sum-of-multiples: missing 1 test cases
[warn] some exercises are missing test cases

The git-emitted text is just chaff, particularly as the temporary .problem-specifications directory gets cleaned up later. It would be nicer if that was all silent:

$  bin/canonical_data_syncer --check --exercise sum-of-multiples
Fri 16 Oct 2020 11:42:00 AM CEST
Checking exercises...
[warn] sum-of-multiples: missing 1 test cases
[warn] some exercises are missing test cases

Bonus: if the git subcommand fails, do print its error output.

sync: Consider adding an option that runs `git pull` in a given `--prob-specs-dir`

The current implementation of --prob-specs-dir just shows an error message if the user-specified problem-specifications directory is not up-to-date with upstream, even if the upstream changes can be safely pulled into the repo's master branch.

It might be useful to provide a "trust me, I know what I'm doing" switch that tells the program to just pull in the upstream changes when it's safe (the specified repo is actually a valid problem-specifications repo, with a clean working directory, and is not in a detached HEAD state), and print that it did so (at the default verbosity level).

Add option for JSON output

we should perhaps look at a JSON output option.

I agree - I don't think people should consider the --check output as stable enough to parse, at least right now.

I want to create an automated workflow that does the following nightly

It'd be great if we can support this kind of workflow.

Originally posted by @ee7 in #61 (comment)

CLI: The help message `--option value` syntax is broken

Example

Take a track with an out-of-sync bob exercise and write:

canonical_data_syncer --exercise bob --default include

Current behaviour

The program makes no changes, and the output is completely wrong:

All exercises are synced!

Expected behaviour

The program should include the missing tests, and show:

[info] bob: included 25 missing test cases
All exercises are synced!

(Or perhaps just "1 exercise was synced!`, but that's a different discussion).

Discussion

The --help message of

Usage: canonical_data_syncer [options]

Options:
  -e, --exercise <slug>        Only sync this exercise
  -c, --check                  Check if there missing tests. Doesn't update the tests. Terminates with a non-zero exit code if any one or more test cases are missing.
  -d, --default <mode>         What to do with missing test cases. Allowed values: c[hoose], i[nclude], e[xclude]
  -o, --verbosity <verbosity>  The verbosity of output. Allowed values: q[uiet], n[ormal], d[etailed]
  -h, --help                   Show CLI usage
  -v, --version                Display version information

is currently wrong, because it claims that the syntax is --option value like:

  • --exercise bob
  • -e bob

But the only supported syntax is actually:

  • --exercise=bob
  • --exercise:bob
  • -e=bob
  • -e:bob

The problem is that --exercise bob is parsed as:

  • --exercise is an option with an empty value
  • bob is an unrecognised argument that is silently ignored

Rather than:

  • --exercise is an option with the value of bob

Solution

Fixed by #26

CLI: Use `--dash-case` rather than `camelCase`

In #76 @SleeplessByte wrote:

I'm trying to use it, and constantly keep making mistakes because what it should be isn't matching my intuition. This is completely anecdotal. There might be many many clis that use the format used here.

  1. Prefer --dash-case over --camelCase,

Almost every single cli I've worked with uses dash-case, some older ones use snake_case.

I'll do this after fixing #79

CLI: Add stricter parsing of repeated `--option value`

Example

Pass an option --foo to configlet more than once. For example:

configlet sync --exercise=bob --exercise=two-fer

Current behavior

Configlet uses the final option (here, --exercise=two-fer) and --exercise=bob is silently ignored.

Possible behavior

We could show something like:

Error: `--exercise` option can only be given once.

Alternatively, we could support passing multiple exercises via the syntax in the above example, or via something like:

configlet sync --exercise bob two-fer

or

configlet sync --exercise bob,two-fer

sync: If a test case was added that reimplements another case, print the reimplemented UUID as well

Related to #44

For this example, assume that the grains case 10f45584-2fc3-4875-8ec6-666065d1163b reimplements another case b4bacc5a-3c12-43b6-ab17-d5ec78d70a15

As of configlet 4.0.0-beta.5:

$ configlet sync --tests --verbosity=detailed
[...]
[warn] grains: missing 2 test cases
       - grains on square 3 (10f45584-2fc3-4875-8ec6-666065d1163b)

Proposed

Put reimplemented case description and UUID on following line (for human readability)

$ configlet sync --tests --verbosity=detailed
[...]
[warn] grains: missing 2 test cases
       - grains on square 3 (10f45584-2fc3-4875-8ec6-666065d1163b)
         - reimplements: square 3 grains (b4bacc5a-3c12-43b6-ab17-d5ec78d70a15)

Alternative notation

Put reimplemented case UUID on same line (simpler to parse for scripts)

$ configlet sync --tests --verbosity=detailed
[...]
[warn] grains: missing 2 test cases
       - grains on square 3 (10f45584-2fc3-4875-8ec6-666065d1163b -> b4bacc5a-3c12-43b6-ab17-d5ec78d70a15)

Improve error message when not being run from a track's root directory

We should check whether the tool is being run from the right directory, and if not, output a nice error message. The things we can check for are:

  • config.json file in the current working directory
  • exercises directory exists if there are exercises specified in the config.json file
  • exercises/<slug> directory exists for each exercise in the config.json file

CLI: syntax given in --help is not recognized

$ bin/canonical_data_syncer -m c -e sum-of-multiple
Fri 16 Oct 2020 11:35:43 AM CEST
Error: '-m' was given without a value

Usage: canonical_data_syncer [options]

Options:
  -e, --exercise <slug>        Only sync this exercise
  -c, --check                  Check if there are missing tests. Doesn't update the tests. Terminates with a non-zero exit code if one or more tests are missing
  -m, --mode <mode>            What to do with missing test cases. Allowed values: c[hoose], i[nclude], e[xclude]
  -o, --verbosity <verbosity>  The verbosity of output. Allowed values: q[uiet], n[ormal], d[etailed]
  -h, --help                   Show this help message and exit
  -v, --version                Show this tool's version information and exit

The CLI option parser appears to require an = sign between the flag and its value:

$  bin/canonical_data_syncer -m=c -e=sum-of-multiple
Fri 16 Oct 2020 11:40:21 AM CEST
Syncing exercises...
Cloning into '/mnt/c/Users/coriolinus/Documents/Projects/exercism/xrust/.problem-specifications'...
remote: Enumerating objects: 540, done.
remote: Counting objects: 100% (540/540), done.
remote: Compressing objects: 100% (512/512), done.
remote: Total 540 (delta 27), reused 149 (delta 17), pack-reused 0
Receiving objects: 100% (540/540), 247.13 KiB | 605.00 KiB/s, done.
Resolving deltas: 100% (27/27), done.
All exercises are synced!

CLI: Always show `--help` or `--version` if encountered

Example

canonical_data_syncer --help --check

Current behaviour

The program ignores the --help and performs --check.

Expected behaviour

  • If the parser encounters -h or --help we should do nothing but show the help.
  • If the parser encounters -v or --version we should do nothing but show the verison.

Try it in your favourite command-line tools.

Allow specifying the track repository directory

We should allow the user to specify the directory of the track to work with. It should default to the current directory if not specified. I'm thinking it should just be the last, non-named parameter.

The configlet tool for Exercism v2 has a similar feature BTW.

Feature: Consider adding an offline mode

It might be useful to specify an existing problem-specifications directory as-is so that we can, for example:

  1. Run tests on the binary in this repo without involving the network.
  2. Still use the tool when we know we have no internet connection, or a very bad one.
  3. Avoid waiting for the git fetch (problem-specifications doesn't update so often) and/or
  4. Deliberately use an older revision so that recent changes are handled incrementally.

sync: Fix misleading message for unrecognised `--exercise` value

Example

Give --exercise a value that doesn't exist, such as via a typo:

configlet sync --exercise bobb

Or spell it correctly on a track that doesn't implement it:

configlet sync --exercise bob

Current behaviour

The program outputs something like:

All exercises are synced!

Expected behaviour

We should probably show something like:

Error: 'bobb' is not a recognised exercise on the foo track

sync: add flag providing path to non-transient `problem-specifications` repo

It's cool that, left to its own devices, the canonical data syncer shallow-clones the problem-specifications repo into a temporary folder, then cleans up after itself. However, this is somewhat wasteful of bandwidth, even though it's normally fast.

If someone has a permanent problem-specifications repo, then it would be nice if they could provide its location to the syncer, giving the syncer permission to do essentially:

cd "$problem_specifications"
git checkout -- . # clear out any temporary modifications
git checkout master # or main, whatever,
git pull

sync: Seems to hang with slow/non-existant internet connection

#49 is great, but it completely removes the git related info messages. Whenever the internet connection is slow or flaky, this looks as if the terminal is hanging.

I recommend adding a single line "checking out into temporary location" or something like that, just so you know it's not broken.

sync: Update `tests.toml` if the corresponding `canonical-data.json` alters a test's `description`

Example

  1. Say that a track has implemented an exercise bob, which has an up-to-date tests.toml file.
  2. Change the description of a test in the canonical-data.json of that exercise in exercism/problem-specifications (assuming that we decide that the description is mutable).
  3. Run configlet sync -u --tests

Current behaviour

The tests.toml file is not changed.

Expected behaviour

The tests.toml file should contain the new description

Context

See exercism/problem-specifications#1687 (comment).

@ee7 wrote:

Should it update the tests.toml file to overwrite the comment that contains the description?"
I think the answer is yes, but it currently doesn't.

@iHiD wrote:

If we went down the mutable route, then yes it should.

CLI: Add an error for option with invalid value

Examples

canonical_data_syncer --verbosity=quiett
canonical_data_syncer --default=inklude
canonical_data_syncer -d=x

Current behaviour

The program silently ignores the invalid value and operates with the default value.

Expected behaviour

We should show an error like

Error: invalid value for '--verbosity': 'quiett'
Error: invalid value for '--default': 'inklude'
Error: invalid value for '-d': 'x'

And show the help message.

sync: Does not produce a `tests.toml` file for `grains`

Example

On a track that contains the grains exercise, but lacks a exercises/grains/.meta/tests.toml file, run:

canonical_data_syncer

Current behaviour

The program does not create a tests.toml file for grains, but we see

All exercises are synced!

Expected behaviour

The program should create a tests.toml file for grains.

The expected contents are:

[canonical-tests]

# 1
"9fbde8de-36b2-49de-baf2-cd42d6f28405" = true

# 2
"ee1f30c2-01d8-4298-b25d-c677331b5e6d" = true

# 3
"10f45584-2fc3-4875-8ec6-666065d1163b" = true

# 4
"a7cbe01b-36f4-4601-b053-c5f6ae055170" = true

# 16
"c50acc89-8535-44e4-918f-b848ad2817d4" = true

# 32
"acd81b46-c2ad-4951-b848-80d15ed5a04f" = true

# 64
"c73b470a-5efb-4d53-9ac6-c5f6487f227b" = true

# square 0 raises an exception
"1d47d832-3e85-4974-9466-5bd35af484e3" = true

# negative square raises an exception
"61974483-eeb2-465e-be54-ca5dde366453" = true

# square greater than 64 raises an exception
"a95e4374-f32c-45a7-a10d-ffec475c012f" = true

# returns the total number of grains on the board
"6eb07385-3659-4b45-a6be-9dc474222750" = true

Discussion

The cause is here:
https://github.com/exercism/canonical-data-syncer/blob/6e162f70b6944e87ffc2ae60314c38f06459541d/src/probspecs.nim#L80-L84

I remember running into this problem myself. It's because grains contains two test cases with large numbers (here and here).

The first number is one larger than we can store in a 64-bit signed integer, so it causes an error (and the second number is the maximum that can be stored in a 64-bit unsigned integer).

CLI: Parse options insensitive to case, '-' and '_'

In #76 (comment) @ee7 wrote:

I was intending to eventually make it style-insensitive anyway. So any of these should be valid:

--probSpecsDir
--probspecsdir
--prob-specs-dir
--prob_specs_dir
--PROBSPECSDIR
--prO_b----sPe____CsDIR

This will allow --prob-specs-dir even if it isn't shown in the --help like that yet.

I'll do this soon.

sync: Fix misleading message when the value of `--exercise` lacks canonical data

Example

Give --exercise an exercise that doesn't have canonical data:

configlet sync --exercise robot-name --tests

Current behaviour

The program outputs:

Checking exercises...
The `robot-name` exercise has up-to-date tests!

Expected behaviour

We could show something like:

Warning: exercise 'robot-name' does not have canonical test data.

lint: Add reminder to add contributors to `.meta/config.json`

From @iHiD on Slack:

You'll also notice that each exercise has a config.json file: https://github.com/exercism/v3/blob/master/languages/csharp/exercises/concept/strings/.meta/config.json
This file will do various things, but for now it lists the contributors to an exercise.
This services two purposes:

  1. Contributors will get kudos next to that exercise on the Exercism website, along with Credits and Reputation
  2. Contributors will get assigned/notified for future PRs to the exercise, along with the codeowners.
    So remember to fill this out if someone makes a significant contribution to an exercise.

It would be great to have a workflow that adds a reminder if a contributor adds to an exercise but is not listed in the file yet. It should never fail, only post a comment, e.g. with actions/github-script

CLI: Add an error for invalid option/argument

Examples

canonical_data_syncer --checkk
canonical_data_syncer -gelp
canonical_data_syncer check
canonical_data_syncer help

Current behaviour

The program:

  • silently ignores the invalid options/arguments
  • does the default "sync" behaviour
  • does not show the help message

Expected behaviour

We should show errors like

Error: invalid option: '--checkk'
Error: invalid option: '-gelp'
Error: invalid argument: 'check'

and show the help message.

As a special case, we can omit the error for

canonical_data_syncer help

CLI: Add an error for option with missing value

Examples

canonical_data_syncer --verbosity
canonical_data_syncer --default
canonical_data_syncer -e

Current behaviour

The program silently ignores the missing value and performs the "sync" behaviour with the default options.

Expected behaviour

We should show an error like

Error: '--verbosity' was given without a value
Error: '--default' was given without a value
Error: '-e' was given without a value

And show the help message.

sync: Consider using a table in the output

Similar to #51.

See #44, where @ee7 wrote:

I think 3 verbosity levels is a good number - maybe we don't want to cram e.g. --verbosity=full or --verbosity=debug into the --help message (or try to explain verbosity levels "1" to "4")?

Other things I thought of before are:

  1. Sort the output by category (missing tests/no canonical data/up-to-date) then alphabetically, rather than just alphabetically. Maybe we could even do this only with --verbosity=detailed. Or add an option to control the sorting.
  2. Add colors to help us distinguish between [warn] and [skip], or between missing/non-canonical/up-to-date.
  3. Use a table for the output.

I think I'd prefer to try out options 1 and 2 before adding another verbosity level. What do you think?

and @ErikSchierboom wrote:

My preference would then be to use colors to help distinguish the output. I'm not in favor of adding sorting options and I'm unsure about the categories. A table could also be nice.

CLI: Consider writing out long-form options

In #76 @SleeplessByte wrote:

  1. Prefer --problem-specifications-dir or --problem-specifications-directory over --prob-spec-dir

The main problem here is that it would occupy a lot of space in the --help message:

Usage: canonical_data_syncer [options]

Options:
  -e, --exercise <slug>                         Only sync this exercise
  -c, --check                                   Terminates with a non-zero exit code if one or more tests are missing. Doesn't update the tests
  -m, --mode <mode>                             What to do with missing test cases. Allowed values: c[hoose], i[nclude], e[xclude]
  -v, --verbosity <verbosity>                   The verbosity of output. Allowed values: q[uiet], n[ormal], d[etailed]
  -p, --problem-specifications-directory <dir>  Use this `problem-specifications` directory, rather than cloning temporarily
  -o, --offline                                 Do not check that the directory specified by `-p, --problem-specifications-directory` is up-to-date
  -h, --help                                    Show this help message and exit
  --version                                     Show this tool's version information and exit

Consider writing out long-form options and using dash-case

I'm trying to use it, and constantly keep making mistakes because what it should be isn't matching my intuition. This is completely anecdotal. There might be many many clis that use the format used here.

  1. Prefer --dash-case over --camelCase,
  2. Prefer --problem-specifications-dir or --problem-specifications-directory over --prob-spec-dir

Almost every single cli I've worked with uses dash-case, some older ones use snake_case.

image

sync: Refactor some less-reviewed, early code

I intend to get around to refactoring some things that were out-of-scope in my first review. In particular, things in these files:

  • cli.nim
  • tracks.nim
  • probspecs.nim
  • exercises.nim

Maybe in this order. I'll do it after adding some tests so that we know it doesn't break everything.

build: Support more platforms

In particular, we should be supporting:

  • Linux 32-bit
  • Windows 32-bit
  • Linux arm64
  • macOS arm64

Edit by @ee7:

@ee7 is aiming towards something like:

  • linux_x86-64
  • macos_x86-64
  • windows_x86-64

then arm64 (AKA aarch64):

then:

  • linux_powerpc64 (which is big endian) (will be added by #799)
  • linux_riscv64 (added in a962b18)

then:

Edit 2023-08-22: These 32-bit builds are possible, but I think are probably not worthwhile:

  • linux_arm
  • linux_i386
  • linux_mipsel
  • linux_powerpc
  • windows_arm
  • windows_i386

see also the Zig 0.11.0 Support Table.

fetch script: wrong OS for mac

In the fetch script, for "Darwin" platform, the OS variable is set to "mac". However, the platform part of the released tarball is "macosx".

Either the fetch script or the build process needs to be updated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.