Add Contributor License Agreement to start accepting contributions

Make sure README acts as a good documentation for a newcomer reading it.

project: foo
tasks:
    off:
        cmd: echo hello

yields:

 PROJECT | TAGS | TASKS 
---------+------+-------
 foo     |      | off   

Right now, if we want to override flags, one must:

export A=... B=...
# or
set -a
source .env-file
# or
A=... B=... myke doSomething

Proposal to support a CLI flag to load an extra dotenv file:

myke -e test.env doSomething

Could be useful for templating

https://goreportcard.com/report/github.com/goeuro/myke

Proposal

Support env and env_files for tasks

Questions

• How will env variable cascading/inheritance work between project, shell, tasks, etc

Alternative

• Use export ... in the task's cmd

• Right now, commands are executed with sh -exc
• Make it configurable

Right now, there are some core tasks like myke list, myke run, etc.

myke --help
COMMANDS:
     list      list available tasks
     run       query to execute of format tag1/tag2/project/task[arg1=val1,arg2=val2,...]
     template  render a template file with environment variables
     license   prints licenses
     help, h   Shows a list of commands or help for one command

They will conflict with tasks defined in YML files (e.g. mysql/run or api/run). Decide on a namespace for built-in tasks.

• Allow wildcard globbing in files
• Things to ponder: not conflicting with shell expansion, using the same globbing in mixin/discover, etc.
• Descoped for 1.0 because its not required now

Moving issue from old repo /cc @vpillac @piotrpolatowski

Option 1

task:
  cmd: doSomething
  finally: doCleanup

Where finally is always executed upon task exit

Option 2

task:
  cmd: doSomething
  after: doCleanup
  error: doCleanup

Where after is already supported (before/after) and error would be a new keyword which runs only upon error. Doing cleanup would mean you would fill in both error and after, which is slightly more verbose (i.e. midway between Option 1 and bash), but at the same time gives an additional flexibility of doing stuff upon error (like alerting, scraping logs, etc). So while its slightly more verbose, it also kills two birds in one stone.

There are also other specifics, like what to do if Ctrl+C/SIGTERM/SIGINT is sent when a command is half-executing - in that case should myke trap those signals and call the failure/finally command, etc. But for now, lets assume that behavior is undefined when forcefully exiting myke, and then define it later...

• Current tabular output is capped at 80 chars. Make it full terminal width. Unfortunately most table formatting libraries in Golang aren't termwidth friendly, might have to make it custom
• Provide flag to switch between tabular or list mode

For vastly smaller binaries. upx had an issue with macOS Sierra but that has been resolved, upx -f -o date.upx /bin/date works. Make myke also use it.

Right now gox compressed myke binaries are failing with upx, maybe because:

• Some library is causing the problem (urfave/cli not the problem)
• Or try with normal GOOS/GOARCH builds instead of gox

Options:

before: taskA[a=b] taskB[c=d]

before: myke taskA[a=b] taskB[c=d]

before:
- taskA[a=b]
- taskB[c=d]

Should it behave like a command or a dependency tree?

• Allow myke tagA/-tagB/task to filter out tasks having tagB
• Explore possible other syntaxes (related to #43) such as myke task+tagA-tagB or myke task --tagA --tagB --a=b ...

I have a myke.yml config which references few others:

discover:
  - ../dir1
  - ../dir2
  - ../dir3

When one of them contains an error I receive the following message when running myke:

⨯ error error=yaml: line 34: could not find expected ':'

Without the actual path to the file causing the trouble, I have no easy way of figuring it out myself.

I would expect an error like:

⨯ error ../dir1/myke.yaml[34]: could not find expected ':'

Right now it exits cleanly, make it fail.

I have a dir0/myke.yml config which references few others:

discover:
  - ../dir1
  - ../dir2
  - ../dir3

However dir1 also discovers dir2:

discover:
  - ../dir2

In such scenario, I will see duplicated dir2 entries when I run myke in dir0.

I would expect that discovery skips configs which have been already discovered...

Use sprig for additional functions - https://github.com/Masterminds/sprig

Test case:

project: foo
tasks:
  test:
    cmd:
      foo=bar
      echo $foo

myke test
• foo/test: Running
+ foo=bar
+ echo

• foo/test: Completed

This shows that myke is eagerly evaluating environment variables. Fix this.

cc @markusjevringgoeuro

Key map by full location of file to prevent recursive mixin/discover calls

Add -f flag to select yml file

• Fix self-documenting examples for env/envfile/envfile.local

Since tasks are now sorted, uncomment test cases in examples/package_test.go

Integrate tests, building binaries and GitHub release (upon tags) and build status images with Travis

Integrate code coverage with CI to make contributions easy after 1.0.0

• includes: is used to "discover"
• extends is used to inherit tasks into current namespace
• Maybe better nomenclature is required to clarify this, because the README says parent environment variable cascading, etc

• Add ability in YML to re-run failed commands
• Its a really cumbersome and platform-specific problem for devs

Support environment variable expansion using templates /cc @vpillac

Few problems:

• Environment variables are used for parameter substitution, but if they themselves have parameters, then 2-pass/n-pass
• Ordering of the passes, with/without nested references
• Potential recursion

Or something that allows to tell myke to be quiet.

Example : Calling a util task to get ip from kubeconfig.

tasks:
  my_task:
    cmd: |-
      netIp=`$myke --loglevel fatal _get_ip`
  _get_ip:
    cmd: |-
      echo `cat $KUBECONFIG | grep server | cut -d: -f2 | xargs`

Could also be :

myke --quiet _get_ip

tested myke on windows 10 pro with cygwin, PowerShell and CMD - no env vars availabe.

• myke.env present
• myke.env.local present
• env section ind myke.yml present

but none of the env vars is available.

Env vars in Windows 10 Pro are only availbe within the ubuntu linux bash layer ...

myke.yml

---
project: booking-system
env_files:
  - default-values.env
env:
  ABC: default
  PATH: path_from_yml
tasks:
  echo:
    cmd: "echo 123 $ABC def"
  test:
    cmd: env
    before: $myke echo

myke.env

ABC=abc

myke.env.local

ABC=def

output

$ myke test
booking-system/test: Running
+ echo

+ env
PWD=/cygdrive/c/Users/goeuro/projects/goeuro/booking-system/myke
HOME=/cygdrive/c/Users/goeuro
TERM=cygwin
SHLVL=1
_=/usr/bin/env
SYSTEMROOT=C:\WINDOWS
WINDIR=C:\WINDOWS
booking-system/test: Completed, Took: 80.0767ms

cmd.exe support was recently contributed, thanks to @blaubaer. But we are discovering that there is more specifics involved:

• Subshells ($(command ...))
• Variable substitution ($VAR, ${VAR}, ${VAR:-default}, etc)
• Environment variable cascading to further subcommands
• Exit codes
• Piping (foo | grep), Tests (&&, ||), I/O Redirection (>, >>)
• Any other sh "standard" expectations that we miss?
• cmd.exe vs powershell

Keeping this issue open and 1.0.0 release on hold until this is decided.

Use a logging library for verbosity levels, to debug what's going on when running the commands

Running myke test on a yaml file with this content

project: test
tasks:
  test:
    cmd: |-
      echo SOMETHING=$(echo test)

provides this result with myke 0.9.0:

++ echo test
+ echo SOMETHING=test
SOMETHING=test
• mobileweb/test: Completed, Took: 11.366067ms

With myke 0.9.1 and higher the output is however an error:

sh: -c: line 0: syntax error near unexpected token `('
⨯ mobileweb/test: Failed, Took: 8.97756ms
⨯ error                     error=error running command: exit status 2

Expected behaviour: Same behaviour as with myke 0.9.0

• We added support for -v level from 0-5
• We should set the shell to sh -ec if level is 0 to make it even less verbose

Right now there are unit tests, but no blackbox CLI tests. Ideally, since we have the self-documenting examples folder, we could have a blackbox CLI test suite that just runs commands and validates the raw STDOUT output. This would cover the CLI contract as well.

TDD driven learning is a good starting point to a new language, so this could be a good starter issue for learning Golang as well.

Right now, we invoke all tasks with /bin/sh. This thread is to discuss if and how to support multiple shells (example, cmd.exe, powershell, bash, zsh, etc). Options to be considered:

• YML syntax
• Multi-shell and/or multi-platform. e.g. Windows can use both /bin/sh or cmd.exe, Linux could use /bin/sh or /bin/bash, etc. So shells and platforms are not 1:1 but rather cross functional
• myke specific feature compatibility: environment variable cascading, retry and exit codes, verbosity levels and stdout/stderr handling, etc

Refers #85

• Make sure README reflects correct environment variables (MYKE_HOME, MYKE_PROJECT, MYKE_TASK, etc)
• Make sure CWD is correctly set

Make sure YML syntax is well-defined and good enough for freezing to 1.0

Now, the task list is a table. Allow a verbose task list option, which will print tasks with full details including the description.

TODO: Decide on command and output semantics

myke -v | myke -l
foo [project=a, tags=tagA,tagB]
  does something
bar [project=b, tags=tagA,tagB]
  does something else
...

Any other better command line UX we can borrow from?

• Currently tasks are printed in a table
• Provide a CLI flag to print vertically as a definition list

Check using go-releaser

• Should we use a language-neutral template engine (handlebars or dtl)?
• Or stick with go built-in text/template?

When a mixin path cannot get resolved, myke is silently ignoring the whole myke.yml file where this mixin is referenced, (even other tasks declared in this myke.yml) without warning.

Proposal is to warn when a whole myke.yml gets ignored because of a missing mixin.

Obligatory introduction blog post

Allow strings in before/after hooks for compatibility

Finalize built-in filters for Pongo2

Right now we have only one filter (required), finalize if that's enough or document where necessary

Tracking status of Golang port

• Dev bootstrap (docker, docker-compose, golang, cli framework, CI, cross compilation)
• Load project/task/workspace/query with old behavior (env/param handling, extends/includes, etc)
• List tasks, Note: Improvements welcome, not using full term width
• Execute tasks
• Templating, 90% done, Note: Slightly breaks compatibility, using pongo2, STDOUT heredoc handling to be fixed
• Release binaries upon tags/releases (after open sourcing with travis-ci)

Support dry-run mode that prints all tasks but doesn't actually run them

Sort task names when displaying in table