It would be nice if one could open the plan repository from shuttle it self. Something like below.

It's kind of related to #7 in that it will provide access to the documentation.

# open repository webview, eg. GitHub, for current plan
shuttle plan --web

To aid troubleshooting issues it would be great to be able to get shuttle version, current plan and version etc. output.

Something like go env in the go tool chain or npm version for Node.js.

$ go env
GOARCH="amd64"
GOBIN="/Users/bjornsorensen/go/bin"
GOCACHE="/Users/bjornsorensen/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/bjornsorensen/go"
...

{ npm: '6.4.1',
  ares: '1.14.0',
  cldr: '33.1',
  http_parser: '2.8.0',
  icu: '62.1',
  modules: '64',
  napi: '3',
  nghttp2: '1.33.0',
  node: '10.11.0',
  openssl: '1.1.0i',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.23.0',
  v8: '6.8.275.32-node.28',
  zlib: '1.2.11' }

When looking into an issue we can derive much information of such reports. I'm thinking something like

$ shuttle config
version 0.5.2

Environment:
LW_PATH=/user/...

Plan:
git://[email protected]:LunarWay/lw-shuttle-go-plan.git c331cbb3b71970dc6a3afabcd9320fb070bb719d

Maybe even let plans specify specific information related to the plan, ie. go env for Go plans etc.

If one runs a script with arguments that are not specified for the script it is silently ignored.
This makes it error prone for non-required fields.

scripts:
  test:
    args:
      - name: version
    actions:
      - shell: echo "Version $version"

$ shuttle run test versoin=123
Version
$ echo $?
0

I would expect the above command to at least exit with a non-zero status.
Preferable with a message like unknown argument "versoin"

The idea is to use --arg=value instead of arg=value.

This is to rely on cobra instead of our own validation logic. This will improve usability and be a more homogenous design

We need to support using a specific version of a plan.

It could be a specific git commit, git tag or similar.

Any scripts that output a percentage sign is wrongly outputtet as it is recognised as a placeholder.

With the following shuttle.yaml

plan: false
scripts:
  percentage:
    actions:
    - shell: echo '%'

Running shuttle run percentage outputs

$ shuttle run percentage
shell: echo '%'
%!(NOVERB)

I expected it to output the following instead.

$ shuttle run percentage
shell: echo '%'
%

We should setup travis to run the tests and do automated releases using goreleaser https://goreleaser.com/ci/

Auto completion for dynamic scripts and arguments would enhance the experience using shuttle

when trying to fire
curl -LO https://github.com/lunarway/shuttle/releases/download/$(curl -Lso /dev/null -w %{url_effective} https://github.com/lunarway/shuttle/releases/latest | grep -o '[^/]*$')/shuttle-darwin-amd64

throws the following error:
parse error near )'`

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update external go packages (dagger.io/dagger, github.com/go-cmd/cmd, github.com/spf13/cobra, go, golang.org/x/exp, golang.org/x/mod, golang.org/x/sync)
• fix(deps): update module gopkg.in/yaml.v2 to v3

Detected dependencies

This effectively means that you cannot use shuttle v0.20.0 outside of a shuttle project, as well as shuttle get being polluted with

Cloning ... <variable>

during the first run of shuttle on a project.

When shuttle scripts call other shuttle scripts, it gets hard to understand what went wrong with the current error logging.

See this example from the root of this repo.

$ shuttle --project examples/moon-base run say-hi && shuttle --project examples/moon-base run crash
shell: echo "test"
test
shell: exit 1
shuttle failed
Failed executing shell script `exit 1`
Exit code: 1

When you get an error from git, using a git-plan, all information about the error is hidden. You only get the status code from git. Would be nice with the full git error printed out

Hello! I'm trying out shuttle, but experiencing some weird behavior when using a remote git plan:

My shuttle.yaml looks like this:

plan: [email protected]:<my-org>/<my-repo>

When I run shuttle plan I get the following output:

Error: exit code 2 - Failed to open plan configuration: open /home/zander/<my-org>/<my-repo>/[email protected]:<my-org>/<my-repo>/plan.yaml: no such file or directory

Make sure you are in a project using shuttle and that a 'shuttle.yaml' file is available.
Usage:
... (help-output) ...

shuttle failed
Failed to open plan configuration: open /home/zander/<my-org>/<my-repo>[email protected]:<my-org>/<my-repo>/plan.yaml: no such file or directory

Make sure you are in a project using shuttle and that a 'shuttle.yaml' file is available.

Which I would expect to work as per the readme.

If I change the plan to the git:// syntax it works as expected:

# shuttle.yaml
plan: git://[email protected]:<my-org>/<my-repo>

# terminal
❯ shuttle plan
git://[email protected]:<my-org>/<my-repo>% 

Am I doing something wrong? Or is the documentation not correct?

PS. Shuttle looks cool, we are thinking about adopting it for our build needs, thanks for doing open-source! 😄

When generating templates using the following template functions, the order of the values are not deterministic.

• TmplObjectArray
• TmplGetFiles
• TmplArray

This becomes clear when the results of tempalted files are committed where the different order will indicates changes of (properly) non-functional changes.

It will prompt for values with default values

It would be great if there was a way to list all available templates for a given plan.

This could help users identify unexpected overwrites of plan templates and also help with discovering what templates are available shuttle template.

Maybe something like below. The most important thing is the name of the template and if it's overwritten by another template.

$ shuttle template --list
Plan templates:
 - deployment.tmpl
 - Dockerfile.tmpl (overwriting plan)
 - secret.tmpl

If run with the --verbose (-v) flag we could show the paths to each template. This could help in understanding where to look when updating or trying to understand them.

$ shuttle template --list -v
Plan templates:
 - deployment.tmpl  $plan/templates/deployment.tmpl
 - Dockerfile.tmpl  $project/templates/Dockerfile.tmpl (overwriting $plan/templates/Dockerfile.tmpl)
 - secret.tmpl      $project/templates/secret.tmpl

I tried to run shuttle get service (version v0.23.0) but got an error that the command was not known to shuttle. When i ran shuttle --help it wasn't showing up. I looked at the ./cmd/get_test.go file and there it test for this command. I checked the pipeline and the test succeeds.

I then created a samle shuttle.yaml file and ran the command again, and then it worked. When i run shuttle --help shuttle suddenly displays the get command.

Simple test :

Here's the diff on the --help output depending on shuttle finds a shuttle.yaml file or not. Based on the output below, get is not the only feature that goes missing depending whether there's a shuttle file or not.

Is this by design, and error, is shuttle bending the fabric of spacetime or is this some kind of metamorphic feature?

If would be great if arguments for scripts were validated before issuing the command.

It would require some syntax for writing requirements of an argument in a plan. This could very well be the same syntax required for #6 for validating the plan schema.

This could add more descriptive help messages as well (#14) as we define what values are allowed up front.

Example

A naive implementation would be something like below with key validation on the argument.

plan: git://[email protected]:LunarWay/lw-shuttle-go-plan.git
scripts:
  my_script:
    description: a description
    args:
    - name: foo
      required: true
      validation:
        - someValue
        - someOtherValue
    actions:
    - shell: './run.sh'

$ shuttle run my_script foo=someWrongValue
Argument foo contains an invalid value "someWrongValue". Allowed values: "someValue", "someOtherValue"

Running shuttle version crashes with error:

panic: unable to redefine 'c' shorthand in "version" flagset: it's already used for "commit" flag

goroutine 1 [running]:
github.com/spf13/pflag.(*FlagSet).AddFlag(0xc4200a4500, 0xc4200b0fa0)
	/home/travis/gopath/src/github.com/spf13/pflag/flag.go:862 +0x72b
github.com/spf13/pflag.(*FlagSet).AddFlagSet.func1(0xc4200b0fa0)
	/home/travis/gopath/src/github.com/spf13/pflag/flag.go:875 +0x71
github.com/spf13/pflag.(*FlagSet).VisitAll(0xc4200a4800, 0xc4200f1c10)
	/home/travis/gopath/src/github.com/spf13/pflag/flag.go:278 +0xa0
github.com/spf13/pflag.(*FlagSet).AddFlagSet(0xc4200a4500, 0xc4200a4800)
	/home/travis/gopath/src/github.com/spf13/pflag/flag.go:873 +0x50
github.com/spf13/cobra.(*Command).mergePersistentFlags(0x148c6e0)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:1495 +0x90
github.com/spf13/cobra.(*Command).InitDefaultHelpFlag(0x148c6e0)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:906 +0x2f
github.com/spf13/cobra.(*Command).execute(0x148c6e0, 0x14b0878, 0x0, 0x0, 0x148c6e0, 0x14b0878)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:686 +0x59
github.com/spf13/cobra.(*Command).ExecuteC(0x148c940, 0x13077ec, 0x26, 0x148c6e0)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:852 +0x30a
github.com/spf13/cobra.(*Command).Execute(0x148c940, 0x1003a8c, 0xc420018118)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:800 +0x2b
github.com/lunarway/shuttle/cmd.Execute()
	/home/travis/gopath/src/github.com/lunarway/shuttle/cmd/root.go:44 +0x31
main.main()
	/home/travis/gopath/src/github.com/lunarway/shuttle/main.go:8 +0x20```

Shuttle golang actions fail because it cannot be utilized inside a go workspace.

A plan could be enhanced with some good visible documentation. One way of doing this is simply making the CLI able to print the README.md of the plan which would make the readme automatically work with both git repos and CLI.

Consider using https://github.com/ec1oud/blackfriday

Hello!

I'm really enjoying using shuttle, so much in fact that I've been trying to add some interactivity to some of my scripts.
Though I have found that it is currently not possible to have interactive sessions through shuttle.

For example if I call a script that uses gum choose foo bar the command simply hangs forever.

Another example: Running an interactive docker container docker run --rm -it alpine returns the error the input device is not a TTY.

I think this could be a valuable addition to shuttle that would open up for using the tool in new and interesting ways.

Thank you for all of your work and for open sourcing shuttle :-)

Currently if shuttle is called outside of a project with shuttle.yaml, it will complain that it can't find a shuttle.yaml file. Which is reasonable in 99% of cases, the 1% is where we want to upload shuttle telemetry data (if wanted)

It is hard and error-prone to use the vars system. What vars can be used? Did I misspell some of them?
Add some level of schema for the vars, so required vars and optional vars are easily validated

we are currently missing license holder for script source copy right claim.

When using shuttle to start a long-running process, eg. web-server, signals from the terminal are not forwarded to the process. This leaves sub-processes running invisible and detached from the client's terminal.

Reproduce

Define a shuttle.yaml that starts a docker image

plan: false
scripts:
  test:
    actions:
    - shell: docker run --rm -i nginx

Run the shuttle script and stop after nginx has started with ctrl + c.

$ shuttle run test
shell: docker run --rm -i nginx
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
^C%

Now verify that the docker image is in fact still running

$ docker ps
CONTAINER ID   IMAGE              COMMAND                  CREATED              STATUS              PORTS                    NAMES
eddda9d31c2a   nginx              "/docker-entrypoint.…"   4 seconds ago        Up 3 seconds        80/tcp                   ecstatic_moser

Expected behaviour is that the docker image was terminated.

It's ignored silently when putting in unsupported properties in the plan.yaml and shuttle.yaml. Would be great if it could be handled with a validation "warning" (maybe not crashing)

When running any shuttle command in a directory without a shuttle.yaml file it panics.

$ mkdir t
$ cd t
$ shuttle ls
panic: yamlFile.Get err   #open /Users/bjornsorensen/repo/t/shuttle.yaml: no such file or directory

goroutine 1 [running]:
github.com/lunarway/shuttle/pkg/config.(*ShuttleConfig).getConf(0xc0000ce270, 0xc000012640, 0x48, 0x0)
	/home/travis/gopath/src/github.com/lunarway/shuttle/pkg/config/shuttleconfig.go:70 +0x2ad
github.com/lunarway/shuttle/pkg/config.(*ShuttleProjectContext).Setup(0xc0000ce240, 0xc000012640, 0x48, 0x13238dc, 0x4, 0x13238dc, 0x4, 0x0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/lunarway/shuttle/pkg/config/shuttleconfig.go:38 +0x63
github.com/lunarway/shuttle/cmd.getProjectContext(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/lunarway/shuttle/cmd/root.go:71 +0x1d7
github.com/lunarway/shuttle/cmd.glob..func6(0x15659e0, 0x158a0b8, 0x0, 0x0)
	/home/travis/gopath/src/github.com/lunarway/shuttle/cmd/ls.go:26 +0x37
github.com/spf13/cobra.(*Command).execute(0x15659e0, 0x158a0b8, 0x0, 0x0, 0x15659e0, 0x158a0b8)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0x15665c0, 0x8, 0x9, 0xc0000bc380)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0x15665c0, 0x1566360, 0xc0000dbf20)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:800 +0x2b
github.com/lunarway/shuttle/cmd.Execute()
	/home/travis/gopath/src/github.com/lunarway/shuttle/cmd/root.go:49 +0x3f
main.main()
	/home/travis/gopath/src/github.com/lunarway/shuttle/main.go:8 +0x20

We should handle the error more gracefully. An error log with the message No shuttle.yaml present in directory should suffice.

Magefile is a make replacement for go, it works by having go files act as commands.

such that:

cmd/run_task.go

mage run_task

will work. It does this automatically without having to register the command anywhere, which greatly helps discovering and building commands. This combined with plans could be great boon for quickly adding actions purely in go.

I will probably experiement a bit with this at some point: https://magefile.org/

The only real hurdle for now, is how it discovers commands stored in plans.

TestConfig often fails and then succeeds on the next run.

It does not make sense that scripts do not have actions, so fail running the script if so. Plus maybe consider a warning when doing an overall validation.

Would be great with a chapter in the readme on how to install shuttle. Should be easy to do since we are using goreleaser to generate the releases (others must have written mostly the same stuff)

When a run script is executed with a exit code other than 0, it's outputted wrong:

shuttle failed
Failed executing shell script `<shell command>`
Exit code: %!(EXTRA int=1)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• Update module github.com/spf13/cobra to v1.6.0
• Update module gopkg.in/yaml.v2 to v3

Detected dependencies

What are shuttle cli extensions.

You may have used git extensions before without knowing. Those are the commands you run after git, such as git ignore, or other third party git plugins. This is a useful way of using the git name, to namespace the cli tool.

The way those work is that git, will look for binaries in PATH on your machine, if it finds a binary with the prefix, git-, it will allow those to be called. Such as git-ignore can now be called with git ignore (notice the lack of -).

git ignore -> git -> git-ignore. Or for shuttle
shuttle <some-binary> -> shuttle -> shuttle-<some-binary> replace with whatever tool you'd like to expose.

shuttle would also just forward all commandline arguments, such as.

shuttle release --env dev -> shuttle-release --env dev

Why is this relevant to shuttle

Right now shuttle is built to handle code sharing between projects often in a multi-repository setting. This feature would allow us to ship additional features, which may make shuttle useful outside of project work. This could be everything from working with other third party software, such as github actions, renovate, reviewing code. Specifically tools which aren't used by one specific project, but goes across a developers workflow on said projects.

If we wanted to, we could even do the same for hamctl, such that we would have a shuttle-release, which would just point at hamctl for actions. This would group all of our tools under one umbrella, without having to add specific code to shuttle itself.

Let me know what you think!

Implementation proposal

We could simply do the same as git, and look for binaries in PATH, if found we add an entry to help menu, such that they show up when using shuttle -h.

This would require looking in all folders mentioned in $PATH, or just a specific location we decide for shuttle, and if the binary is in there we pick it up and make it available.

As a user, I would like to architect my scripts in smaller chunks, but be able to call them all with a "super target", so that I can make only one call to shuttle.
Example (written in make syntax)

# Super target for destroying infra. 
.PHONY: all-destroy
all-destroy: vacuum-cluster \
             cluster-rm \
             networking-rm \
             vpc-rm \

If a shuttle has_script integration, then we could easily extend plan with additional functionality

Shuttle telemetry doesn't react to an old lock file, or clean up after itself

It would be great if one could get the description of a specific script and it's arguments like below.
If the available arguments could be displayed as with the build in commands (shuttle run --help) it would be even better.

shuttle run build --help

Support scripts with default values for non-required arguments