tacc-cloud / abaco-cli Goto Github PK

The original behavior for abaco init was to create a new project in $PWD. This caused some minor degree of annoyance if the parent directory was already managed by Git, or if it was just inconvenient for this to be the location of the directory.

Enable specification of the base path to abaco init, checking that the designated path exists and failing with a CRITICAL error if not.

The abaco workers command will only scale worker count up or return the current state. It should also support scaling workers down, in LIFO ordering.

The abaco init workflow sets up the project directory as a git repo as of CLI v1.3.0. This behavior will be extended by attempting to add and commit the initial file set to the git repository.

Other requirements:

• Process must be able to fail gracefully
• Users should be able to disable this (and subsequent) steps in the abaco init workflow on an ad hoc basis.
• Users should be able to set a global default for this behavior

With the release of the autoscaler in Abaco 1.2.0, it is critical to support the actor stateless parameter. Actors registered as being stateless will automatically be scaled via the auto-scaler, so it is important that this become the default.

This should be implemented as follows:

• In reactor.rc the variable REACTOR_STATEFUL sets project-level behavior
  • Default is empty (REACTOR_STATEFUL=)
• In abaco init a new line for REACTOR_STATEFUL needs to be added to the templated config file
• In abaco create support the -s|-S options to create a stateless or stateful actor
• In abaco deploy support -s|-S > REACTOR_STATEFUL to manage actor statelessness

The CLI doesn't refresh the authentication session. If we factored the auth CLI from agave-cli into its own module and made it a dependency of agave-cli and abaco-cli it might resolve this common problem without having to duplicate authn/authz code in Abaco CLI.

Abaco create and deploy need to support the new resource limits for individual actors: max_cpus/maxCpus & mem_limit/memLimit

New in Abaco 1.3.0 is a setting that allows users to toggle automatic generation of Oauth tokens at Actor execution. If this is turned off for an actor, it is no longer possible to (automatically) make scripted Tapis API calls, but startup time is reduced substantially since there is no roundtrip to the Tapis /token endpoint to fetch an access token.

This needs to be supported in the CLI as follows:

• In reactor.rc, the REACTOR_ISSUE_TOKEN variable sets behavior for the project.
  • Default is 1 or True.
• In abaco init we must add a line for REACTOR_ISSUE_TOKEN to the templated reactor.rc file
• In abaco create support -A to disable token issuance
• In abaco deploysupport -A > REACTOR_ISSUE_TOKEN to manage token issuance

Although only for administrators to use, Abaco does now feature a named queue field.

This will be supported in the CLI as follows:

• In reactor.rc, the REACTOR_QUEUE variable sets behavior for the project.
  • Default is none
• In abaco init we must add a line for REACTOR_QUEUE to the templated reactor.rc file
• In abaco create support -Q to specify the queue
• In abaco deploysupport -q > REACTOR_QUEUE to manage queue

The CLI needs support (create, update, delete, list, permissions ) for Abaco's new aliases feature

As part of assisting with routine debugging, it is helpful to ask users to report the CLI version, and it is possible for older versions Abaco to be deployed on various tenants or even offsite, so having that detail is helpful, too.

Implement this as follows:

• Add an abaco version command
• Report CLI version based on a VERSION file embedded in the repo
• Report Abaco version by querying an Abaco service and extracting the version response
• Return either plaintext or JSON responses

I think we need a separate command to generate nonces against aliases. For example:

abaco nonces create nonce_test

would generate this curl call (I think):

curl -sk -H "Authorization: Bearer $TOKEN" -d "maxUses=-1&level=EXECUTE" https://api.sd2e.org/actors/v2/nonce_test/nonces

However this will actually resolve to an actor ID, and the resulting nonce will not work with an alias. See Joe's comment here:
https://tacc-team.slack.com/archives/C8QCFPLBW/p1572611242102700

To generate a nonce against an alias, we need to use the alias endpoint:

curl -sk -H "Authorization: Bearer $TOKEN" -d "maxUses=-1&level=EXECUTE" https://api.sd2e.org/actors/v2/aliases/nonce_test/nonces

So I think we'd need a separate CLI command abaco aliases nonces create maybe?

To support templates for non-TACC Abaco deployments, we will add an additional level to the templates directory corresponding to the tenants service host. For example, the tenants URL for the public, multitenant TACC Tapis is by https://api.tacc.utexas.edu/tenants. The corresponding top-level template directory is api.tacc.utexas.edu, with directories for each tenant therein.

To implement:

1. Add the TACC top-level directory to templates
2. Extend abaco init to accept an override tenants URL via -t
3. Add support for TAPIS_TENANTS_URL as a means of setting a global default

Information about the Abaco CLI should be discoverable from the main Abaco docs. There is a stub (https://tacc-cloud.readthedocs.io/projects/abaco/en/latest/tools/cli.html) but it simply says Coming soon. The CLI is mature enough to be publicly referenced.

A workflow that grew out of some user's CICD workflows is to tell abaco deploy to override the value specified by DOCKER_IMAGE_VERSION in the reactor.rc file with the current Git repo short hash.

This is achieved by detecting the hash and passing it to the -t option, but this requires a separate, out-of-band step to detect the hash.

Extend the abaco deploy workflow to support this automatically:

1. Add DOCKER_USE_COMMIT_HASH as a reactor.rc variable
2. Add detection of current git commit hash to abaco deploy
3. Update behavior of -t so the image tag is set in the following order when DOCKER_USE_COMMIT_HASH=1

Argument -t [value] > [git commit hash] > latest if git | DOCKER_IMAGE_VERSION if ! git

For example, using a -l switch to get the last execution:

$ abaco executions -l <my_actor_alias>
P0BNvJo1M4xyl  COMPLETE

In practice, I typically only care about the most recent execution. In addition, I am using shell scripts that parse the most recent execution from an ever-growing list of executions, which might not be sustainable in the long term.

The abaco init workflow creates a directory containing various source and configuration files, but the directory itself is not managed by source control. This makes it challenging to use features like tagging with the most recent git commit hash, or (eventually), deploying actors via push to a remote repository rather than directly from the development host.

There are a handful of steps to be added to the abaco init workflow to make this a reality:

1. Initialize the new directory as a git repository
2. Add and commit the initial set of files
3. Script creation of a remote (GitHub, Gitlab, on-prem Gitlab, etc.) and set as upstream for the repository
4. Optional (and open for discussion): Push the initial commit

Steps 2-4 will be managed in other issues. This issue covers only initialization of the repository.

The original behavior for abaco init was to write a stub username your_registry_username in the DOCKER_HUB_ORG config variable. This was because the Docker client is not able to reliably discern a user's currently configured username.

It was sometimes possible when clients used the legacy ~/.docker/config.json form that included a hashed version of user credentials, but the Docker CLI has moved (rightly) to using platform-level credential managers like the OSX Keychain. It is possible to query any of these managers, as they all follow an approximate specification, but an implementation for each manager is required.

As a stopgap, allow username (or org) to be provided when abaco init runs:

1. The environment variable $DOCKER_HUB_ORG sets a global default value
2. If it is not provided, default to current value of $USER
3. If a value for -O is provided, override the established default
4. Write final value in the DOCKER_HUB_ORG setting of project reactor.rc

After working with the Abaco CLI a lot and amassing multiple actors with semantically informative names, it is a common mistake to do abaco ls <name> instead of abaco ls <actorId>. At present this returns an error, but it could just as easily act as a filter, and this would be much more friendly and useful.

Project templates date to the first release of the CLI. Since then, several new features were added to the abaco deploy workflow and additional tenants now feature Abaco.

1. Update the reactor.rc files to include the full complement of variables
2. Update the Dockerfile to point to tenant-specific or abacosamples base images

The CLI needs support for creating, listing, deleting Abaco nonces. This should include a user-facing feature that will return the complete URL for an actor + nonce, for use in populating webhook fields in applications like Github.

The abaco deploy workflow checks the major version reported by docker --version to detect a workable Docker installation. The CLI is intended for use with Docker CE 17+. However, the Docker Desktop is shipping Docker version 19.03 as of 8/1/2019 and that is not interpreted correctly by abaco deploy

As of Abaco 1.3.0, Abaco has a new synchronous execution mode, where a POST to an actor's mailbox will block until a response is received or timeout occurs. The result of the execution is returned as the body of the response. If bytes have been sent to the actor's /result endpoint, those bytes are returned. If not, the execution logs are returned.

Support this in abaco run as follows:

1. Add an -x option that specifies synchronous execution
2. Revise the message sending and response code to expect raw results in sync mode, no matter what the value of -v (verbose) is.

End users do not know what project types are available for a given tenant. Given that it is easy to contribute new ones to the CLI via PR, it is expected that diversification in type, configuration, and naming will happen quickly between tenants.

It should be possible to iterate over the tenant-specific template directories and return at least the template directory name if not some little bit of metadata included in each directory.

For instance, for the sd2e tenant, the output from abaco init -h might look like so, where the * next to reactor-python indicates it is the default:

Usage: abaco init [OPTION]... [IMAGE]

Initializes an Abaco actor project in a new directory.

The default value for -O can be changed by setting the
DOCKER_HUB_ORG environment variable (e.g. DOCKER_HUB_ORG=tacc).

Options:
  -h    show help message
  -n    project name (e.g. my-actor-name)
  -d    project description
  -l    project type (bionic-python | disco-python | reactor-python* | shell)
  -O    registry username/organization (sd2eadm)
  -B    base path (current directory)

A popular feature of the agave-cli is the ability to submit most complex data structures from files on disk. This is not really needed for most Abaco operations. However, abaco run is an exception.

When a user wishes to send a structured JSON document as the message payload, they must encode and quote it themselves, a shaky proposition in the UNIX command line. Add in any need to script job submission in subshell and it's nearly impossible. It is also challenging to share example messages with other users.

To address this:

1. Extend abaco run to accept -F <filename>
2. Read the file from disk, compressing and validating it as JSON using jq
3. Send to the target actor's mailbox with a Content-Type: application/json header

Abaco supports DELETE on actors/v2/<actorId>/messages as of version 0.12.0 which clears out the pending queue of messages. The CLI should support this.