chef-boneyard / delivery-cli Goto Github PK

The command line tool for the workflow capabilities in Chef Automate.

Home Page: https://www.chef.io/automate/

License: Apache License 2.0

Ruby 6.90% Shell 0.65% Rust 77.84% Makefile 0.98% Gherkin 13.63%

delivery-cli's Introduction

Delivery CLI

A command line tool for continuous delivery workflow. The delivery command is a component of Chef Delivery. It can be used to setup and execute phase jobs as well as interact with a Chef Delivery server.

It is a part of the ChefDK and can be downloaded here.

Getting Started With Delivery

In particular, you will want to be familar with:

Development

To get started make sure you have the following installed:

Homebrew
Ruby 2.4+
Rust 1.37.0 (brew install rust)
Openssl (brew install openssl)
ChefDK 15.15 or later

Main technologies used in this project:

Rust (learn more here)
Cucumber and Aruba for functional testing
Clap, a CLI parsing library for Rust

We use make to perform various development operations like building and testing. The commands reside in the Makefile, but the Makefile is only used for development. It is not used by omnibus or our delivery cookbooks.

Make targets:

make builds the project.
make test runs the unit and functional tests.
make cucumber will run the cucumber tests.
make clean will clean the state of the build.
make update_deps will clean the project and update the Cargo.lock file, this should be run periodically to pull in new deps and at the very least when upgrading to a new version of Rust.
make release builds the project with the --release flag.
make check builds and runs unit tests.

After you make the project, you can execute your compiled binary by running target/debug/delivery <delivery_args>. So, you could run something like

$ target/debug/delivery review

to test the review command with your code in it.

If, for whatever reason, you want to compile and test with cargo's --release flag, run make release, and use target/release/delivery, but it will take longer to compile.

Tips

You can set the logging level by exporting the RUST_LOG environment variable (e.g RUST_LOG=debug cargo run).
Export RUST_BACKTRACE=1 for better stack traces on panics.
Run only the embedded unit tests (not functional tests in the tests/ directory nor code embedded in doc comments) via cargo test --lib. You can also pass a module name to only run matching tests (e.g. cargo test --lib job).
To test a specific cucumber module, after a bundle install, you can run bin/cucumber features/<feature>.feature.

Cucumber Testing

We heavily rely on git in the project. Some of the cucumber tests mock out parts of git. Those mocks exist in features/support/fakebin/git.

To mock a git call in your cucumber test, set <NAME_OF_GIT_COMMAND>_MOCKED to true in your cucumber test. For example, features/job.feature mocks several git commands with:

  Given I set the environment variables to:
    | variable             | value      |
    | CHECKOUT_MOCKED      | true       |
    | MERGE_MOCKED         | true       |
    | CLEAN_MOCKED         | true       |
    | RESET_MOCKED         | true       |

You can also mock all of git by setting MOCK_ALL_BASH to true:

  Given I set the environment variables to:
    | variable           | value      |
    | MOCK_ALL_BASH      | true       |

NOTE THAT A FEW COMMANDS ARE MOCKED BY DEFAULT STILL! We want to change this eventually. They are:

clone
push
fetch
ls-remote
show

These commands will never actuall execute until their mocks are refactored out of features/support/fakebin/.

Updating Rust Version

When a new version of Rust comes out and it is on homebrew, it's time to update the Rust version in this repo. There are a few spots to update:

At the very top of the Makefile, change RUST_VERSION
omnibus-software's rust definitions default version (or override appropriately)

You should also run make release to bump the Cargo.lock file to get new versions of our dependencies.

Delivery Job Implementation Details

The delivery job subcommand is used to execute phase recipes for a project in a workspace. The Delivery server uses delivery job to execute the phases of each stage in the pipeline. The same command can be used locally to execute a phase job in the same way. You can also wire delivery job into your own pipeline orchestration if you are not using the Delivery server.

Jobs are run by specifying the stage and the phase. For example, the following command will run the unit phase of the verify stage:

delivery job verify unit --local

The delivery job subcommand will execute the phase recipe by carrying out the following steps:

Currently, the job command assumes the current branch is a feature branch with commits that are not on the target branch.
Clone the project repository to a workspace. The location can be customized using --job-root.
Merge the feature branch into the target branch. This merge will not be pushed anywhere and is often referred to as "the hypothetical merge"; it is the merge that would result if this change were to be approved at this time.
Fetch the build cookbook for the project. The build cookbook location is specified in the project's .delivery/config.json file.
Use berks to fetch dependencies of the build cookbook (as specified in the build cookbook's Berksfile).
Run chef-client in local mode with a run list consisting of only the default recipe of the build cookbook. The default recipe is intended for preparing a given node to be a "build node" for the project. Typical setup handled in the default recipe might include installing compilers, test frameworks, and other build and test dependencies. The default recipe is skipped when delivery job is invoked by a non-root and when the --skip-default option is specified.
Run chef-client in local mode with a run list consisting of only the specified phase recipe (e.g. unit).

Without the --local option, the command will look for configuration required when interacting with a Delivery server from either .delivery/cli.toml or additional command line options.

For local use, you can run multiple phases within a single run like so:

delivery job verify "lint syntax unit"

Node Attributes

Attributes specific to the project and change are made available for use in build cookbook recipes.

The global workspace path is accesssible at node['delivery']['workspace_path']

Workspace Details

Attributes in the node['delivery']['workspace'] namespace provide paths to the various directories in your change's workspace on your build node.

node['delivery']['workspace']['root']
node['delivery']['workspace']['repo']
node['delivery']['workspace']['chef']
node['delivery']['workspace']['cache']

Change Details

Attributes in the node['delivery']['change'] namespace provide details about this particular job execution.

node['delivery']['change']['enterprise']
node['delivery']['change']['organization']
node['delivery']['change']['project']
node['delivery']['change']['pipeline']
node['delivery']['change']['change_id']
node['delivery']['change']['patchset_number']
node['delivery']['change']['stage']
node['delivery']['change']['phase']
node['delivery']['change']['git_url']
node['delivery']['change']['sha']
node['delivery']['change']['patchset_branch']

Project Configuration Details

The contents of your .delivery/config.json file are made available to you in the node['delivery']['config'] namespace.

Note for Git Bash + MinTTY Users

If you're running delivery token on Windows in Git Bash with MinTTY you must include winpty before delivery token to avoid errors.

Contributing

For information on contributing to this project see https://github.com/chef/chef/blob/master/CONTRIBUTING.md

License & Authors

Author:: Adam Jacob ([email protected])
Author:: Seth Falcon ([email protected])
Author:: Jean Rouge ([email protected])
Author:: Tom Duffield ([email protected])
Author:: Jon Anderson ([email protected])
Author:: Salim Afiune ([email protected])

Copyright:: 2015 Chef Software, Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

delivery-cli's People

Contributors

Stargazers

Watchers

delivery-cli's Issues

Inconsistent --version output

> delivery --version
delivery 0.0.15 (454c3f37819ed508a49c971f38e42267ce8a47de)
> delivery api --version
delivery-api

Provide option to turn off terminal escape codes (color codes)

Delivery review emits terminal escape codes to provide color in its output. This is problematic for tools that wish to parse the output of delivery commands. For example, delivery-matrix attempts to strip the escape codes but failed to account for the fact that the underlying library delivery-cli is using will output different output based on the value of TERM. Thus, the regex doesn't cover all the possible codes that might be emitted for the sgr0 escape code that delivery-cli attempts to send to reset the color.

https://github.com/chef-cookbooks/delivery-cluster/blob/31308903144ece68898233882181d36892e3bdf7/.delivery/build/cookbooks/delivery-matrix/libraries/delivery_duplicate_change_on_pipeline.rb#L95

A --no-color option would allow users to bypass such complications altogether.

delivery setup fails if terminal does not support colors.

Our default mandated ssh program by default uses vt-100 as a terminal type, which does not support extended colors. (strict vt100 only supports 8 colors).

This results in the following traceback:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ColorOutOfRange', /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libcore/result.rs:837
note: Run with `RUST_BACKTRACE=1` for a backtrace.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ColorOutOfRange', /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libcore/result.rs:837

Perhaps disable colors by default on terminals that don't support the full set?

Replace `--auto-bump` with `--bump [ patch | minor | major ]`

Auto-bump for delivery review is really handy! Would be great if it could handle other version point releases

Thinking that --auto-bump flag could be deprecated for a --bump flag that could be provided with or without args. Without an arg, it would default to patch type, but minor and major could also be specified.

Triggering jobs locally should not change my `HOME` directory

As a user that loves to test locally using the delivery-cli I would like to consume my local gems and extra configuration I have on my local workstation, so I don't have to install the dependencies on the workspace on my own.

Problem Description

When executing delivery job verify "unit syntax lint" --local the cli changes the HOME directory to workspace/cache causing me to lose all the gems that I have in my ~/.chefdk. Additionally as we do not run the default.rb recipe that installs the dependencies of my cookbook/project then I enter into a state where I can't run it without manually installing the deps into the local workspace/cache.

Proposal Solution

Do not change the HOME directory if we pass --local as an argument.

delivery cli does not support debug mode

Issue:
There are many scenarios where delivery cli just fails with a generic, non-descript error.

Desired resolution:
Provide a debug flag for delivery cli.

Additional benefits:
Debug flag will greatly increase customers' ability to self-service
Debug flag will greatly increase supports' ability to assist customers with troubleshooting delivery related issues.

Example:

 delivery token
Chef Delivery
Loading configuration from C:\Users\chef\scott_ssh
Requesting Token
An API Error occured
SAML lookup request returned 404 Not Found

Other examples:
Delivery init fails on ssh command to the delivery server, there is no stack trace or reference to what actually failed. The ssh command and git commands have to be ran individually in debug mode to get any actionable output.

@glasschef for more examples

Enable failing cucumber tests inside the pipeline

Describe the problem

Some of the cucumber test pass locally but not in the pipeline, this issue is to
troubleshoot the difference between them, currently the tests have been disabled
with the following PR #65

Root dir and project path not correct for git submodules

Delivery does not work correctly for git submodules. Since version 1.7.x, git, by default, absorbs .git directories from submodules into a modules folder within the parent .git directory leaving only a .git file containing the path to the .git directory

Delivery, therefore cannot find the correct root based on the logic here:
https://github.com/chef/delivery-cli/blob/master/src/delivery/project/mod.rs#L178-L216

delivery init for example, instead of recognizing that the cwd is actually a git submodule from the .git file, present in the cwd, delivery walks into the parent directory to find a .git directory with a config file.

This, in turn, returns an incorrect project path, adds an incorrect remote path to the parent repo instead of adding the remote to the submodule as it's supposed to via:
https://github.com/chef/delivery-cli/blob/master/src/delivery/git/mod.rs#L294-L315

If the project root was correctly determined and the git remote add command was run from the correct directory, delivery would function with git submodules.

Delivery local mode fails without a git config

Delivery local mode will not work without .git/config, which shouldn't be necessary for a delivery mode that doesn't interact with the SCM system.

Acquiring first token not possible non-interactively

Noticed this while trying to build a demo and there are two possible resolutions and think we should do both:

delivery token doesn't accept a password non-interactively so one may try to use delivery api post get-token but that also requires a token so it's a chicken-egg problem. The workaround is just using curl raw but it's reasonable to expect delivery api to behave similarly to curl.

add password option to delivery token
allow delivery api to work without a token, at least for posting to the get-token endpoint

delivery's cli.toml has no dynamism for user

In order for delivery to have its config stored for the project, you either have to run delivery setup with a bunch of flags that stay the same for the project between users while gitignoring the resulting cli.toml (to prevent the file from being committed with the username) or commit the cli.toml without a username and then encourage people to use the --user flag, although this doesn't work either.

You get this in the latter case:

$ delivery review 
Chef Delivery
A configuration value is missing
User not set; try --user or set it in your .toml config file

$ delivery review --user myuser
error: Found argument '--user' which wasn't expected, or isn't valid in this context

USAGE:
    delivery review [FLAGS] [OPTIONS]

For more information try --help

If there were some erb allowed in the cli.toml (ala Kitchen's allowance for it in kitchen.yml), you could just specify an environment variable for it or some other logic. At worst, a statically pulling the username (OPSCODE_USER?) from a specific environment variable would still be okay.

What's the best practice here? I want my developers to just be able to clone a repo and use delivery-cli without having to configure anything.

Delivery init fails if you have uncommitted changes in the repo

I had a github pull request template that I changed and didn't commit yet before I ran this:

MBP:~/dev/chef-cookbooks/tomcat (master) delivery init --local
Chef Delivery
Loading configuration from /Users/tsmith/dev/chef-cookbooks/tomcat
Generating default build cookbook...
Tried to fork a process, and failed
Failed to execute chef generate:
STDOUT: Recipe: code_generator::build_cookbook
  * generator_desc[Ensuring delivery configuration] action write
    - Ensuring delivery configuration

  * directory[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery] action create (up to date)
  * cookbook_file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/config.json] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/config.json
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/config.json from none to 2730e6
    (diff output suppressed by config)
  * cookbook_file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/project.toml] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/project.toml
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/project.toml from none to 25cf8a
    (diff output suppressed by config)
  * generator_desc[Ensuring correct delivery build cookbook content] action write
    - Ensuring correct delivery build cookbook content

  * directory[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook] action create
    - create new directory /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/metadata.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/metadata.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/metadata.rb from none to d2307d
    (diff output suppressed by config)
  * cookbook_file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/README.md] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/README.md
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/README.md from none to 1925b1
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/LICENSE] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/LICENSE
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/LICENSE from none to f750c0
    (diff output suppressed by config)
  * cookbook_file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/chefignore] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/chefignore
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/chefignore from none to f728e0
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/Berksfile] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/Berksfile
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/Berksfile from none to 7130db
    (diff output suppressed by config)
  * directory[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes] action create
    - create new directory /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/default.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/default.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/default.rb from none to 862109
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/deploy.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/deploy.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/deploy.rb from none to 1d5409
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/functional.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/functional.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/functional.rb from none to 16cf75
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/lint.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/lint.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/lint.rb from none to 89d88a
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/provision.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/provision.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/provision.rb from none to 0570a0
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/publish.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/publish.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/publish.rb from none to c23f2d
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/quality.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/quality.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/quality.rb from none to fc515b
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/security.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/security.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/security.rb from none to 8b199d
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/smoke.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/smoke.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/smoke.rb from none to e9cc27
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/syntax.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/syntax.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/syntax.rb from none to 571d4c
    (diff output suppressed by config)
  * template[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/unit.rb] action create_if_missing
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/unit.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/recipes/unit.rb from none to 97bfd5
    (diff output suppressed by config)
  * cookbook_file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/.kitchen.yml] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/.kitchen.yml
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/.kitchen.yml from none to af1492
    (diff output suppressed by config)
  * directory[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/data_bags/keys] action create
    - create new directory /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/data_bags/keys
  * file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/data_bags/keys/delivery_builder_keys.json] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/data_bags/keys/delivery_builder_keys.json
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/data_bags/keys/delivery_builder_keys.json from none to eedd06
    (diff output suppressed by config)
  * directory[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/secrets] action create
    - create new directory /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/secrets
  * file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/secrets/fakey-mcfakerton] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/secrets/fakey-mcfakerton
  * directory[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/recipes] action create
    - create new directory /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/recipes
  * file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/metadata.rb] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/metadata.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/metadata.rb from none to 2a9650
    (diff output suppressed by config)
  * cookbook_file[/Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/recipes/default.rb] action create
    - create new file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/recipes/default.rb
    - update content in file /Users/tsmith/dev/chef-cookbooks/tomcat/.delivery/build_cookbook/test/fixtures/cookbooks/test/recipes/default.rb from none to 28715b
    (diff output suppressed by config)
  * generator_desc[Adding delivery configuration to feature branch] action write
    - Adding delivery configuration to feature branch

  * execute[git-create-feature-branch] action run
    - execute git checkout -t -b add-delivery-configuration
  * execute[git-add-delivery-config-json] action run
    - execute git add .delivery/config.json
  * execute[git-add-delivery-project-toml] action run
    - execute git add .delivery/project.toml
  * execute[git-commit-delivery-config] action run

    ================================================================================
    Error executing action `run` on resource 'execute[git-commit-delivery-config]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of git commit -m "Add generated delivery configuration" ----
    STDOUT: On branch add-delivery-configuration
    Your branch is up-to-date with 'master'.
    Changes not staged for commit:
        modified:   .github/PULL_REQUEST_TEMPLATE.md

    no changes added to commit
    STDERR:
    ---- End output of git commit -m "Add generated delivery configuration" ----
    Ran git commit -m "Add generated delivery configuration" returned 1

    Resource Declaration:
    ---------------------
    # In /opt/chefdk/embedded/lib/ruby/gems/2.3.0/gems/chef-dk-0.19.1/lib/chef-dk/skeletons/code_generator/recipes/build_cookbook.rb

    137:   execute("git-commit-delivery-config") do
    138:     command("git commit -m \"Add generated delivery configuration\"")
    139:     cwd delivery_project_dir
    140:
    141:     only_if "git status --porcelain |grep \".\""
    142:   end
    143:

    Compiled Resource:
    ------------------
    # Declared in /opt/chefdk/embedded/lib/ruby/gems/2.3.0/gems/chef-dk-0.19.1/lib/chef-dk/skeletons/code_generator/recipes/build_cookbook.rb:137:in `from_file'

    execute("git-commit-delivery-config") do
      action [:run]
      retries 0
      retry_delay 2
      default_guard_interpreter :execute
      command "git commit -m \"Add generated delivery configuration\""
      backup 5
      cwd "/Users/tsmith/dev/chef-cookbooks/tomcat"
      returns 0
      declared_type :execute
      cookbook_name :code_generator
      recipe_name "build_cookbook"
      only_if "git status --porcelain |grep ".""
    end

    Platform:
    ---------
    x86_64-darwin13


STDERR: ERROR: Chef failed to converge: execute[git-commit-delivery-config] (code_generator::build_cookbook line 137) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of git commit -m "Add generated delivery configuration" ----
STDOUT: On branch add-delivery-configuration
Your branch is up-to-date with 'master'.
Changes not staged for commit:
    modified:   .github/PULL_REQUEST_TEMPLATE.md

no changes added to commit
STDERR:
---- End output of git commit -m "Add generated delivery configuration" ----
Ran git commit -m "Add generated delivery configuration" returned 1
Caused by: (Mixlib::ShellOut::ShellCommandFailed) execute[git-commit-delivery-config] (code_generator::build_cookbook line 137) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of git commit -m "Add generated delivery configuration" ----
STDOUT: On branch add-delivery-configuration
Your branch is up-to-date with 'master'.
Changes not staged for commit:
    modified:   .github/PULL_REQUEST_TEMPLATE.md

no changes added to commit
STDERR:
---- End output of git commit -m "Add generated delivery configuration" ----
Ran git commit -m "Add generated delivery configuration" returned 1

delivery local should have the ability to leverage build cookbook

The fact that delivery local uses a totally different approach than when using Automate (aka, delivery local defines its stuff in project.toml), makes the glide path from local testing before connecting to an Automate server a lot tougher.

It would be super helpful if delivery local leveraged build cookbooks, so then there is a consistent approach to building out tests for cookbooks whether you are using Automate or not.

Doing a 'make dev' fails during CCR due to missing recipe

Repro'd on a Mac OSX.

Clone delivery-cli (master).
Follow the instructions in README.md by doing a 'make dev all'

Result:
Chef client run fails with error:

================================================================================
Recipe Compile Error

Chef::Exceptions::RecipeNotFound

could not find recipe _openssl for cookbook delivery_rust

Agent Output on Windows introduces errors for git porcelain parsing

https://github.com/chef/delivery-cli/blob/6862f27aba89109a9630f0b6c6798efec56b4efe/src/delivery/git/mod.rs#L240

comes back with:

updating local tracking ref 'refs/remotes/delivery/_for/master/bugfix'
A line of git porcelain did not match!
Failed to match: Agent pid 6112
[hhla][cookbook-sina] on [bugfix ≡]

the Agent pid 6112 part is from the ssh-agent right after any of the wrapped git commands

https://github.com/chef/delivery-cli/blob/6862f27aba89109a9630f0b6c6798efec56b4efe/src/delivery/git/mod.rs#L118

Old data in config.json contaminates future runs

Description

Array fields in the delivery config.json are merged with saved attribute state from old runs, resulting in unexpected and somewhat random results.

Replication case

Use a single build node, so successive runs will be in the same node and workspace.

Start with config.json like this:

{
    "version":"2",
    "build_cookbook":{
        "path":".delivery/build-cookbook",
        "name":"build-cookbook"
    },
    "skip_phases":[
        "security",
        "quality",
        "smoke",
        "functional"
    ],
    "build_nodes":{       
    },
    "dependencies":[  
    ],
    "delivery-truck":{
        "publish":{
            "chef_server":true
        },
        "lint": {
            "foodcritic": { 
                "fail_tags": [ "any" ]
            }
        }
    }
}

Run a changed cookbook through verify... lint should have -f any
Now change the config.json to this:

        "lint": {
            "foodcritic": { 
                "fail_tags": [ "correctness", "metadata" ]
            }
        }

Run the changed cookbook through... lint has -f any,correctness,metadata

Logon to build node, go to the workspace for the phase:

cd /var/opt/delivery/workspace/33.33.33.11/test/test/mvt/master/verify/lint

Look at the attribute json for the phase in chef/dna.json and it is correct:
{"lint":{"foodcritic":{"fail_tags":["correctness","metadata"]}}
Look at the stored node state in chef/nodes/build-node-test-1 and it has the merged data:

          "lint": {
            "foodcritic": {
              "fail_tags": [
                "any",
                "correctness",
                "metadata"
              ]
            }

The stored node state between the first and second run would have been:

          "lint": {
            "foodcritic": {
              "fail_tags": [
                "any"
              ]
            }

which is being merged with the dna.json. This happens for any array attribute that's being passed through for build recipes to use. It's the defined Chef array merge behavior, but in this case it really isn't what the user wants, and there isn't an obvious workaround.

Seems like either:

chef-client should be run with -o rather than -r so node state isnt stored
OR
the node state should be cleared before each run

Find .delivery config even when traversing symlinks

Due to issues with an encrypted home directory, I sometime am in a workspace that lives outside $HOME, but is symlinked into it. pwd command still tracks where I am properly.

~/repos/confluence (automate-master ✔) ᐅ ll ~ | grep repos
lrwxrwxrwx  1 patcon     20 Apr 25  2014 repos -> /non_encrypted/repos
~/repos/confluence (automate-master ✔) ᐅ pwd              
/home/patcon/repos/confluence

But it seems that delivery gets confused and can't load config. This caused me to lose quite a bit of time, and would be wonderful if it could orient itself :)

Error's during omnibus build delivery-cli

We are getting some errors during omnibus build https://github.com/chef/delivery-cli/tree/master/omnibus-delivery-cli#build
$ bin/omnibus build delivery-cli

Refer this gist for logs- https://gist.github.com/siddheshwar-more/7ad40bc0816fb51492b3

Note- we are using current master branch .

Could you please help us to resolve this build issue.

Thank you!

Delivery init without a delivery server just hangs

If you're not actually setup for Delivery and you delivery init in a cookbook it just hangs forever:

MBP:~/dev/chef-cookbooks/tomcat (master) delivery init
Chef Delivery
Loading configuration from /Users/tsmith/dev/chef-cookbooks/tomcat

...nada...

local does not skip `functional` when commented out

I have delivery installed via chef-dk with the following versions:

Chef Development Kit Version: 2.3.4
delivery version: master (73ebb72a6c42b3d2ff5370c476be800fee7e5427)

It is my understanding, that if the functional phase is commented out then delivery will skip it. However, when running one of the roll-up commands, it still attempts to run the functional phase resulting in this error:

Unable to execute an empty phase.
Please verify that your project.toml has a functional phase configured as follows:

[local_phases]
functional = "insert script here"
Phase failed with exit code (1)!

I built the cookbook with the generate command, so chef was kind enough to autogenerate both the project.toml and the config.json files. The autogenerated project.toml file contains the functional phase commented out. I attempted to add functional to the skip_phases in the config.json, unfortunately, this did not cause delivery to skip the phase.

What else am I missing? Or is this a legitimate bug?

Berks Dependency Issue for delivery_rust

Hi,

I tried to use the latest version, but berks threw an error when it tried to pull the artifactory-pro cookbook.

Looking at https://github.com/chef/delivery-cli/blob/master/cookbooks/delivery_rust/Berksfile, it seems that the following cookbooks haven't been published to github.com

Am I missing something here?

Cheers

Delivery local mode does not work as expected in mono repos

It looks like delivery local mode is really build around the concept of a single cookbook repository and there's some error messaging that doesn't entirely explain that.

Given a repo structure like this:

/Users/tsmith/repo/cookbooks/my_cookbook

I cannot run delivery local mode in the my_cookbook directory as expected. Even though chef generate cookbook created a .delivery directory with the project.toml delivery says there is no .delivery/project.toml file. That's because it looks for the .delivery directory where the .git directory is. It traverses up the tree until it finds the .git directory in the chef-repo itself and then of course it can't find the .delivery directory.

Problem 1: It should just use the .delivery directory we generate
Problem 2: The error message it provides is incorrect

Could not find the configuration file.
The .delivery/project.toml file was not found.

You can generate this file using the command:
	chef generate build-cookbook [NAME]

The error should give the full path to the toml it can't find.

It would be useful to be able to control the logging level of the chef-client run used during an automate job

For debugging purposes and whatnot, it could be useful to increase or change the logging level of the chef-client used during an automate job.

However, I haven't been able to find any setting that affects the configuration used during an automate chef-client run. Looking at the delivery cli code, I can see that it uses a static configuration defined at https://github.com/chef/delivery-cli/blob/master/src/delivery/job/workspace.rs#L51

and static command line options defined at https://github.com/chef/delivery-cli/blob/master/src/delivery/job/workspace.rs#L271 to determine options for the chef-client run.

Neither of these exposes any way to affect the logging level, so I believe you're always stuck with the default.

Delivery init with --local flag creates a build cookbook and config

When doing a delivery init with the local flag you still get a lot of Delivery the product sort of files. It seems like it would be really helpful to handle this different so we could push adoption of delivery CLI as a local workflow tool which leads into use of Automate.

delivery review claims to fail to push branch to BitBucket repo

What is actually happening here?

Chef Delivery
Loading configuration from /Users/matt/src/myproject
Review for change mybranch targeted for pipeline master
Created new patchset        
Failed to push branch to Bitbucket: fatal: remote error: Repository not found        
The requested repository does not exist, or you do not have permission to        
access it. 
(url)

The branch exists on both origin and delivery, I can manipulate them just fine with git. Which is failing to push? Not being able to put delivery into a debug output mode is rather frustrating.

The updated changeset does appear on the Chef Automate server despite whatever supposedly failing push.

Unable to run delivery init successfully

Hi,

I have my workstation installed and configured in my Chef Server which has coookbooks in them. I tried running the delivery set up and created a token. While trying to initialize using 'delivery init' I get:

root@chef-server:/tmp/test_cookbook# delivery init --repo-name test_cookbook --server 10.X.X.XX --ent delivery-org --org delivery-org --skip-build-cookbook --user admin
Chef Delivery
Creating Delivery project...
Delivery project named test_cookbook was created.
Setting up the 'delivery' git remote...
The delivery git remote has been configured to 'ssh://admin@[email protected]:8989/delivery-org/delivery-org/test_cookbook'.
Pushing initial git history...
Git command failed!
STDOUT:
STDERR: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I have created the keys and have placed them in the git account that is the public key but unfortunately I didn't find it successful.

root@chef-server:/tmp/test_cookbook# delivery init --repo-name test_cookbook --server 10.X.X.X --ent delivery-org --org delivery-org --skip-build-cookbook --user admin --github delivery
Chef Delivery
Github Source Code Provider configuration not found; a Delivery administrator must first configure the link with Github.

Does anyone know why this is happening? It would be a great help. Thank you!

rename to `automate-cli`

I was looking for the repo and looked for automate-cli. Should it be renamed?

Running `delivery init` in the wrong directory leads to a half-configured/broken delivery project and unhelpful error message.

Environment information

$ chef -v
Chef Development Kit Version: 0.17.17
chef-client version: 12.13.37
delivery version: master (f68e5c5804cd7d8a76c69b926fbb261e1070751b)
berks version: 4.3.5
kitchen version: 1.11.1

Delivery server version: 0.5.125

We use delivery with Github Enterprise for our SCM system. When we accidentally run delivery init in the wrong directory, a project is set up in delivery with the name of the directory in which the init was run, but is pointing to the github repo specified in the delivery init command.

If you try to fix it by running the same delivery init command in the correct directory:

a standard 500 error is thrown that doesn't contain information specifying what to do;
One must 'know' that the other project must be deleted, and that the project name is the errant directory in which the command was run, vs. what the user thinks it should be.

Repro steps

Create a cookbook and initialize it on a Github Enterprise server, e.g.

cd /tmp
chef generate cookbook test_cookbook
cd test_cookbook
hub create my_github_org/test_cookbook
git add .
git commit -m "Initial commit."
git push origin master

Create another git repo and accidentally initialize it in delivery pointing at the repo from step 1:

cd /tmp
mkdir derp
cd derp
git init
delivery init --repo-name test_cookbook --github my_github_org --server my.delivery-server.com --ent my_delivery_ent --org my_delivery_org --skip-build-cookbook --user $USER

Realize your mistake and try to run the init properly:

cd /tmp/test_cookbook
delivery init --repo-name test_cookbook --github my_github_org --server my.delivery-server.com --ent my_delivery_ent --org my_delivery_org --skip-build-cookbook --user $USER

Results

An unhelpful error message on the CLI:

delivery init --repo-name test_cookbook --github my_github_org --server my.delivery-server.com --ent my_delivery_ent --org my_delivery_org --skip-build-cookbook --user $USER
Chef Delivery
Loading configuration from /tmp/test_cookbook
Creating Github backed Delivery project...
An API Error occured
API request returned 500 Internal Server Error

A somewhat helpful error in the delivery logs:

==> /var/log/delivery/delivery/console.log <==
2016-09-06 17:53:54.769 [error] <0.18659.6> failed_call={deliv_db:qfetch, [deliv_project,create_scm_project,[<<"my_delivery_ent">>,<<"my_delivery_org">>,<<"test_cookbook">>,<<"master">>,deliv_scm_github,<<"my_github_org">>,<<"test_cookbook">>]]}; reason={conflict,<<"duplicate key value violates unique constraint \"project_github_metadata_repo_owner_repo_name_key\"">>}

In this case, the database is set up with a project named derp

$ /opt/delivery/embedded/bin/psql -U delivery -c 'select * from projects order by id desc limit 1;'
 id  | organization_id |                 guid                 | name |    scm_module
-----+-----------------+--------------------------------------+------+------------------
 268 |               1 | bf17c4d7-759f-4eb0-a74f-15a2a366873b | derp | deliv_scm_github
(1 row)

and a project_github_metadata entry for test_cookbook

$ /opt/delivery/embedded/bin/psql -U delivery -c 'select * from project_github_metadata order by project_id desc limit 1;'
 project_id |  repo_owner  |   repo_name   | token
------------+--------------+---------------+--------
        268 | my_github_org | test_cookbook | unused
(1 row)

Workaround

One must delete the project, specifying the name of the directory in which the errant delivery command was run:
delivery api delete orgs/my_delivery_org/projects/derp --server my.delivery-server.com --ent my_delivery_ent --user $USER

Ideas for making this better

Potentially showing a warning/error if the directory name in which the init command is run doesn't match the repo name?
Make delivery init name the project within delivery whatever the github repo name is?
Reporting a better error message from the delivery init command, e.g. ERROR: existing project <name> already pointing at <github repo>
Reporting a better error message in the delivery server logs e.g. ERROR: existing project <name> already pointing at <github repo> in addition to/instead of the existing one, which assumes at least some knowledge of how the DB schema works with delivery.

delivery doesn't recognize review --user

Using the -e flag of delivery tells you to specify a user in the toml file or use --user, but doesn't actually allow you to use the --user flag.

~/src/chef$ delivery review --help
delivery-review master (0b746cafed65a9ea1a79de3cc546e7922de9187c)
Submit current branch for review

USAGE:
    delivery review [FLAGS] [OPTIONS]

FLAGS:
    -a, --auto-bump          Automatic cookbook version bump
    -e, --edit               Edit change title and description
    -h, --help               Prints help information
    -n, --no-open            Do not open the change in a browser
        --no-spinner         Disable the spinner
        --non-interactive    Disable cli interactions
    -V, --version            Prints version information

OPTIONS:
    -f, --pipeline <pipeline>    Target pipeline for change (default: master) [aliases: for]


~/src/chef$ delivery review -e
Chef Delivery
A configuration value is missing
User not set; try --user or set it in your .toml config file

~/src/chef$ delivery review -e --user matt
error: Found argument '--user' which wasn't expected, or isn't valid in this context

USAGE:
    delivery review --edit

For more information try --help

delivery local should have the ability to leverage build cookbook

It would be super helpful if delivery local leveraged build cookbooks, so then there is a consistent approach to building out tests for cookbooks whether you are using Automate or not.

Delivery local mode no longer pulls remote files.

Describe the problem

delivery local mode used to accept a remote toml file. It no longer does. As a result all of our CI pipelines were blocked. This stopped 160+ cookbooks from being tested.

Software Version

Latest at time of writing. We're using GitHub Actions which pulls down the latest Chef. It's been broken since 17.0.

Replication Case

Diff here is from the old remote file to a local file in repo
sous-chefs/java@ceaf80f

Stacktrace

$ chef exec delivery local all
An HTTP Error occurred

Possible Solution

Change every repository to use a local delivery file.

Delivery review does not respect cli toml when changing server.

We recently had an edge case where we were transitioning from one automate server to another for workflow. We'll call them "automate-old" and "automate-new" fr simplicity.
With a local project checked out from automate-old originally, I ran "delivery setup" with "--server automate-new" to reset my cli.toml. When I made a test change and ran "delivery review" it instead used the current git "delivery" remote and opened the change on automate-old.

Expected Behavior: Delivery review would note that the git remote did not agree with the cli.toml and repoint it appropriately.

"Invalid argument" for api requests with spaces in data

Putting a space inside the JSON (either within a field or simply between fields) results in an "Invalid arguments" error and usage message.

$ delivery api put users/edanaher  --data='{"ssh_pub_key":"ssh-rsaabc12345"}'
$ delivery api put users/edanaher  --data='{"ssh_pub_key":"ssh-rsa abc12345"}'
Invalid arguments.

Further, using "+" or "%20" (in the hopes that it's a URI-encoding issue) produces a literal "+" or "%20", and further escaping the space also resulted in the error.

Spaces work fine using the API directly; for example the following results in an ssh key with a space in it:

curl -H 'chef-delivery-user: edanaher' -H 'chef-delivery-token: [snip]' -i -k -H "content-type: application/json" -X PUT https://delivery/api/v0/e/ooyala/users/edanaher -d '{"ssh_pub_key": "abc 12345"}'

So this seems to be pretty clearly a bug in the CLI.

delivery local lint works but automate server lint fails

I have a class defined for a custom profile.But there is no comment given on top of the class.

$delivery local lint -> says no offenses detected

But when I submitted to the automate pipeline i.e delivery init, it fails at linting phase with an error message briefly 'The top-level class documentation missing'. This got solved as soon as I keep comments for the class.

I would have loved solving this before submitting to the automate server.

Is it inconsistency between local delivery run and server delivery run?

Delivery init fails if existing token is invalid

Authentication failed
API request returned 401
Details:
{
  "error": "token_denied"
}

I think the expected behavior is to request the password and fetch a new token.

chef-boneyard / delivery-cli Goto Github PK

delivery-cli's Introduction

Delivery CLI

Getting Started With Delivery

Development

Tips

Cucumber Testing

Updating Rust Version

Delivery Job Implementation Details

Node Attributes

Workspace Details

Change Details

Project Configuration Details

Note for Git Bash + MinTTY Users

Contributing

License & Authors

delivery-cli's People

Contributors

Stargazers

Watchers

Forkers

delivery-cli's Issues

Describe the problem

================================================================================ Recipe Compile Error

Description

Replication case

Describe the problem

Software Version

Replication Case

Stacktrace

Possible Solution

Recommend Projects

Recommend Topics

Recommend Org

================================================================================
Recipe Compile Error