poise / application_git Goto Github PK

View Code? Open in Web Editor NEW

11.0 4.0 8.0 70 KB

A Chef cookbook to handle deploying code from git when using the application cookbook.

License: Apache License 2.0

Ruby 100.00%

application_git's Introduction

Application_Git Cookbook

A Chef cookbook to handle deploying code from git when using the application cookbook.

Quick Start

To deploy from a private GitHub repository:

application '/srv/myapp' do
  git '[email protected]:example/myapp.git' do
    deploy_key chef_vault_item('deploy_keys', 'myapp')['key']
  end
end

Requirements

Chef 12.1 or newer is required.

Resources

`application_git`

The application_git resource deploys code from git. It extends the core git resource to support deploy keys and disabling strict host key verification.

application '/srv/myapp' do
  git '[email protected]:example/myapp.git'
end

Actions

All actions work the same as the core git resource.

:sync – Clone and checkout the requested revision (default)
:checkout – Checkout the request revision. If the repository isn't already cloned, this action does nothing.
:export – Export the repository without the .git folder.

Properties

All properties from the core git resource work the same way with the following additions:

deploy_key – SSH key to use with git. Can be specified either as a path to key file already created or as a string value containing the key directly.
strict_ssh – Enable strict SSH host key checking. (default: false)

DSL Usage

The application_git resource can be used directly as a replacement for the core git resource:

application_git '/srv/myapp' do
  repository '[email protected]:example/myapp.git'
  deploy_key chef_vault_item('deploy_keys', 'myapp')['key']
end

Within the application resource, a simplified DSL is available. As with other application plugins, the default name of the resource if unspecified is the application path. The following two examples are equivalent:

application '/srv/myapp' do
  git do
    repository '[email protected]:example/myapp.git'
  end
end

application '/srv/myapp' do
  git '[email protected]:example/myapp.git'
end

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

application_git's People

Contributors

Stargazers

Watchers

Forkers

h4ck3rm1k3 nsocheleau paulca giannicarlo baileyvw hieunba isabella232 rayat

application_git's Issues

Silently fails if directory exists

I had problems cloning the repo and the logs gives of something similar to this:
[2017-01-17T14:45:07+00:00] INFO: Processing application_git[a/b] action sync ()

When I enable debug log level I also see this:
[2017-01-17T14:45:08+00:00] DEBUG: application_git[[email protected]:a/b] checkout destination /path/to/dest already exists or is a non-empty directory

Of course the recipe breaks later on but it would be useful if the above step also passed on the cloning error in the logs, would ease troubleshooting.

Using the 'git' resource outside of an application block is not working as expected

Description

Using the git resource's environment attribute to indicate a PATH does not work as expected. The code works as expected when exercised as a simple core Chef resource via chef-apply but not when used in a cookbook that has application_git as a dependency (for use in recipes other than this one).

Chef Version

12.6.0

Platform Version

Solaris 10 SPARC

Replication Case

On our Solaris 10 SPARC machines, the following fails to find git in /usr/rcf/bin where it absolutely does exist and is world executable. This happens only when the code below is part of a cookbook that has application_git as a dependency:

env = {'PATH' => '/usr/rcf/bin:/usr/bin:/bin'}
git '/tmp/something' do
  repository 'https://github.com/jblaine/cookbook-sysmon.git'
  environment env
end

Client Output

Note that in the compiled resource information it does show the PATH set as desired, so there's something low-level going on here I think.

/:faron # /usr/rcf/bin/git --version
git version 1.8.4.1
/:faron # chef-client -E development -o 'r701-hostdb::_update-advanced'
...
resolving cookbooks for run list: ["r701-hostdb::_update-advanced"]
Synchronizing Cookbooks:
  - application_python (4.0.0)
  - application (5.0.0)
  - build-essential (2.2.4)
  - chef_handler (1.2.0)
  - dmg (2.3.0)
  - git (4.3.4)
  - cron (1.7.2)
  - windows (1.39.0)
  - yum (3.8.2)
  - chef-vault (1.3.0)
  - yum-epel (0.6.5)
  - r701-automounting (0.2.0)
  - r701-ldap (0.1.5)
  - r701-yum (0.2.15)
  - r701-chef-vault (0.1.2)
  - r701-rcftools (0.1.2)
  - r701-openafs (1.2.0)
  - r701-apache (0.4.61)
  - r701-hostdb (2.1.8)
  - poise (2.4.0)
  - poise-languages (1.2.0)
  - poise-python (1.1.2)
  - poise-service (1.0.3)
  - application_git (1.0.0)
Compiling Cookbooks...
Converging 4 resources
Recipe: r701-hostdb::_update-advanced
  * git[/usr/tools] action sync

================================================================================
Error executing action `sync` on resource 'git[/usr/tools]'
================================================================================

Errno::ENOENT
-------------
No such file or directory - git
...
    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/r701-hostdb/recipes/_update-advanced.rb:15:in `from_file'

git("/usr/tools") do
...
  environment {"PATH"=>"/usr/rcf/bin:/usr/bin:/bin"}
...
end

Stacktrace

https://gist.github.com/jblaine/45a902e46d1e9047bc6f

More importantly, truss -e -f (like strace but for Solaris) shows the search for git as:

20824/1:        chdir("/usr/rcf-tools/hostdb")                  = 0
20824/1:        stat64("/usr/bin/git", 0xFFBFAC58)              Err#2 ENOENT
20824/1:        stat64("bin/git", 0xFFBFAC58)                   Err#2 ENOENT
20824/1:        stat64("/opt/chef/embedded/bin/git", 0xFFBFAC58) Err#2 ENOENT
20824/1:        stat64("/opt/chef/embedded/bin/git", 0xFFBFAC58) Err#2 ENOENT
20824/1:        stat64("/usr/local/sbin/git", 0xFFBFAC58)       Err#2 ENOENT
20824/1:        stat64("/usr/local/bin/git", 0xFFBFAC58)        Err#2 ENOENT
20824/1:        stat64("/usr/sbin/git", 0xFFBFAC58)             Err#2 ENOENT
20824/1:        stat64("/sbin/git", 0xFFBFAC58)                 Err#2 ENOENT
20824/1:        stat64("/bin/git", 0xFFBFAC58)                  Err#2 ENOENT

And also shows that PATH is never including /usr/rcf/bin and is always set to PATH=/usr/bin:bin:/opt/chef/embedded/bin:/opt/chef/embedded/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/sbin:/bin for every exec-like call.

`user` and `group` are not honoured

application_git ignores both application's and git's user and group attributes.

The former is just ignored; the latter is not taken into account in creating deploy_key and ssh_wrapper, as non-root users are not allowed to read or execute root's files.

Branches

Is it possible to do a branch checkout when calling using this?

:user is required field on some hosts

For some reason, on a subset of my (essentially identical) hosts, chef fails unless I explicitly provide a :user to the git block:

#<ArgumentError: couldn't find HOME environment -- expanding `~'> had an error:
couldn't find HOME environment -- expanding `~'

It's fine when I run the client manually (sudo chef-client), but the automated runs fail.

The top of the stacktrace:

var/chef/cache/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:87:in `expand_path'
/var/chef/cache/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:87:in `ssh_wrapper_path'
/var/chef/cache/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:75:in `deploy_key'
/var/chef/cache/cookbooks/workbench/recipes/app.rb:89:in `block (2 levels) in from_file'

Where line 89 in my app.rb is:

 83 application node['workbench']['app']['home'] do                                                                                                                                                                 
 84   owner 'workbench'                                                                                                                                                                                             
 85   group 'workbench'                                                                                                                                                                                             
 86                                                                                                                                                                                                                 
 87   git do                                                                                                                                                                                                        
 88     repository node['workbench']['app']['repo']                                                                                                                                                                 
 89     deploy_key '/etc/id_rsa-workbench_deploy'                                                                                                                                                                   
 90     revision node['workbench']['app']['repo_revision']                                                                                                                                                          
 91   end                                                                                                                                                                                                           
 92 end

...and it fails in ssh_wrapper_path from this codebase:

    def ssh_wrapper_path
      @ssh_wrapper_path ||= ::File.expand_path("~#{user}/.ssh/ssh_wrapper_#{Zlib.crc32(name)}")
    end

If I add user 'root' to the git block, it works. (The id_rsa file is only readable by root), though it only works if the user declaration is above the deploy_key declaration.

The strange thing about this is that it only fails on a subset of my hosts.

Now that I've figured out that adding the user fixed it, I'm fine. Leaving this issue here in case anyone is searching for a solution later, or in case ya'll think a docs update is warranted.

Checkout a specific branch

Is there a way to specify which branch to checkout after cloning a repo? I'm trying to set up both staging and production environments using the same recipes and then select which git branch to use dynamically. Is this currently possible?

`user` does not exist during the first phase of chef recipe

There appears to be an issue where the user does not exist during the first phase of the chef recipe. What happens is that the user (to be created during convergence) does yet exist and so when the script attempts to write the ssh wrapper it cannot because the user is not yet in the system.

================================================================================
         Recipe Compile Error in /tmp/kitchen/cookbooks/topics/recipes/deploy.rb
         ================================================================================

         ArgumentError
         -------------
         user deploy doesn't exist

         Cookbook Trace:
         ---------------
           /tmp/kitchen/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:87:in `expand_path'
           /tmp/kitchen/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:87:in `ssh_wrapper_path'
           /tmp/kitchen/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:75:in `deploy_key'
           /tmp/kitchen/cookbooks/topics/recipes/deploy.rb:77:in `block (2 levels) in from_file'
           /tmp/kitchen/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:182:in `instance_exec'
           /tmp/kitchen/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:182:in `block (3 levels) in _rewire_dsl!'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:150:in `instance_exec'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:150:in `block (2 levels) in declare_resource'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:140:in `block in declare_resource'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `instance_eval'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `subcontext_block'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:121:in `declare_resource'
           /tmp/kitchen/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:175:in `public_send'
           /tmp/kitchen/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:175:in `block (2 levels) in _rewire_dsl!'
           /tmp/kitchen/cookbooks/topics/recipes/deploy.rb:72:in `block in from_file'
           /tmp/kitchen/cookbooks/topics/recipes/deploy.rb:64:in `from_file'

         Relevant File Content:
         ----------------------
         /tmp/kitchen/cookbooks/application_git/files/halite_gem/poise_application_git/resource.rb:

          80:      end
          81:  
          82:      # Default SSH wrapper path.
          83:      #
          84:      # @api private
          85:      # @return [String]
          86:      def ssh_wrapper_path
          87>>       @ssh_wrapper_path ||= ::File.expand_path("~#{user}/.ssh/ssh_wrapper_#{Zlib.crc32(name)}")
          88:      end
          89:  
        90:      # Guess if the deploy key is a local path or literal value.
          91:      #
          92:      # @api private

          94:      # @return [Boolean]
          95:      def deploy_key_is_local?(key=nil)
        96:        key ||= deploy_key

Where can I download this cookbook?

I realised that I can't download this cookbooks neither using Berkshelf, using git, supermarket, and so on. How can I use it with application_python?