Giter VIP home page Giter VIP logo

application_python's Introduction

Application_Python Cookbook

Build Status Gem Version Cookbook Version Coverage Gemnasium License

A Chef cookbook to deploy Python applications.

Quick Start

To deploy a Django application from git:

application '/srv/myapp' do
  git 'https://github.com/example/myapp.git'
  virtualenv
  pip_requirements
  django do
    database 'sqlite:///test_django.db'
    secret_key 'd78fe08df56c9'
    migrate true
  end
  gunicorn do
    port 8000
  end
end

Requirements

Chef 12.1 or newer is required.

Resources

application_celery_beat

The application_celery_beat resource creates a service for the celery beat process.

application '/srv/myapp' do
  celery_beat do
    app_module 'myapp.tasks'
  end
end

Actions

  • :enable – Create, enable and start the service. (default)
  • :disable – Stop, disable, and destroy the service.
  • :start – Start the service.
  • :stop – Stop the service.
  • :restart – Stop and then start the service.
  • :reload – Send the configured reload signal to the service.

Properties

  • path – Base path for the application. (name attribute)
  • app_module – Celery application module. (default: auto-detect)
  • service_name – Name of the service to create. (default: auto-detect)

user – User to run the service as. (default: application owner)

application_celery_config

The application_celery_config creates a celeryconfig.py configuration file.

application '/srv/myapp' do
  celery_config do
    options do
      broker_url 'amqp://'
    end
  end
end

Actions

  • :deploy – Create the configuration file. (default)

Properties

  • path – Path to write the configuration file to. If given as a directory, create path/celeryconfig.py. (name attribute)
  • options – Hash or block of options to set in the configuration file.

application_celery_worker

The application_celery_worker resource creates a service for the celery worker process.

application '/srv/myapp' do
  celery_worker do
    app_module 'myapp.tasks'
  end
end

Actions

  • :enable – Create, enable and start the service. (default)
  • :disable – Stop, disable, and destroy the service.
  • :start – Start the service.
  • :stop – Stop the service.
  • :restart – Stop and then start the service.
  • :reload – Send the configured reload signal to the service.

Properties

  • path – Base path for the application. (name attribute)
  • app_module – Celery application module. (default: auto-detect)
  • worker_pool – The Pool implementation used by the Celery worker (gevent,eventlet or prefork). (default: prefork)
  • service_name – Name of the service to create. (default: auto-detect)

user – User to run the service as. (default: application owner)

application_django

The application_django resource creates configuration files and runs commands for a Django application deployment.

application '/srv/myapp' do
  django do
    database 'sqlite:///test_django.db'
    migrate true
  end
end

Actions

  • :deploy – Create config files and run required deployments steps. (default)

Properties

  • path – Base path for the application. (name attribute)
  • allowed_hosts – Value for ALLOWED_HOSTS in the Django settings. (default: [])
  • collectstatic – Run manage.py collectstatic during deployment. (default: true)
  • database – Database settings for the default connection. See the database section below for more information. (option collector)
  • debug – Enable debug mode for Django. (default: false)
  • local_settings – A Poise template property for the content of the local settings configuration file.
  • local_settings_path – Path to write local settings to. If given as a relative path, will be expanded against path. Set to false to disable writing local settings. (default: local_settings.py next to settings_module)
  • migrate – Run manage.py migrate during deployment. (default: false)
  • manage_path – Path to manage.py. (default: auto-detect)
  • secret_key – Value for SECRET_KEY in the Django settings. If unset, no key is added to the local settings.
  • settings_module – Django settings module in dotted notation. (default: auto-detect)
  • syncdb – Run manage.py syncdb during deployment. (default: false)
  • wsgi_module – WSGI application module in dotted notation. (default: auto-detect)

Database Parameters

The database parameters can be set in three ways: URL, hash, and block.

If you have a single URL for the parameters, you can pass it directly to database:

django do
  database 'postgres://myuser@dbhost/myapp'
end

Passing a single URL will also set the $DATABASE_URL environment variable automatically for compatibility with Heroku-based applications.

As with other option collector resources, you can pass individual settings as either a hash or block:

django do
  database do
    engine 'postgres'
    user 'myuser'
    host 'dbhost'
    name 'myapp'
  end
end

django do
  database({
    engine: 'postgres',
    user: 'myuser',
    host: 'dbhost',
    name: 'myapp',
  })
end

application_gunicorn

The application_gunicorn resource creates a service for the Gunicorn application server.

application '/srv/myapp' do
  gunicorn do
    port 8000
    preload_app true
  end
end

Actions

  • :enable – Create, enable and start the service. (default)
  • :disable – Stop, disable, and destroy the service.
  • :start – Start the service.
  • :stop – Stop the service.
  • :restart – Stop and then start the service.
  • :reload – Send the configured reload signal to the service.

Properties

  • path – Base path for the application. (name attribute)
  • app_module – WSGI module to run as an application. (default: auto-detect)
  • bind – One or more addresses/ports to bind to.
  • config – Path to a Guncorn configuration file.
  • preload_app – Enable code preloading. (default: false)
  • port – Port to listen on. Alias for bind("0.0.0.0:#{port}").
  • service_name – Name of the service to create. (default: auto-detect)
  • user – User to run the service as. (default: application owner)
  • version – Version of Gunicorn to install. If set to true, use the latest version. If set to false, do not install Gunicorn. (default: true)

application_pip_requirements

The application_pip_requirements resource installs Python packages based on a requirements.txt file.

application '/srv/myapp' do
  pip_requirements
end

All actions and properties are the same as the pip_requirements resource.

application_python

The application_python resource installs a Python runtime for the deployment.

application '/srv/myapp' do
  python '2.7'
end

All actions and properties are the same as the python_runtime resource.

application_python_execute

The application_python_execute resource runs Python commands for the deployment.

application '/srv/myapp' do
  python_execute 'setup.py install'
end

All actions and properties are the same as the python_execute resource, except that the cwd, environment, group, and user properties default to the application-level data if not specified.

application_python_package

The application_python_package resource installs Python packages for the deployment.

application '/srv/myapp' do
  python_package 'requests'
end

All actions and properties are the same as the python_package resource, except that the group and user properties default to the application-level data if not specified.

application_virtualenv

The application_virtualenv resource creates a Python virtualenv for the deployment.

application '/srv/myapp' do
  virtualenv
end

If no path property is given, the default is to create a .env/ inside the application deployment path.

All actions and properties are the same as the python_virtualenv resource.

Examples

Some test recipes are available as examples for common application frameworks:

Sponsors

Development sponsored by Chef Software, Symonds & Son, and Orion.

The Poise test server infrastructure is sponsored by Rackspace.

License

Copyright 2015-2017, Noah Kantrowitz

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

application_python's People

Contributors

amccloud avatar bastien-jove-cbp avatar coderanger avatar comandrei avatar farshidce avatar guilhem avatar jblaine avatar lamont-granquist avatar mbacchi avatar mpaolini avatar rcloran avatar sethvargo avatar tinnet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

application_python's Issues

Non-simplistic gunicorn example?

depends 'application', '> 5.0'
depends 'application_python', '
> 4.0'

This is working for me:

...
  gunicorn do
    app_module 'gravatarFlask:app'
    port 80
  end
...

This isn't working for me:

...
  gunicorn do
    app_module 'gravatarFlask:app'
    config '--error-logfile /var/log/gunicorn_error.log --log-syslog-to tcp://speaker.our.org:514 --log-level info --log-syslog-prefix gravatar_gunicorn'
    port 80
  end
...

Any ideas what am I doing wrong?

application_python failing to install gunicorn on first deploy

The first deploy of an app that has the gunicorn service enabled fails with the following error:

[2013-12-15T07:15:38+00:00] FATAL: Chef::Exceptions::EnclosingDirectoryDoesNotExist: directory[/srv/splots/shared/env](/opt/aws/opsworks/releases/20131125133823_214/site-cookbooks/python/providers/virtualenv.rb line 31) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /srv/splots/shared does not exist, cannot create /srv/splots/shared/env

There is also a longer description of this bug on stackoverflow: http://stackoverflow.com/questions/20351292/application-python-failing-to-install-gunicorn-on-first-deploy

Django app fails when specifying Postgres port

When I specify a port for postgres, URI parse :5432 as an integer.
The code however checks to see if the value is empty?, which causes the following error:

:15:in `block in parse_database_url': undefined method `empty?' for 5432:Integer (NoMethodError)
	from delme.rb:4:in `tap'
	from delme.rb:4:in `parse_database_url'
	from delme.rb:19:in `<main>'

Recipe Compile Error on deploying gunicorn

My recipe:

application app_path do
  path app_path
  owner "www-data"
  group "www-data"
  package "git"

  git app_path do
    repository "~~~~"
    user "www-data"
    group "www-data"
  end
  pip_requirements

  gunicorn do
    port 9001
    app_module "manage:app"
  end

  environment ({'HOME' => '/var/www'})
end

Error message:

================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb
================================================================================

Errno::ENOENT
-------------
No such file or directory @ dir_initialize - /usr/share/classifier-api

Cookbook Trace:
---------------

  /tmp/kitchen/cache/cookbooks/application_python/files/halite_gem/poise_application_python/resources/gunicorn.rb:59:in `entries'
  /tmp/kitchen/cache/cookbooks/application_python/files/halite_gem/poise_application_python/resources/gunicorn.rb:59:in `default_app_module'

  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lazy_default.rb:50:in `instance_eval'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lazy_default.rb:50:in `set_or_return'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lwrp_polyfill.rb:114:in `block in attribute'
  /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb:70:in `block (2 levels) in from_file'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:149:in `instance_exec'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:149:in `block (3 levels) in _rewire_dsl!'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:125:in `instance_exec'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:125:in `block (2 levels) in declare_resource'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:115:in `block in declare_resource'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `instance_eval'

  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:96:in `declare_resource'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:144:in `public_send'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:144:in `block (2 levels) in _rewire_dsl!'
  /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb:68:in `block in from_file'
  /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb:55:in `from_file'


----------------------
/tmp/kitchen/cache/cookbooks/application_python/files/halite_gem/poise_application_python/resources/gunicorn.rb:

 52:          # checks the app state and then looks for commonly used filenames.
 53:          # Raises an exception if no default can be found.

 55:          # @return [String]
 56:          def default_app_module
 57:            # If set in app_state, use that.
 58:            return app_state[:python_wsgi_module] if app_state[:python_wsgi_module]
 59>>           files = Dir.entries(path)
 60:            # Try to find a known filename.
 61:            candidate_file = %w{wsgi.py main.py app.py application.py}.find {|file| files.include?(file) }
 62:            # Try the first Python file. Do I really want this?
 63:            candidate_file ||= files.find {|file| file.end_with?('.py') }
 64:            if candidate_file
 65:              ::File.basename(candidate_file, '.py')
 66:            else
 67:              raise PoiseApplicationPython::Error.new("Unable to determine app module for #{self}")
 68:            end

It occurs because the deploy target directory is not created yet on the compilation time.
It even happens even if specified app_module attribute.
How can I avoid it? How to specify python_wsgi_module of app_state?

metadata.rb missing

Hello,

My org maintains an internal supermarket with strict license restrictions, automated by checking the metadata.rb file. This repo doesn't have metadata.rb so I am not able to pull in the cookbook internally despite your licensing being compatible.

Would you please consider adding metadata.rb and tagging a new release?

Thank you!

COOK-4059: application_python's gunicorn.rb attempts to install requirements attached to django block as root, system-wide, outside of the virtualenv

When attempting to install a Django application using application_python, with a recipe based on the packaginator example, the recipe will install requirements listed in the django block as follows:

  • 1st: From django.rb, install the requirements inside django's Virtualenv, as the owner in the application block, using the GIT_SSH wrapper if one is specified
  • 2nd: From gunicorn.rb, install the requirements inside django block, as root, system-wide (outside of Virtualenv).

It appears that there is some code that attempts to pull in Virtualenv details from the django block if app_module == :django, but this doesn't seem to be used properly in providers/gunicorn.rb line 135-140.
Also, git may refuse to use the wrapper script as root if it is owned by a different application.owner.

This is a problem because it means:

  • Everything in the requirements.txt file is installed twice (once in the virtualenv and once system-wide)
  • The system-wide installation cannot pull from SSH-based git repositories.

This means that code stored in private repositories referenced from a requirements.txt on GitHub/Bitbucket/whatever (git+ssh) will fail to install and have an error of Permission denied (publickey).

As a workaround in my recipe, I've put in the gunicorn block reqirements false. This seems to prevent gunicorn.rb from trying to install any dependencies, or try to be smart by looking for requirements.txt at the root of the repository.

If this is all intended behaviour, then maybe the example for packaginator should be updated and this behaviour be documented.

COOK-4094 : python_pip resource(s) for packages have no support for pip options

python_pip has the 'options' attribute

This option is not catered to in the protected install_packages() in providers.rb ( https://github.com/poise/application_python/blob/master/providers/django.rb#L111 )

I cannot, therefore, install any pip packages as I am behind an HTTP proxy. And so I am dead in the water and not sure what the right/acceptable approach to the code would be.

Reported here: https://tickets.opscode.com/browse/COOK-4094

Chef 14 - #<NameError: instance variable @cwd not defined>

Issue opened after a talk with coderanger, here's my summary, workaround as comment on the issue:

With the rewrite of the execute resource in Chef 14, the following lines create an error:

remove_instance_variable(:@cwd)

Error as seen from chefspec test:

 expected no Exception, got #<NameError: instance variable @cwd not defined> with backtrace:
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application_python/files/halite_gem/poise_application_python/resources/python_execute.rb:43:in `remove_instance_variable'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application_python/files/halite_gem/poise_application_python/resources/python_execute.rb:43:in `initialize'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:140:in `block in declare_resource'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:121:in `declare_resource'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:175:in `public_send'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:175:in `block (2 levels) in _rewire_dsl!'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/app-python/recipes/default.rb:24:in `block in from_file'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/app-python/recipes/default.rb:21:in `from_file'
         # ./spec/unit/recipes/default_spec.rb:29:in `block (3 levels) in <top (required)>'
         # ./spec/unit/recipes/default_spec.rb:33:in `block (4 levels) in <top (required)>'
         # ./spec/unit/recipes/default_spec.rb:33:in `block (3 levels) in <top (required)>'
     # ./spec/unit/recipes/default_spec.rb:33:in `block (3 levels) in <top (required)>'

Seems like git is broken

Thoughts or ideas why I would get this?

Generated at 2016-06-30 19:26:02 +0000
Mixlib::ShellOut::ShellCommandFailed: git[https://github.com/moos3/captain-slackhook.git] (wiw_slackhook::default line 7) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '128'
---- Begin output of git ls-remote "" "HEAD" ----
STDOUT:
STDERR: fatal: No path specified. See 'man git-pull' for valid url syntax
---- End output of git ls-remote "" "HEAD" ----
Ran git ls-remote "" "HEAD" returned 128
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:289:in `invalid!'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:276:in `error!'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/shell_out.rb:56:in `shell_out!'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:286:in `git_ls_remote'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:244:in `remote_resolve_reference'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:230:in `target_revision'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:65:in `block (2 levels) in define_resource_requirements'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:231:in `call'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:231:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:321:in `block in run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:320:in `each'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:320:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider.rb:153:in `process_resource_requirements'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider.rb:132:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource.rb:596:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:69:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:97:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:97:in `each'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:97:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/resource_list.rb:84:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/resource_list.rb:82:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:96:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:667:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:662:in `catch'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:662:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:701:in `converge_and_save'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:281:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:285:in `block in fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:273:in `fork'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:273:in `fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:238:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:226:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:301:in `block in interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:290:in `loop'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:290:in `interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:269:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:58:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/bin/chef-solo:25:in `<top (required)>'

Can not install python v3

Error:

PoiseLanguages::Error
---------------------
Package python would install 2.7.11-1, which does not match 3.4.3. Please set the package_name or package_version provider options.

Code:

application "/srv/xyz" do
  only_if { app['deploy'] }

  git url do
    deploy_key deploy_key
  end
  python '3' do
    version '3.4.3'
  end
  virtualenv
  pip_requirements
  django 'app' do
  end
  gunicorn 'app' do
    port 8000
  end
end

Application environment not carried over to Django resource

I use an environment variable in my manage.py to determine a specific set of credentials to load. It appears that the environment attribute of the application resource isn't used by the django resource as it has no impact on the django python commands that are execute for collectstatic and migrate.

Here's the example recipe:

application release_dir do
  python '2.7'
  environment ({
    :SITE_CONFIG => deploy_bag.fetch('default_site_config', 'abc'),
  })
...
  django do
    path release_dir
    collectstatic true
    local_settings_path false
    migrate false
  end
end

I don't get a SITE_CONFIG environment variable when manage.py is run from these resources.

Application_python cookbook should take environment variables in file.

I am creating gunicorn service for flask application. My application needs a couple of env variables whose length is increasing 2048, which is systemd LINE_MAX.
If we pass environment variables in form of a file, then this issue can be solved.
This can be the enhancement in application_python to overcome line_max issue.

Possible fix for virtualenv references

I spent some time thinking about how to solve the virtualenv issues with gunicorn installing on the host vs the virtualenv associated with the (django) application.

If we modify the resource to support a symbol argument we can use that symbol to identify another resource in the application resource. As an example you could specify a virtualenv of :django to install to the same virtualenv as that resource.

Check my commit here:
BNOTIONS@a65a9e6

There is a way to give celery worker parameters?

Hi,

I'm using application_celery_worker and I need to set --concurrency parameter, but I didn't found any place talking about parameters to celery service.

There is a way to set this parameter to celery service?

Thanks!

unable to deploy python application

Hi,
I'm trying to install a python application from a private git repo.
According to application_python documentation, this is my recipe

application '/opt/sd-cif' do
  packages   ['git', 'swig', 'openssl-devel']
  owner      'centos'
  group      'centos'
  path       '/opt/sd-cif'
  git '/opt/sd-cif' do
    repository '.......'
    deploy_key 'ssh-rsa ..........'
    revision   'master'
  end  
  python_runtime 'sd-cif' do
  provider :scl
  version '3'
  end
  pip_requirements '/opt/sd-cif/requirements.txt'  
  python_virtualenv '/opt/sd-cif'    
end

Error is

[2015-09-30T17:14:02+00:00] DEBUG: Resources for generic application resource enabled on node include: [LWRP resource application from cookbook application]
[2015-09-30T17:14:02+00:00] DEBUG: Resource for application is LWRP resource application from cookbook application
[2015-09-30T17:14:02+00:00] DEBUG: Resources for generic git resource enabled on node include: [Chef::Resource::Git]
[2015-09-30T17:14:02+00:00] DEBUG: Resource for git is Chef::Resource::Git
[2015-09-30T17:14:02+00:00] DEBUG: filtered backtrace of compile error: /var/chef/cache/cookbooks/application_cif/recipes/default.rb:17:in `block (2 levels) in from_file',/var/chef/cache/cookbooks/application_cif/recipes/default.rb:15:in `block in from_file',/var/chef/cache/cookbooks/application_cif/recipes/default.rb:10:in `from_file'
[2015-09-30T17:14:02+00:00] DEBUG: backtrace entry for compile error: '/var/chef/cache/cookbooks/application_cif/recipes/default.rb:17:in `block (2 levels) in from_file''
[2015-09-30T17:14:02+00:00] DEBUG: Line number of compile error: '17'

================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/application_cif/recipes/default.rb
================================================================================

NoMethodError
-------------
undefined method `deploy_key' for Chef::Resource::Git

systemd service name collisions

On an Ubuntu 16.04 system, I observed that an application created via this cookbook will attempt to create multiple systemd services with the same name.

Am I misunderstanding the README by mixing a gunicorn and celery_worker resource in the same application resource?

Given the following cookbook code:

application '/opt/myapp' do
  git '[email protected]:myorg/myapp.git'
  pip_requirements
  python '2.7'
  gunicorn do
    port 9001
  end
  celery_worker do
   app_module 'app.tasks'
  end
end

this will happen:

* application_gunicorn[/opt/myapp] action enable
  * python_package[gunicorn] action install (up to date)
  * poise_service[/opt/myapp] action enable
    * execute[systemctl daemon-reload] action nothing (skipped due to action :nothing)
    * template[/etc/systemd/system/myapp.service] action create
      - create new file /etc/systemd/system/myapp.service
      - update content in file /etc/systemd/system/myapp.service from none to bd37a3
      --- /etc/systemd/system/myapp.service	2017-05-29 17:46:47.668598009 +0000
      +++ /etc/systemd/system/.chef-myapp20170529-24061-1i17040.service	2017-05-29 17:46:47.636598009 +0000
      @@ -1 +1,15 @@
      +[Unit]
      +Description=myapp
      +
      +[Service]
      +Environment=
      +ExecStart=/usr/bin/python2.7 -m gunicorn.app.wsgiapp --bind 0.0.0.0:9001 setup
      +ExecReload=/bin/kill -HUP $MAINPID
      +KillSignal=TERM
      +User=root
      +WorkingDirectory=/opt/myapp
      +Restart=on-failure
      +
      +[Install]
      +WantedBy=multi-user.target
      - change mode from '' to '0644'
      - change owner from '' to 'root'
      - change group from '' to 'root'
    * execute[systemctl daemon-reload] action run
      - execute systemctl daemon-reload
    * service[myapp] action enable
      - enable service service[myapp]
    * service[myapp] action start
      - start service service[myapp]
  

* application_celery_worker[/opt/myapp] action enable
  * poise_service[/opt/myapp] action enable
    * execute[systemctl daemon-reload] action nothing (skipped due to action :nothing)
    * template[/etc/systemd/system/myapp.service] action create
      - update content in file /etc/systemd/system/myapp.service from bd37a3 to a61b3d
      --- /etc/systemd/system/myapp.service	2017-05-29 17:46:47.636598009 +0000
      +++ /etc/systemd/system/.chef-myapp20170529-24061-1fov4ul.service	2017-05-29 17:46:50.100598009 +0000
      @@ -3,7 +3,7 @@

[Service]
Environment=
      -ExecStart=/usr/bin/python2.7 -m gunicorn.app.wsgiapp --bind 0.0.0.0:9001 setup
      +ExecStart=/usr/bin/python2.7 -m celery --app=app.tasks worker
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=TERM
User=root
    * execute[systemctl daemon-reload] action run
      - execute systemctl daemon-reload
    * service[myapp] action enable (up to date)
    * service[myapp] action start (up to date)

manage.py syncdb --noinput fails to run

during a chef run syncdb fails to run. this is the error, i cannot see what is causing this... any ideas ? cheers

Chef::Exceptions::Exec: deploy_revision[datagether](/var/chef/cache/cookbooks/application/providers/default.rb line 122) had an error: Chef::Exceptions::Exec: /home/wolfpack/datagether/shared/env/bin/python manage.py syncdb --noinput returned 1, expected 0

Sync/Migrate Multiple Databases?

I have an django application that uses multiple databases and requires multiple syncdb/migrate commands to be executed to sync all of the databases. Is there a possible work-around for this?

Environment vars not accessible by service

Hi guys!

I'm trying to fill systemd Environment variables from celery_worker. I saw it has ServiceMixin from poise-service interface but unfortunately I can't find a way to pass my env vars to service environment.

I'm trying to do like this:

application 'blabla' do
   environment ({
     :REMAP_SIGTERM => 'SIGQUIT'
   })

   celery_worker do
     app_module 'tasks'
     user 'celery'
     service_name "myapp"
   end
end

But my Environment entry on systemd service conf still empty.

Am I doing something wrong?

Thanks in advance!
Alisson

Understanding 4.0.0 'local_settings'

Having read the poise-developer-centric link to "Poise template property" I don't really understand how to use local_settings:

local_settings – A Poise template property for the content of the local settings configuration file.

Can someone shed some more light on the topic?

  NoMethodError
  -------------
  undefined method `local_settings' for PoiseApplicationPython::Resources::Django::Resource
   40:    django do
   41:      migrate false
   42>>     local_settings 'settings.py.erb'
   43:      debug true
   44:      database do
   45:        host 'hostdb'
   46:        database 'hostdb'
   47:        adapter  'mysql'
   48:        username chef_vault_item('secrets', 'hostdb')['username']
   49:        password chef_vault_item('secrets', 'hostdb')['password']
   50:        time_zone 'America/New_York'
   51:      end

gunicorn resource "config" option breaks things?

Does anyone have any thoughts how to debug this?

  gunicorn do
    app_module 'hostdb.wsgi'
    port 9999
    config "/root/hostdb.new/gunicorn_config.py"
  end

This results in the service failing due to too many respawns:

Jan 28 17:17:53 rcf-webapps init: hostdb.new main process (5075) terminated with status 1
Jan 28 17:17:53 rcf-webapps init: hostdb.new main process ended, respawning
...
Jan 28 17:17:55 rcf-webapps init: hostdb.new respawning too fast, stopped
# hostdb.new generated by poise-service for poise_service[/root/hostdb.new]

description "hostdb.new"

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 10 5
umask 022
chdir /root/hostdb.new
env DJANGO_SETTINGS_MODULE="hostdb.settings"
env PATH="/opt/rh/python27/root/usr/bin:/usr/lib/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin:/bin:/sbin:/usr/sbin:/usr/rcf-krb5/bin:/root/bin:/opt/chef/embedded/bin:/opt/chef/embedded/bin"
env LD_LIBRARY_PATH="/opt/rh/python27/root/usr/lib64"
env MANPATH="/opt/rh/python27/root/usr/share/man:"
env XDG_DATA_DIRS="/opt/rh/python27/root/usr/share"
env PKG_CONFIG_PATH="/opt/rh/python27/root/usr/lib64/pkgconfig"

exec /opt/chef/embedded/bin/ruby -e 'Process::UID.change_privilege("apache"); ENV["HOME"] = Dir.home("apache") rescue nil; exec(*["/root/hostdb.new/.virtualenv/bin/python", "-m", "gunicorn.app.wsgiapp", "--config", "/root/hostdb.new/gunicorn_config.py", "--bind", "0.0.0.0:9999", "hostdb.wsgi"])'

If I run gunicorn --config /root/hostdb.new/gunicorn_config.py --bind 0.0.0.0:9999 hostdb.wsgi by hand, it works.

If I comment out the 'config' property in the resource definition, everything works (but without the desired configuration as indicated in the gunicorn config file of course).

The config file contents are:

syslog = True
syslog_prefix = "hostdb_gunicorn"
syslog_facility = "daemon"
loglevel = "info"
errorlog = "/var/log/gunicorn_error.log"
accesslog = "/var/log/gunicorn_access.log"
worker_class = 'gevent'

I have to believe then that something is wrong with the generated upstart init file?

Allow private GIT repo in pip_requeriments

This is my application definition. All code is in a private GIT server, so I need to use application_git cookbook who provides deploy_key resource to checkout code successfully.

application  "/data/envs/app_name" do
   application_git "/data/envs/app_name" do
         repository  "[email protected]:atomic/app_name.git"
         revision 'develop'
         deploy_key key
   end
   virtualenv
   pip_requirements "/data/envs/app_name/requeriments/development.txt"
   django do
      database do
              engine 'mysql'
              username 'someuser'
              password 'somepassword'
              host     'localhost'
        end
   end
  gunicorn do
        app_module 'django'
        port 8100
        timeout 800
  end
end

So far, so good. The problem arise when pip_requirements finds this line into requeriments/development.txt, who points to an egg archive into another private GIT repo:

-e git+ssh://[email protected]/atomic/puzzlequestserver.git@develop#egg=pqs

In this case, the command fails because lack of permissions in the GIT repo

  Permission denied, please try again.
  Permission denied, please try again.
  Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  Command "git clone -q ssh://[email protected]/atomic/puzzlequestserver.git /data/envs/app_name/.virtualenv/src/pqs" failed with error code 128 in None
       ---- End output of ["/data/envs/app_name/.virtualenv/bin/python", "-m", "pip.__main__", "install", "--requirement", "/data/envs/app_name/requeriments/development.txt"] ----
       Ran ["/data/envs/app_name/.virtualenv/bin/python", "-m", "pip.__main__", "install", "--requirement", "/data/envs/app_name/requeriments/development.txt"] returned 1

How can I install this egg if cookbook don't provides this functionality ?

Cannot find a resource matching python_runtime

Hi, I'm getting a weird error trying to use application_python

Cannot find a resource matching python_runtime[myapp] (did you define it first?)

I'm using Chef: 12.7.2. Here is the relevant part of my recipe...

python_runtime 'myapp' do
  version '2.7'
  options :system, dev_package: false
end

application '/var/www/myapp' do
  python 'myapp'

  pip_requirements do
    path '/var/www/myapp/requirements_prod.txt'
    python 'myapp'
  end

  gunicorn do
    service_name "myapp"
    user app.user
    path "#{app.path}/myapp"
    config "/etc/gunicorn.d/myapp.py"
    bind '127.0.0.1'
    port 9009
  end

  django do
    manage_path '/var/www/myapp/gallery.py'
    debug node[:env] != "prod"
    secret_key app.secret

    database do
      engine 'django.db.backends.mysql'
      host db.host
      name db.name
      user db.user
      password db.password
    end

    migrate true
    collectstatic true
  end
end

CONTRIBUTING.md directs people with bugs to Opscode's JIRA instance

When attempting to report a bug following instructions in CONTRIBUTING.md, I recieved the following message:

Seth Vargo closed COOK-4059.
----------------------------

    Resolution: Duplicate

Ohai!

The new maintainer for this cookbook, [~coderanger], has moved the existing issues and Pull Requests to GitHub.

Unable to install local wheel package via "python_package" resource

Before requesting admin upload a new Python package to their repo I'm testing via local install. Installing from command line like so works fine:

source /home/fooapp/env/bin/activate && pip install /tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl

but when I attempt to do via python_package resource I have problems.

Input

# no support for source property so using absolute filename here
python_package /tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl do
  virtualenv conf['virtualenv']
end

Output

================================================================================
Error executing action `install` on resource 'python_package[/tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl]'
================================================================================

Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of ["/home/fooapp/env/bin/python", "-", "/tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl"] ----
STDOUT:
STDERR: /home/fooapp/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
 SNIMissingWarning
/home/fooapp/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
 InsecurePlatformWarning
No handlers could be found for logger "pip.index"
Traceback (most recent call last):
 File "<stdin>", line 38, in <module>
 File "/home/fooapp/env/lib/python2.7/site-packages/pip/index.py", line 491, in find_requirement
'No matching distribution found for %s' % req

I use this resource with remote packages all the time with no issues so assume I'm just misusing or otherwise misunderstanding. Appreciate the assistance.

How to add settings to local_settings.py?

In the old application_python django resource, you could specify your own settings.py.erb and pass in variables for it in additional_settings resource attribute. How would one do this with the new poise/application_python?

Is current master branch usable?

First, thank you for your work. I am new to chef and i am currently implementing a new cookbook to deploy our django based applications. As the last release is 3.0.0 and it fundamentally differs from the current implementation in the master branch i am wondering if i can use the version in master branch already. I tried to include it via the git repo but chef complains about the missing metadata.rb. So my question is: should i use the old version 3.0.0 or the current master branch? If i can use the master branch, how is that possible?

Thanks much for your help!
Jakob

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.