poise / poise-python Goto Github PK

A Chef cookbook to provide a unified interface for installing Python, managing Python packages, and creating virtualenvs.

License: Apache License 2.0

Ruby 100.00%

poise-python's Introduction

Poise-Python Cookbook

A Chef cookbook to provide a unified interface for installing Python, managing Python packages, and creating virtualenvs.

Quick Start

To install the latest available version of Python 2 and then use it to create a virtualenv and install some packages:

python_runtime '2'

python_virtualenv '/opt/myapp/.env'

python_package 'Django' do
  version '1.8'
end

pip_requirements '/opt/myapp/requirements.txt'

Installing a Package From a URI

While using python_package 'git+https://github.com/example/mypackage.git' will sometimes work, this approach is not recommended. Unfortunately pip's support for installing directly from URI sources is limited and cannot support the API used for the python_package resource. You can run the install either directly from the URI or through an intermediary git resource:

# Will re-install on every converge unless you add a not_if/only_if.
python_execute '-m pip install git+https://github.com/example/mypackage.git'

# Will only re-install when the git repository updates.
python_execute 'install mypackage' do
  action :nothing
  command '-m pip install .'
  cwd '/opt/mypackage'
end
git '/opt/mypackage' do
  repository 'https://github.com/example/mypackage.git'
  notifies :run, 'python_execute[install mypackage]', :immediately
end

Supported Python Versions

This cookbook can install at least Python 2.7, Python 3, and PyPy on all supported platforms (Debian, Ubuntu, RHEL, CentOS, Fedora).

Windows Support

The latest version of poise-python includes basic support for managing Python on Windows. This currently doesn't support Python 3.5, but everything should be working. Consider this support tested but experimental at this time.

Requirements

Chef 12.1 or newer is required.

Attributes

Attributes are used to configure the default recipe.

node['poise-python']['install_python2'] – Install a Python 2.x runtime. (default: true)
node['poise-python']['install_python3'] – Install a Python 3.x runtime. (default: false)
node['poise-python']['install_pypy'] – Install a PyPy runtime. (default: false)

Recipes

`default`

The default recipe installs Python 2, 3, and/or PyPy based on the node attributes. It is entirely optional and can be ignored in favor of direct use of the python_runtime resource.

Resources

`python_runtime`

The python_runtime resource installs a Python interpreter.

python_runtime '2'

Actions

:install – Install the Python interpreter. (default)
:uninstall – Uninstall the Python interpreter.

Properties

version – Version of Python to install. If a partial version is given, use the latest available version matching that prefix. (name property)
get_pip_url – URL to download the get-pip.py bootstrap script from. (default: https://bootstrap.pypa.io/get-pip.py)
pip_version – Version of pip to install. If set to true, use the latest. If set to false, do not install pip. For backward compatibility, can also be set to a URL instead of get_pip_url. (default: true)
setuptools_version – Version of Setuptools to install. If set to true, use the latest. If set to false, do not install Setuptools. (default: true)
virtualenv_version – Version of virtualenv to install. If set to true, use the latest. If set to false, do not install virtualenv. Will never be installed if the venv module is already available, such as on Python 3. (default: true)
wheel_version – Version of wheel to install. If set to true, use the latest. If set to false, do not install wheel.

Provider Options

The poise-python library offers an additional way to pass configuration information to the final provider called "options". Options are key/value pairs that are passed down to the python_runtime provider and can be used to control how it installs Python. These can be set in the python_runtime resource using the options method, in node attributes or via the python_runtime_options resource. The options from all sources are merged together in to a single hash.

When setting options in the resource you can either set them for all providers:

python_runtime 'myapp' do
  version '2.7'
  options pip_version: false
end

or for a single provider:

python_runtime 'myapp' do
  version '2.7'
  options :system, dev_package: false
end

Setting via node attributes is generally how an end-user or application cookbook will set options to customize installations in the library cookbooks they are using. You can set options for all installations or for a single runtime:

# Global, for all installations.
override['poise-python']['options']['pip_version'] = false
# Single installation.
override['poise-python']['myapp']['version'] = 'pypy'

The python_runtime_options resource is also available to set node attributes for a specific installation in a DSL-friendly way:

python_runtime_options 'myapp' do
  version '3'
end

Unlike resource attributes, provider options can be different for each provider. Not all providers support the same options so make sure to the check the documentation for each provider to see what options the use.

`python_runtime_options`

The python_runtime_options resource allows setting provider options in a DSL-friendly way. See the Provider Options section for more information about provider options overall.

python_runtime_options 'myapp' do
  version '3'
end

Actions

:run – Apply the provider options. (default)

Properties

resource – Name of the python_runtime resource. (name property)
for_provider – Provider to set options for.

All other attribute keys will be used as options data.

`python_execute`

The python_execute resource executes a Python script using the configured runtime.

python_execute 'myapp.py' do
  user 'myuser'
end

This uses the built-in execute resource and supports all the same properties.

Actions

:run – Execute the script. (default)

Properties

command – Script and arguments to run. Must not include the python. (name attribute)
python – Name of the python_runtime resource to use. If not specified, the most recently declared python_runtime will be used. Can also be set to the full path to a python binary.
virtualenv – Name of the python_virtualenv resource to use. This is mutually exclusive with the python property.

For other properties see the Chef documentation.

`python_package`

The python_package resource installs Python packages using pip.

python_package 'Django' do
  version '1.8'
end

This uses the built-in package resource and supports the same actions and properties. Multi-package installs are supported using the standard syntax.

Actions

:install – Install the package. (default)
:upgrade – Install using the --upgrade flag.
:remove – Uninstall the package.

The :purge and :reconfigure actions are not supported.

Properties

group – System group to install the package.
package_name – Package or packages to install. (name property)
version – Version or versions to install.
python – Name of the python_runtime resource to use. If not specified, the most recently declared python_runtime will be used. Can also be set to the full path to a python binary.
user – System user to install the package.
virtualenv – Name of the python_virtualenv resource to use. This is mutually exclusive with the python property.
options – Options to pass to pip.
install_options – Options to pass to pip install (and similar commands).
list_options – Options to pass to pip list (and similar commands).

For other properties see the Chef documentation. The response_file, response_file_variables, and source properties are not supported.

`python_virtualenv`

The python_virtualenv resource creates Python virtual environments.

python_virtualenv '/opt/myapp'

This will use the venv module if available, or virtualenv otherwise.

Actions

:create – Create the virtual environment. (default)
:delete – Delete the virtual environment.

Properties

group – System group to create the virtualenv.
path – Path to create the environment at. (name property)
pip_version – Version of pip to install. If set to true, use the latest. If set to false, do not install pip. Can also be set to a URL to a copy of the get-pip.py script. (default: true)
python – Name of the python_runtime resource to use. If not specified, the most recently declared python_runtime will be used. Can also be set to the full path to a python binary.
setuptools_version – Version of Setuptools to install. If set to true, use the latest. If set to false, do not install Setuptools. (default: true)
system_site_packages – Enable or disable visibilty of system packages in the environment. (default: false)
user – System user to create the virtualenv.
wheel_version – Version of wheel to install. If set to true, use the latest. If set to false, do not install wheel.

`pip_requirements`

The pip_requirements resource installs packages based on a requirements.txt file.

pip_requirements '/opt/myapp/requirements.txt'

The underlying pip install command will run on every converge, but notifications will only be triggered if a package is actually installed.

Actions

:install – Install the requirements. (default)
:upgrade – Install using the --upgrade flag.

Properties

path – Path to the requirements file, or a folder containing the requirements file. (name property)
cwd – Directory to run pip from. (default: directory containing the requirements.txt)
group – System group to install the packages.
options – Command line options for use with pip install.
python – Name of the python_runtime resource to use. If not specified, the most recently declared python_runtime will be used. Can also be set to the full path to a python binary.
user – System user to install the packages.
virtualenv – Name of the python_virtualenv resource to use. This is mutually exclusive with the python property.

Python Providers

Common Options

These provider options are supported by all providers.

pip_version – Override the pip version.
setuptools_version – Override the Setuptools version.
version – Override the Python version.
virtualenv_version – Override the virtualenv version.
wheel_version – Override the wheel version.

`system`

The system provider installs Python using system packages. This is currently only tested on platforms using apt-get and yum (Debian, Ubuntu, RHEL, CentOS Amazon Linux, and Fedora) and is a default provider on those platforms. It may work on other platforms but is untested.

python_runtime 'myapp' do
  provider :system
  version '2.7'
end

Options

dev_package – Install the package with the headers and other development files. Can be set to a string to select the dev package specifically. (default: true)
package_name – Override auto-detection of the package name.
package_upgrade – Install using action :upgrade. (default: false)
package_version – Override auto-detection of the package version.

`scl`

The scl provider installs Python using the Software Collections packages. This is only available on RHEL, CentOS, and Fedora. SCL offers more recent versions of Python than the system packages for the most part. If an SCL package exists for the requested version, it will be used in preference to the system provider.

python_runtime 'myapp' do
  provider :scl
  version '3.4'
end

`portable_pypy`

The portable_pypy provider installs Python using the Portable PyPy packages. These are only available for Linux, but should work on any Linux OS.

python_runtime 'myapp' do
  provider :portable_pypy
  version 'pypy'
end

`portable_pypy3`

The portable_pypy3 provider installs Python 3 using the Portable PyPy packages. These are only available for Linux, but should work on any Linux OS.

python_runtime 'myapp' do
  provider :portable_pypy3
  version 'pypy3'
end

Options

folder – Folder to install PyPy in. (default: /opt/)
url – URL to download the package from. (default: automatic)

`deadsnakes`

Coming soon!

`python-build`

Coming soon!

Upgrading from the `python` Cookbook

The older python cookbook is not directly compatible with this one, but the broad strokes overlap well. The python::default recipe is roughly equivalent to the poise-python::default recipe. The python::pip and python::virtualenv recipes are no longer needed as installing those things is now part of the python_runtime resource. The python::package recipe corresponds with the system provider for the python_runtime resource, and can generally be replaced with poise-python::default. At this time there is no provider to install from source so there is no replacement for the python::source recipe, however this is planned for the future via a python-build provider.

The python_pip resource can be replaced with python_package, though the environment property has been removed. The python_virtualenv resource can remain unchanged except for the interpreter property now being python and the options property has been removed.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

poise-python's People

Contributors

Stargazers

Watchers

Forkers

marklcl ravenb technicalabuse jblaine beezzwax h4ck3rm1k3 jrake-revelant pombredanne kisoku boxidau asavoy rafael2 pidydx philipdouglas aarranz ldesiqueira zhufenggnss florent-babylon cmastro24 pecigonzalo khaibnd dixis maxnasonov mehedikamal symondsandson maiee004 reliablepenguin dylancanfield biinari waltercool edwlarkey atheiman rudymccomb capellini dimsh99 binary-devops sujithktkm sauraus nurus jharrington22 mdbdba john-bodley nickfitz sbcas grizo espoelstra isuftin signe whitemoondreamsinc sselva757 thekatargit coreyjonoliver afcarl kommado ndpgroup ideaship hartmantis techyshishy twistle shareaholic rondineli thetradedesk gena01 zeitonline scalp42 forksbot rapid7 itjamie jesbourne pescetto mattcl zr2d2 damm dcroswell-vfa hieunba greenlogistics jjustice6 maxwell-gregory yuanrongzheng parityinc criteo-forks stsully evergage scottvidmar jamesbjackson jpmat296 sangeethavisa monsfaujdar bdwyertech autodevsec zendesk zyelabs iniiio jackzaharoff yongju-hong

poise-python's Issues

Python location troubles...

I'm using a Centos 6.6 instance provided to me with Python 2.7.6 already installed. I call it by executing python2.7. Calling which python2.7 shows it is in a directory like /opt/somDir/python/bin. The default Python 2.6.6 is also available by calling python

When I use the python_runtime resource to get the latest version of Python, it appears to install to /opt/rh/python27/root/usr/bin. However, calling python or python2.7 still uses the older versions.

I looked at the documentation for the Software Collections package that is used, and it says to call scl enable python27 bash. After I do that, calling which python shows /opt/rh/python27/root/usr/bin

I'm curious if the python_runtime resource does this by default, or if maybe due to my environment, I need to call it manually?

python_package fails with a user specified

python_package 'hipsaint' do
  user  node[:nagios][:user]
  group node[:nagios][:group]
end

results in:

STDERR: The directory '/root/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/root/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Seems like $HOME needs to be set.

Not sure if #12 will fix it.

python_virtualenv doesn't install pip

python_virtualenv with system_site_packages true is not behaving like virtualenv --system-site-packages

  python_runtime '2' do
    provider :system
  end

  python_virtualenv '/tmp/noodle' do
    user 'jblaine'
    system_site_packages true
  end

* python_virtualenv[/tmp/noodle] action create
  - Creating virtualenv at /tmp/noodle
  * python_runtime_pip[/tmp/noodle] action install (up to date)
  * python_package[setuptools] action install (up to date)
  * python_package[wheel] action install (up to date)

tmp:etc-metrics# ls noodle/bin
activate  activate.csh  activate.fish  activate_this.py  python  python2  python2.7  python-config
tmp:etc-metrics#

Contrast with:

tmp:etc-metrics# virtualenv without-poise --system-site-packages
New python executable in /tmp/without-poise/bin/python
Installing setuptools, pip, wheel...done.
tmp:etc-metrics# ls without-poise/bin
activate      activate.fish     easy_install      pip   pip2.7  python2    python-config
activate.csh  activate_this.py  easy_install-2.7  pip2  python  python2.7  wheel
tmp:etc-metrics# rm -rf without-poise/

python_runtime resource fails to install setuptools

default.rb:

python_runtime '2' do
  action :install
  version '2.7'
  pip_version true
end

metadata.rb:

name 'elden-test'
maintainer 'Elden Seropian'
maintainer_email '[email protected]'
license 'all_rights'
description 'dummy test cookbook'

version '0.1.2'

depends 'poise-python'

.kitchen.yml:


---
driver:
  name: vagrant
  customize:
      memory: 1024

provisioner:
  name: chef_zero

platforms:
  - name: ubuntu-14.04

suites:
  - name: default
    run_list:
      - recipe[elden-test::default]

I'm attaching the entirety of the kitchen converge default -l debug output. Here are the most relevant bits:

[2016-01-20T20:32:28+00:00] DEBUG: [python_runtime[2]] Installing setuptools latest
[2016-01-20T20:32:28+00:00] DEBUG: [python_runtime[2]] Adding python_package[setuptools] to subresources
[2016-01-20T20:32:28+00:00] DEBUG: [python_runtime[2]] Installing setuptools latest
 * python_package[setuptools] action install[2016-01-20T20:32:36+00:00] INFO: Processing python_package[setuptools] action install (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 136)
       [2016-01-20T20:32:36+00:00] DEBUG: Provider for action install on resource python_package[setuptools] is PoisePython::Resources::PythonPackage::Provider
[2016-01-20T20:32:36+00:00] DEBUG: [python_package[setuptools]] Running python command: /usr/bin/python2.7 -m pip.__main__ list
       apt-xapian-index (0.45)
       chardet (2.0.1)
       pip (8.0.0)
       python-apt (0.9.3.5ubuntu1)
       python-debian (0.1.21-nmu2ubuntu2)
       requests (2.2.1)
       six (1.5.2)
       ssh-import-id (3.21)
       urllib3 (1.7.1)
       The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       /usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
         SNIMissingWarning
       /usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
         InsecurePlatformWarning
  [2016-01-20T20:32:42+00:00] DEBUG: [python_package[setuptools]] Running python command: /usr/bin/python2.7 - list --outdated setuptools
       The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       /usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
         SNIMissingWarning
       /usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
         InsecurePlatformWarning
       setuptools (0) - Latest: 19.4 [wheel]
       The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       /usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
         InsecurePlatformWarning
       [2016-01-20T20:32:47+00:00] DEBUG: [python_package[setuptools]] Unparsable line in pip outdated: setuptools (0) - Latest: 19.4 [wheel]


           ================================================================================
           Error executing action `install` on resource 'python_package[setuptools]'
           ================================================================================

           Chef::Exceptions::Package
           -------------------------
           No candidate version available for setuptools

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb

           136:         python_package 'setuptools' do
           137:           parent_python new_resource
           138:           version setuptools_version if setuptools_version.is_a?(String)

           140:       end

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:136:in `install_setuptools'

           python_package("setuptools") do
         action [:install]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             package_name "setuptools"
             timeout 900
             declared_type :python_package
             cookbook_name "elden-test"
             parent_python python_runtime[2]
           end



         ================================================================================
         Error executing action `install` on resource 'python_runtime[2]'
         ================================================================================

         Chef::Exceptions::Package
         -------------------------
         python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 136) had an error: Chef::Exceptions::Package: No candidate version available for setuptools

         Cookbook Trace:
         ---------------
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

         Resource Declaration:
       ---------------------
         # In /tmp/kitchen/cache/cookbooks/elden-test/recipes/default.rb

           1: python_runtime '2' do
           2:   action :install
           3:   version '2.7'
           4:   pip_version true
           5: end

         Compiled Resource:
         ------------------
       # Declared in /tmp/kitchen/cache/cookbooks/elden-test/recipes/default.rb:1:in `from_file'

         python_runtime("2") do
           action [:install]
           updated true
           updated_by_last_action true
           retries 0
           retry_delay 2
           default_guard_interpreter :default
           subresources [python_runtime_pip[2], python_package[setuptools], python_package[wheel], python_package[virtualenv]]
           declared_type :python_runtime
           cookbook_name "elden-test"
           recipe_name "default"
           version "2.7"
         pip_version true
           setuptools_version true
           virtualenv_version true
           wheel_version true
         end


       [2016-01-20T20:32:47+00:00] DEBUG: Re-raising exception: Chef::Exceptions::Package - python_runtime[2] (elden-test::default line 1) had an error: Chef::Exceptions::Package: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 136) had an error: Chef::Exceptions::Package: No candidate version available for setuptools
  [2016-01-20T20:32:47+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2016-01-20T20:32:47+00:00] DEBUG: Chef::Exceptions::Package: python_runtime[2] (elden-test::default line 1) had an error: Chef::Exceptions::Package: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 136) had an error: Chef::Exceptions::Package: No candidate version available for setuptools
       [2016-01-20T20:32:47+00:00] ERROR: python_runtime[2] (elden-test::default line 1) had an error: Chef::Exceptions::Package: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 136) had an error: Chef::Exceptions::Package: No candidate version available for setuptools
       [2016-01-20T20:32:48+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
D      Cleaning up local sandbox in /var/folders/dm/ppd0vgp93r91yrry993hhm5r0000gp/T/default-ubuntu-1404-sandbox-20160120-36762-1k8ttn1
>>>>>> Converge failed on instance <default-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

sudo -E /opt/chef/bin/chef-client --local-mode --config /tmp/kitchen/client.rb --log_level debug --force-formatter --no-color --json-attributes /tmp/kitchen/dna.json --chef-zero-port 8889
']
>>>>>> ----------------------
D      Converge failed on instance <default-ubuntu-1404>.
D      ------Exception-------
D      Class: Kitchen::InstanceFailure
D      Message: Converge failed on instance <default-ubuntu-1404>.  Please see .kitchen/logs/default-ubuntu-1404.log for more details
D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: SSH exited (1) for command: [sh -c '

We encountered this when trying to move our code over from using the deprecated python cookbook to poise-python. Please let me know if you need any more information!

converge_output.txt

Error executing action `install` on resource 'python_runtime[2]'

strongly suspecting new versions to bring up this issue; ruby-2.3.0 running chef-12.6:

         Error executing action `install` on resource 'python_runtime[2]'
         ================================================================================

         NoMethodError
         -------------
         NoMethodError

         Cookbook Trace:
         ---------------
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:125:in `response_file_variables'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lwrp_polyfill.rb:34:in `initialize'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/resource_name.rb:32:in `initialize'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:105:in `initialize'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:136:in `install_setuptools'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:51:in `block in action_install'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `instance_eval'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `subcontext_block'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:67:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

         Resource Declaration:
         ---------------------
         # In /tmp/kitchen/cache/cookbooks/poise-python/recipes/default.rb

          20: python_runtime '2' if node['poise-python']['install_python2']

         Compiled Resource:
         ------------------
         # Declared in /tmp/kitchen/cache/cookbooks/poise-python/recipes/default.rb:20:in `from_file'

         python_runtime("2") do
           action [:install]
           retries 0
           retry_delay 2
           default_guard_interpreter :default
           subresources [python_runtime_pip[2]]
           declared_type :python_runtime
           cookbook_name "poise-python"
           recipe_name "default"
           pip_version true
           setuptools_version true
           version "2"
           virtualenv_version true
           wheel_version true
         end

poise_shell_out is only available from poise 2.5.0

poise-python 1.2.1 requires poise_shell_out which has been added to poise 2.5.0.
However the dependencies on poise-pyton are ~> 2.0 https://supermarket.chef.io/cookbooks/poise-python#dependencies. not ~> 2.5

I've upgraded poise-python but got an error because poise_shell_out was not there.

https://github.com/poise/poise-python/blob/v1.2.1/lib/poise_python/resources/python_runtime_pip.rb#L119

scl provider on RHEL inconsistencies

Hi there,

So I'm having some real problems getting a consistent python 2.7 install on RHEL 6.6.

This is how I'm installing it in my recipe:

python_runtime 'vagrant' do
  provider :scl
  version '2.7'
end

Most of the time it fails with this error during provisioning (I'm using the cookbook with vagrant+chef):

==> vagrant-slave-2: Mixlib::ShellOut::ShellCommandFailed
==> vagrant-slave-2: ------------------------------------
==> vagrant-slave-2: rpm_package[rhscl-python27] (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/resource.rb line 112) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
==> vagrant-slave-2: 
==> vagrant-slave-2: ---- Begin output of rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
==> vagrant-slave-2: 
==> vagrant-slave-2: ' https://www.softwarecollections.org/en/scls/rhscl/python27/epel-6-x86_64/download/rhscl-python27-epel-6-x86_64.noarch.rpm ----
==> vagrant-slave-2: 
==> vagrant-slave-2: STDOUT: 
==> vagrant-slave-2: 
==> vagrant-slave-2: STDERR: curl: (7) couldn't connect to host
==> vagrant-slave-2: 
==> vagrant-slave-2: error: open of https://www.softwarecollections.org/en/scls/rhscl/python27/epel-6-x86_64/download/rhscl-python27-epel-6-x86_64.noarch.rpm failed: No such file or directory
==> vagrant-slave-2: 
==> vagrant-slave-2: ---- End output of rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
==> vagrant-slave-2: 
==> vagrant-slave-2: ' https://www.softwarecollections.org/en/scls/rhscl/python27/epel-6-x86_64/download/rhscl-python27-epel-6-x86_64.noarch.rpm ----
==> vagrant-slave-2: 
==> vagrant-slave-2: Ran rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
==> vagrant-slave-2: 
==> vagrant-slave-2: ' https://www.softwarecollections.org/en/scls/rhscl/python27/epel-6-x86_64/download/rhscl-python27-epel-6-x86_64.noarch.rpm returned 1

I'm not sure exactly what it's doing here? It looks like it's running an rpm command which results in a curl call? And this curl call is failing (Trying it manually results in an empty file?).

However if I run the exact same chef cookbook again a 2nd time, it goes through fine and seems to install python from the repository:

==> vagrant-slave-2: [2015-11-13T12:57:32-05:00] INFO: rpm_package[rhscl-python27] installed rhscl-python27 at 1-2
==> vagrant-slave-2: [2015-11-13T12:57:32-05:00] INFO: ruby_block[flush_yum_cache] called
==> vagrant-slave-2: [2015-11-13T12:57:35-05:00] INFO: yum_package[python27] installing python27-1.1-17.el6 from rhscl-python27-epel-6-x86_64 repository
==> vagrant-slave-2: [2015-11-13T12:58:31-05:00] INFO: yum_package[python27] installed python27 at 1.1-17.el6

I've been trying to troubleshoot the first failure and I have no idea what's causing it, or why the 2nd run would work fine. Any ideas?

Thanks!

python_execute with virtualenv specified can't see pip it just installed

I can't win this week :(

python_runtime '2' do
  provider :system
end

python_virtualenv node['r701-graphite']['server']['approot'] do
  user node['r701-graphite']['server']['user']
end

python_package 'whisper' do
  version node['r701-graphite']['server']['whisper_version']
end

python_package 'carbon' do
  version node['r701-graphite']['server']['carbon_version']
end

# graphite-web 0.9.15 package has a bug:
# https://github.com/graphite-project/graphite-web/issues/1508
# led to me using options with python_package
# which led to:
# https://github.com/poise/poise-python/issues/53
# which led to this code being commented out :(
# python_package 'graphite-web' do
#   options '--install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp"'
#   version node['r701-graphite']['server']['graphite_web_version']
# end

python_execute 'Install graphite-web a specific way' do
  command "pip install graphite-web==#{node['r701-graphite']['server']['graphite_web_version']} --install-option='--prefix=/opt/graphite' --install-option='--install-lib=/opt/graphite/webapp'"
  virtualenv node['r701-graphite']['server']['approot']
end

* python_runtime_pip[2] action install (up to date)
* python_package[setuptools] action install (up to date)
* python_package[wheel] action install (up to date)
* python_package[virtualenv] action install (up to date)
 (up to date)
* python_virtualenv[/etc-metrics.our.org/graphite] action create
  * python_runtime_pip[/etc-metrics.our.org/graphite] action install (up to date)
  * python_package[setuptools] action install (up to date)
  * python_package[wheel] action install (up to date)
   (up to date)
* python_package[whisper] action install (up to date)
* python_package[carbon] action install (up to date)
* python_execute[Install graphite-web a specific way] action run
  [execute] /etc-metrics.our.org/graphite/bin/python: can't open file 'pip': [Errno 2] No such file or directory
...
  Mixlib::ShellOut::ShellCommandFailed
  ------------------------------------
  Expected process to exit with [0], but received '2'
  ---- Begin output of /etc-metrics.our.org/graphite/bin/python pip install graphite-web==0.9.15 --install-option='--prefix=/opt/graphite' --install-option='--install-lib=/opt/graphite/webapp' ----
  STDOUT:
  STDERR: /etc-metrics.our.org/graphite/bin/python: can't open file 'pip': [Errno 2] No such file or directory
  ---- End output of /etc-metrics.our.org/graphite/bin/python pip install graphite-web==0.9.15 --install-option='--prefix=/opt/graphite' --install-option='--install-lib=/opt/graphite/webapp' ----
  Ran /etc-metrics.our.org/graphite/bin/python pip install graphite-web==0.9.15 --install-option='--prefix=/opt/graphite' --install-option='--install-lib=/opt/graphite/webapp' returned 2

Incorrect linker arguments for python_package

See chef/chef#4705

Somehow we are trying to link against Chef's libs, which is incorrect.

System provider fails on OS X

Recipe: poise-python::default
  * python_runtime[2] action install[2015-11-16T08:40:42-08:00] INFO: Processing python_runtime[2] action install (poise-python::default line 20)

    * poise_languages_system[python] action install[2015-11-16T08:40:42-08:00] INFO: Processing poise_languages_system[python] action install (/Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32)


      ================================================================================
      Error executing action `install` on resource 'poise_languages_system[python]'
      ================================================================================

      TypeError
      ---------
      no implicit conversion of Array into String

      Cookbook Trace:
      ---------------
      /Users/jacob/sprout-wrap/cookbooks/homebrew/libraries/homebrew_package.rb:97:in `get_version_from_formula'
      /Users/jacob/sprout-wrap/cookbooks/homebrew/libraries/homebrew_package.rb:74:in `current_installed_version'
      /Users/jacob/sprout-wrap/cookbooks/homebrew/libraries/homebrew_package.rb:44:in `load_current_resource'
      /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:189:in `block (2 levels) in patch_load_current_resource!'
      /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:172:in `block in run_package_action'
      /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `each'
      /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `run_package_action'
      /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:100:in `action_install'
      /Users/jacob/sprout-wrap/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
      /Users/jacob/sprout-wrap/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

      Resource Declaration:
      ---------------------
      # In /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb

       32:         poise_languages_system options['package_name'] || system_package_name do
       33:           # Otherwise use the default install action.
       34:           action(:upgrade) if options['package_upgrade']
       35:           parent new_resource
       36:           # Don't pass true because we want the default computed behavior for that.
       37:           dev_package options['dev_package'] unless options['dev_package'] == true
       38:           dev_package_overrides dev_package_overrides
       39:           package_version options['package_version'] if options['package_version']
       40:           version options['version']
       41:         end
       42:       end

      Compiled Resource:
      ------------------
      # Declared in /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb:32:in `install_system_packages'

      poise_languages_system("python") do
        action [:install]
        retries 0
        retry_delay 2
        default_guard_interpreter :default
        declared_type :poise_languages_system
        cookbook_name :"poise-python"
        parent # Declared in /Users/jacob/sprout-wrap/cookbooks/poise-python/recipes/default.rb:20:in `from_file'

      python_runtime("2") do
        action [:install]
        retries 0
        retry_delay 2
        default_guard_interpreter :default
        declared_type :python_runtime
        cookbook_name :"poise-python"
        recipe_name "default"
        pip_version true
        setuptools_version true
        version "2"
        virtualenv_version true
        wheel_version true
      end

        version "2"
        package_name "python"
      end

[2015-11-16T08:40:42-08:00] INFO: Running queued delayed notifications before re-raising exception

    ================================================================================
    Error executing action `install` on resource 'python_runtime[2]'
    ================================================================================

    TypeError
    ---------
    poise_languages_system[python] (/Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32) had an error: TypeError: no implicit conversion of Array into String

    Cookbook Trace:
    ---------------
    /Users/jacob/sprout-wrap/cookbooks/homebrew/libraries/homebrew_package.rb:97:in `get_version_from_formula'
    /Users/jacob/sprout-wrap/cookbooks/homebrew/libraries/homebrew_package.rb:74:in `current_installed_version'
    /Users/jacob/sprout-wrap/cookbooks/homebrew/libraries/homebrew_package.rb:44:in `load_current_resource'
    /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:189:in `block (2 levels) in patch_load_current_resource!'
    /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:172:in `block in run_package_action'
    /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `each'
    /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `run_package_action'
    /Users/jacob/sprout-wrap/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:100:in `action_install'
    /Users/jacob/sprout-wrap/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
    /Users/jacob/sprout-wrap/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

    Resource Declaration:
    ---------------------
    # In /Users/jacob/sprout-wrap/cookbooks/poise-python/recipes/default.rb

     20: python_runtime '2' if node['poise-python']['install_python2']

    Compiled Resource:
    ------------------
    # Declared in /Users/jacob/sprout-wrap/cookbooks/poise-python/recipes/default.rb:20:in `from_file'

    python_runtime("2") do
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      declared_type :python_runtime
      cookbook_name :"poise-python"
      recipe_name "default"
      pip_version true
      setuptools_version true
      version "2"
      virtualenv_version true
      wheel_version true
    end

[2015-11-16T08:40:42-08:00] INFO: Running queued delayed notifications before re-raising exception

System pip packages

Is there any way to install packages with pip for system python other than having to specify the path? I have virutalenv_version set to false, and the only declared python_runtime is using the :system option, but all pip packages are installed with the opt/rh/python27 binary.

Python 3.3 venv does not support --without-pip

Python 3.3 does have venv but support for --without-pip was not added until 3.4. Currently it seems that it is not going to work to create virtual environments for python 3.3 using this cookbook.

Package names like 'Django==1.5.12' not supported

Is this syntax for packages supported (and therefore something is going unintentionally wrong), or do I need to break the version information out into a different attribute and use a version property on the resource?

[2016-01-07T12:53:00-05:00] DEBUG: Resource for python_runtime is PoisePython::Resources::PythonRuntime::Resource
[2016-01-07T12:53:00-05:00] DEBUG: [application_python_package[django==1.5.12]] Running python command: /root/hostdb.new/.virtualenv/bin/python - list --outdated django\=\=1.5.12
django (Current: 0 Latest: 1.5.12)

================================================================================
Error executing action `install` on resource 'application_python_package[django==1.5.12]'
================================================================================

Chef::Exceptions::Package
-------------------------
No candidate version available for django==1.5.12

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/r701-hostdb/recipes/server.rb

 49:   python_package node['r701-hostdb']['django']['packages']
 50:

Fails when the package name is not the install directory for system provider.

When using the system provider to install the python34 package from the EPEL repo on Centos7 the cookbook fails. The system provider assumes that the binary will install with the same name as the package name which is incorrect in this case as the python34 package will install the binary to '/usr/bin/python3.4' while the cookbook expects to find it at /usr/bin/python34. Could an option be added to specify the expected binary?

All the SoftwareCollections URLs have broke

poise-python is no longer able to install versions of python for CentOS 7, and probably many others. The error traces back to a 404 with the SoftwareCollections URL.

Here's some relevant error text:

146.148.49.248         Mixlib::ShellOut::ShellCommandFailed
146.148.49.248         ------------------------------------
146.148.49.248         Expected process to exit with [0], but received '1'
146.148.49.248         ---- Begin output of rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
146.148.49.248         ' https://www.softwarecollections.org/en/scls/rhscl/python27/epel-7-x86_64/download/rhscl-python27-epel-7-x86_64.noarch.rpm ----
146.148.49.248         STDOUT:
146.148.49.248         STDERR: curl: (22) The requested URL returned error: 404 Not Found
146.148.49.248         error: open of https://www.softwarecollections.org/en/scls/rhscl/python27/epel-7-x86_64/download/rhscl-python27-epel-7-x86_64.noarch.rpm failed: No such file or directory
146.148.49.248         ---- End output of rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
146.148.49.248         ' https://www.softwarecollections.org/en/scls/rhscl/python27/epel-7-x86_64/download/rhscl-python27-epel-7-x86_64.noarch.rpm ----
146.148.49.248         Ran rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
146.148.49.248         ' https://www.softwarecollections.org/en/scls/rhscl/python27/epel-7-x86_64/download/rhscl-python27-epel-7-x86_64.noarch.rpm returned 1

Visiting that URL indeed shows a missing file. It seems that the URL format has changed to https://www.softwarecollections.org/repos/rhscl/rh-python34/rhscl-rh-python34-el7-epel-7-x86_64/noarch/rhscl-rh-python34-epel-7-x86_64-1-2.noarch.rpm

This change happened at 2015-12-04 12:13. Maybe it would be best to not rely on these hard coded URLs by softwarecollections so a breaking change doesn't break this cookbook in the future.

Package fails to install when options are specified

Whenever I add an options property to my python_package resource, it fails. For example, my resource looks similar to:

python_package 'tinydb' do
  version '2.4'
  virtualenv my_venv_dir
  options '--extra-index-url http://www.example.com/api/pypi/my-repo/simple'
end

Even a '-v' option fails.

[2015-10-07T14:55:54-04:00] DEBUG: [python_package[tinydb]] Running python command: /opt/my-venv/bin/python #- list -v --outdated tinydb\=\=2.4

Usage:
  pip <command> [options]

Commands:
  install                     Install packages.
...<snip>...
[2015-10-07T14:55:55-04:00] DEBUG: [python_package[tinydb]] Unparsable line in pip outdated:
[2015-10-07T14:55:55-04:00] DEBUG: [python_package[tinydb]] Unparsable line in pip outdated: Usage:
[2015-10-07T14:55:55-04:00] DEBUG: [python_package[tinydb]] Unparsable line in pip outdated:   pip <command> [options]
[2015-10-07T14:55:55-04:00] DEBUG: [python_package[tinydb]] Unparsable line in pip outdated:
[2015-10-07T14:55:55-04:00] DEBUG: [python_package[tinydb]] Unparsable line in pip outdated: Commands:
[2015-10-07T14:55:55-04:00] DEBUG: [python_package[tinydb]] Unparsable line in pip outdated:   install                     Install packages.
...<snip>...

    * No candidate version available for tinydb
    ================================================================================
    Error executing action `install` on resource 'python_package[tinydb]'
    ================================================================================

    Chef::Exceptions::Package
    -------------------------
    No candidate version available for tinydb

    Resource Declaration:
    ---------------------
    # In /home/me/chef-repo/cookbooks/my-cookbook/recipes/default.rb

     34: python_package 'tinydb' do
     35:   version '2.4'
     36:   virtualenv my_venv_dir
     37:   options '-v'
     38: end
     39:

    Compiled Resource:
    ------------------
    # Declared in /home/me/chef-repo/cookbooks/my-cookbook/recipes/default.rb:34:in `from_file'

    python_package("tinydb") do
      action :install
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      options "-v"
      package_name "tinydb"
      version "2.4"
      timeout 900
      declared_type :python_package
      cookbook_name :"my-cookbook"
      recipe_name "default"
      parent_python python_virtualenv[/opt/my-venv]
    end

[2015-10-07T14:55:55-04:00] INFO: Running queued delayed notifications before re-raising exception
[2015-10-07T14:55:55-04:00] DEBUG: Re-raising exception: Chef::Exceptions::Package - python_package[tinydb] (my-cookbook::default line 34) had an error: Chef::Exceptions::Package: No candidate version available for tinydb
/opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:241:in `run'
  /opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:322:in `block in run'
  /opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:321:in `each'
  /opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:321:in `run'
  /opt/chefdk/embedded/apps/chef/lib/chef/provider.rb:144:in `process_resource_requirements'
  /opt/chefdk/embedded/apps/chef/lib/chef/provider.rb:120:in `run_action'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource.rb:562:in `run_action'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:339:in `block in converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `catch'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:353:in `converge_and_save'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:457:in `run'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:271:in `block in fork_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:225:in `block in run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:213:in `run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:280:in `block in interval_run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `loop'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `interval_run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:247:in `run_application'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:60:in `run'
  /opt/chefdk/embedded/apps/chef/bin/chef-solo:25:in `<top (required)>'
  /opt/chefdk/bin/chef-solo:51:in `load'
  /opt/chefdk/bin/chef-solo:51:in `<main>'

Running handlers:
[2015-10-07T14:55:55-04:00] ERROR: Running exception handlers
Running handlers complete
[2015-10-07T14:55:55-04:00] ERROR: Exception handlers complete
[2015-10-07T14:55:55-04:00] FATAL: Stacktrace dumped to /home/me/chef-solo/chef-stacktrace.out
[2015-10-07T14:55:55-04:00] DEBUG: Chef::Exceptions::Package: python_package[tinydb] (my-cookbook::default line 34) had an error: Chef::Exceptions::Package: No candidate version available for tinydb
/opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:241:in `run'
/opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:322:in `block in run'
/opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:321:in `each'
/opt/chefdk/embedded/apps/chef/lib/chef/mixin/why_run.rb:321:in `run'
/opt/chefdk/embedded/apps/chef/lib/chef/provider.rb:144:in `process_resource_requirements'
/opt/chefdk/embedded/apps/chef/lib/chef/provider.rb:120:in `run_action'
/opt/chefdk/embedded/apps/chef/lib/chef/resource.rb:562:in `run_action'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:339:in `block in converge'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `catch'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `converge'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:353:in `converge_and_save'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:457:in `run'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:271:in `block in fork_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:225:in `block in run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:213:in `run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:280:in `block in interval_run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `loop'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `interval_run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:247:in `run_application'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:60:in `run'
/opt/chefdk/embedded/apps/chef/bin/chef-solo:25:in `<top (required)>'
/opt/chefdk/bin/chef-solo:51:in `load'
/opt/chefdk/bin/chef-solo:51:in `<main>'
Chef Client failed. 2 resources updated in 23.201344547 seconds
[2015-10-07T14:55:55-04:00] ERROR: python_package[tinydb] (my-cookbook::default line 34) had an error: Chef::Exceptions::Package: No candidate version available for tinydb
[2015-10-07T14:55:55-04:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Cookbook can't be trivially copied with git

We use git submodules extensively as a basis for building machine images using chef-zero. This is one of the first cookbooks I've seen that makes it impossible to git-clone it into a cookbooks directory from which we can converge a host.

I'm aware that the cookbook is uploaded to Supermarket, but that would entail a number of extra steps as opposed to git submodule update. Another advantage of using Git submodules is that I can pin a specific cookbook ref in my repository.

I appreciate your creativity here, but as a user I'm not feeling the benefits.

Unable to find cookbook for file "/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/system.rb"

I'm at a loss for figuring out what is going wrong here. Any help would be appreciated. OS is RHEL 7, FWIW.

# metadata.rb
...
depends 'application', '~> 5.0'
depends 'application_python', '~> 4.0'
depends 'application_git', '~> 1.0'
depends 'poise-python', '~> 1.1'
depends 'r701-apache'
depends 'r701-yum'
depends 'r701-chef-vault'

# recipe snippet
application node['r701-gravatar']['server_root_path'] do
  python '2' do
    provider :system
  end

...

Error:

/:gravatar-dev# chef-client -o 'r701-gravatar' -E development
...
  * application_python[2] action install

    ================================================================================
    Error executing action `install` on resource 'application_python[2]'
    ================================================================================

    Poise::Error
    ------------
    Unable to find cookbook for file "/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/system.rb"

    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/utils.rb:64:in `find_cookbook_name'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/defined_in.rb:69:in `poise_defined_in_cookbook'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:237:in `default_inversion_attributes'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:256:in `resolve_inversion_attribute'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:272:in `block in inversion_options'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:271:in `tap'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:271:in `inversion_options'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:177:in `options'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb:32:in `install_system_packages'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/system.rb:55:in `install_python'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:45:in `block in action_install'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `instance_eval'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `subcontext_block'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:67:in `notifying_block'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/r701-gravatar/recipes/default.rb

     18:   python '2' do
     19:     provider :system
     20:   end
     21:

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/r701-gravatar/recipes/default.rb:18:in `block in from_file'

    application_python("2") do
      provider PoisePython::PythonProviders::System
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      subresources [application_virtualenv[/gravatar-dev/gravatar-server]]
      declared_type :application_python
      cookbook_name "r701-gravatar"
      recipe_name "default"
      parent application[/gravatar-dev/gravatar-server]
      parent_python nil
    end

Softwarecollections.org is throwing 404 not found errors

    ================================================================================
    Error executing action `install` on resource 'python_runtime[2]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    poise_languages_scl[python27] (/etc/chef/local-mode-cache/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/mixin.rb line 27) had an error: Mixlib::ShellOut::ShellCommandFailed: rpm_package[rhscl-python27] (/etc/chef/local-mode-cache/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/resource.rb line 112) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
    ---- Begin output of rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
    ' https://www.softwarecollections.org/en/scls/rhscl/python27/fedora-21-x86_64/download/rhscl-python27-fedora-21-x86_64.noarch.rpm ----
    STDOUT: 
    STDERR: curl: (22) The requested URL returned error: 404 NOT FOUND
    error: open of https://www.softwarecollections.org/en/scls/rhscl/python27/fedora-21-x86_64/download/rhscl-python27-fedora-21-x86_64.noarch.rpm failed: No such file or directory
    ---- End output of rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
    ' https://www.softwarecollections.org/en/scls/rhscl/python27/fedora-21-x86_64/download/rhscl-python27-fedora-21-x86_64.noarch.rpm ----
    Ran rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE}
    ' https://www.softwarecollections.org/en/scls/rhscl/python27/fedora-21-x86_64/download/rhscl-python27-fedora-21-x86_64.noarch.rpm returned 1

    Cookbook Trace:
    ---------------
    /etc/chef/local-mode-cache/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
    /etc/chef/local-mode-cache/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/resource.rb:68:in `action_install'
    /etc/chef/local-mode-cache/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
    /etc/chef/local-mode-cache/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

    Resource Declaration:
    ---------------------
    # In /etc/chef/local-mode-cache/cache/cookbooks/etucker/recipes/default.rb

     78: python_runtime "2"
     79:

    Compiled Resource:
    ------------------
    # Declared in /etc/chef/local-mode-cache/cache/cookbooks/etucker/recipes/default.rb:78:in `from_file'

    python_runtime("2") do
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      subresources [python_package[astroid], python_package[awscli], python_package[awsebcli], python_package[aws-shell], python_package[boto3], python_package[docker-compose], python_package[docstring], python_package[ndg-httpsclient], python_package[pip], python_package[pyasn1], python_package[pyopenssl], python_package[pylint], python_package[s4cmd], python_package[wand], python_package[websocket], python_package[xkcdpass]]
      declared_type :python_runtime
      cookbook_name "etucker"
      recipe_name "default"
      pip_version true
      setuptools_version true
      version "2"
      virtualenv_version true
      wheel_version true
    end

Perhaps you an decipher? Is Fedora 21 not supported by this cookbook?

python_runtime does not install pip on CentOS/RHEL 7

Test-case recipe only has the following inside

Berksfile

cookbook 'poise'
cookbook 'poise-languages'
cookbook 'poise-python'

metadata.rb

depends 'poise-python'

recipes/default.rb

python_runtime '2.7'

Logging in after a "kitchen converge" results in the following

[vagrant@default-centos-71 ~]$ which pip
/usr/bin/which: no pip in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/vagrant/.local/bin:/home/vagrant/bin)

This causes a subsequent python_package call to fail, but it is a silent failure which results in the recipe continuing and failing at a later point in time because the expected python tool was never installed so its command was unavailable.

Let me know if there's any additional info you need from me to help replicate this issue to resolve it. Thanks.

Please help to talk with python-beaver community

Guys,

I got this error from poise-python:

Requirement already satisfied (use --upgrade to upgrade): boto>=2.8.0 in /usr/local/lib/python2.7/dist-packages (from beaver==31)
Requirement already satisfied (use --upgrade to upgrade): conf-d>=0.0.4 in /usr/local/lib/python2.7/dist-packages (from beaver==31)
Requirement already satisfied (use --upgrade to upgrade): glob2==0.3 in /usr/local/lib/python2.7/dist-packages (from beaver==31)
Collecting mosquitto>=1.1 (from beaver==31)
STDERR: /usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Could not find a version that satisfies the requirement mosquitto>=1.1 (from beaver==31) (from versions: )
No matching distribution found for mosquitto>=1.1 (from beaver==31)
---- End output of ["/usr/bin/python2.7", "-m", "pip.__main__", "install", "beaver==31"] ----
Ran ["/usr/bin/python2.7", "-m", "pip.__main__", "install", "beaver==31"] returned 1

Basically is possible to turn around this error by use:

pip install git+git://github.com/python-beaver/python-beaver.git@master#egg=beaver

But your cookbook needs to use a version instead off a branch, because it I open this ticket at python-beaver github:
python-beaver/python-beaver#401

Pip hack script fails with --extra-index-url option

Running into a problem where I think the PIP_HACK_SCRIPT is failing due to an --extra-index-url option given in the options property. My resource looks similar to:

python_package 'tinydb' do
  version '2.4'
  virtualenv my_venv_dir
  options '--extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple'
end

Running the cookbook gives me this stacktrace:

...<snip>...
[2015-10-08T09:20:55-04:00] DEBUG: [python_package[tinydb]] Running python command: /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4
Exception:
Traceback (most recent call last):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 95, in run
    self.run_outdated(options)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 104, in run_outdated
    for dist, version, typ in self.find_packages_latest_versions(options):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 119, in find_packages_latest_versions
    user_only=options.user):
  File "<stdin>", line 16, in replacement
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2980, in parse
    reqs = list(parse_requirements(s))
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2924, in parse_requirements
    "version spec")
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2889, in scan_list
    raise RequirementParseError(msg, line, "at", line[p:])
pip._vendor.pkg_resources.RequirementParseError: Expected version spec in http://artifactory.example.com/api/pypi/my-repo/simple at ://artifactory.example.com/api/pypi/my-repo/simple


    ================================================================================
    Error executing action `install` on resource 'python_package[tinydb]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '2'
    ---- Begin output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
    STDOUT:
    STDERR: Exception:
    Traceback (most recent call last):
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/basecommand.py", line 211, in main
        status = self.run(options, args)
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 95, in run
        self.run_outdated(options)
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 104, in run_outdated
        for dist, version, typ in self.find_packages_latest_versions(options):
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 119, in find_packages_latest_versions
        user_only=options.user):
      File "<stdin>", line 16, in replacement
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2980, in parse
        reqs = list(parse_requirements(s))
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2924, in parse_requirements
        "version spec")
      File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2889, in scan_list
        raise RequirementParseError(msg, line, "at", line[p:])
    pip._vendor.pkg_resources.RequirementParseError: Expected version spec in http://artifactory.example.com/api/pypi/my-repo/simple at ://artifactory.example.com/api/pypi/my-repo/simple
    ---- End output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
    Ran /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 returned 2

    Cookbook Trace:
    ---------------
    /home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:206:in `tap'
    /home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:206:in `language_command_shell_out!'
    /home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:221:in `block in language_command_mixin'
    /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:260:in `pip_command'
    /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:284:in `pip_outdated'
    /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:166:in `check_package_versions'
    /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:146:in `load_current_resource'

    Resource Declaration:
    ---------------------
    # In /home/me/chef-repo/cookbooks/my-cookbook/recipes/default.rb

     34: python_package 'tinydb' do
     35:   version '2.4'
     36:   virtualenv my_venv_dir
     37:   options '--extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple'
     38: end
     39:

    Compiled Resource:
    ------------------
    # Declared in /home/me/chef-repo/cookbooks/my-cookbook/recipes/default.rb:34:in `from_file'

    python_package("tinydb") do
      action :install
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      options "--extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple"
      package_name "tinydb"
      version "2.4"
      timeout 900
      declared_type :python_package
      cookbook_name :"my-cookbook"
      recipe_name "default"
      parent_python python_virtualenv[/opt/my-venv]
    end

[2015-10-08T09:20:56-04:00] INFO: Running queued delayed notifications before re-raising exception
[2015-10-08T09:20:56-04:00] DEBUG: Re-raising exception: Mixlib::ShellOut::ShellCommandFailed - python_package[tinydb] (my-cookbook::default line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '2'
---- Begin output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
STDOUT:
STDERR: Exception:
Traceback (most recent call last):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 95, in run
    self.run_outdated(options)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 104, in run_outdated
    for dist, version, typ in self.find_packages_latest_versions(options):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 119, in find_packages_latest_versions
    user_only=options.user):
  File "<stdin>", line 16, in replacement
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2980, in parse
    reqs = list(parse_requirements(s))
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2924, in parse_requirements
    "version spec")
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2889, in scan_list
    raise RequirementParseError(msg, line, "at", line[p:])
pip._vendor.pkg_resources.RequirementParseError: Expected version spec in http://artifactory.example.com/api/pypi/my-repo/simple at ://artifactory.example.com/api/pypi/my-repo/simple
---- End output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
Ran /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 returned 2
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.1.0/lib/mixlib/shellout.rb:289:in `invalid!'
  /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.1.0/lib/mixlib/shellout.rb:276:in `error!'
  /home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:206:in `tap'
  /home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:206:in `language_command_shell_out!'
  /home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:221:in `block in language_command_mixin'
  /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:260:in `pip_command'
  /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:284:in `pip_outdated'
  /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:166:in `check_package_versions'
  /home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:146:in `load_current_resource'
  /opt/chefdk/embedded/apps/chef/lib/chef/provider.rb:113:in `run_action'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource.rb:562:in `run_action'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  /opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
  /opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:339:in `block in converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `catch'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `converge'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:353:in `converge_and_save'
  /opt/chefdk/embedded/apps/chef/lib/chef/client.rb:457:in `run'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:271:in `block in fork_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:225:in `block in run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:213:in `run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:280:in `block in interval_run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `loop'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `interval_run_chef_client'
  /opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:247:in `run_application'
  /opt/chefdk/embedded/apps/chef/lib/chef/application.rb:60:in `run'
  /opt/chefdk/embedded/apps/chef/bin/chef-solo:25:in `<top (required)>'
  /opt/chefdk/bin/chef-solo:51:in `load'
  /opt/chefdk/bin/chef-solo:51:in `<main>'

Running handlers:
[2015-10-08T09:20:56-04:00] ERROR: Running exception handlers
Running handlers complete
[2015-10-08T09:20:56-04:00] ERROR: Exception handlers complete
[2015-10-08T09:20:56-04:00] FATAL: Stacktrace dumped to /home/me/chef-solo/chef-stacktrace.out
[2015-10-08T09:20:56-04:00] DEBUG: Mixlib::ShellOut::ShellCommandFailed: python_package[tinydb] (my-cookbook::default line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '2'
---- Begin output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
STDOUT:
STDERR: Exception:
Traceback (most recent call last):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 95, in run
    self.run_outdated(options)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 104, in run_outdated
    for dist, version, typ in self.find_packages_latest_versions(options):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 119, in find_packages_latest_versions
    user_only=options.user):
  File "<stdin>", line 16, in replacement
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2980, in parse
    reqs = list(parse_requirements(s))
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2924, in parse_requirements
    "version spec")
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2889, in scan_list
    raise RequirementParseError(msg, line, "at", line[p:])
pip._vendor.pkg_resources.RequirementParseError: Expected version spec in http://artifactory.example.com/api/pypi/my-repo/simple at ://artifactory.example.com/api/pypi/my-repo/simple
---- End output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
Ran /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 returned 2
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.1.0/lib/mixlib/shellout.rb:289:in `invalid!'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.1.0/lib/mixlib/shellout.rb:276:in `error!'
/home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:206:in `tap'
/home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:206:in `language_command_shell_out!'
/home/me/chef-repo/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:221:in `block in language_command_mixin'
/home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:260:in `pip_command'
/home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:284:in `pip_outdated'
/home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:166:in `check_package_versions'
/home/me/chef-repo/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:146:in `load_current_resource'
/opt/chefdk/embedded/apps/chef/lib/chef/provider.rb:113:in `run_action'
/opt/chefdk/embedded/apps/chef/lib/chef/resource.rb:562:in `run_action'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chefdk/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
/opt/chefdk/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:339:in `block in converge'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `catch'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:334:in `converge'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:353:in `converge_and_save'
/opt/chefdk/embedded/apps/chef/lib/chef/client.rb:457:in `run'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:271:in `block in fork_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:259:in `fork_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:225:in `block in run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:213:in `run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:280:in `block in interval_run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `loop'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:269:in `interval_run_chef_client'
/opt/chefdk/embedded/apps/chef/lib/chef/application/solo.rb:247:in `run_application'
/opt/chefdk/embedded/apps/chef/lib/chef/application.rb:60:in `run'
/opt/chefdk/embedded/apps/chef/bin/chef-solo:25:in `<top (required)>'
/opt/chefdk/bin/chef-solo:51:in `load'
/opt/chefdk/bin/chef-solo:51:in `<main>'
Chef Client failed. 6 resources updated in 29.366737526 seconds
[2015-10-08T09:20:56-04:00] ERROR: python_package[tinydb] (my-cookbook::default line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '2'
---- Begin output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
STDOUT:
STDERR: Exception:
Traceback (most recent call last):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 95, in run
    self.run_outdated(options)
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 104, in run_outdated
    for dist, version, typ in self.find_packages_latest_versions(options):
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/commands/list.py", line 119, in find_packages_latest_versions
    user_only=options.user):
  File "<stdin>", line 16, in replacement
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2980, in parse
    reqs = list(parse_requirements(s))
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2924, in parse_requirements
    "version spec")
  File "/opt/my-venv/lib64/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2889, in scan_list
    raise RequirementParseError(msg, line, "at", line[p:])
pip._vendor.pkg_resources.RequirementParseError: Expected version spec in http://artifactory.example.com/api/pypi/my-repo/simple at ://artifactory.example.com/api/pypi/my-repo/simple
---- End output of /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 ----
Ran /opt/my-venv/bin/python - list --extra-index-url http://artifactory.example.com/api/pypi/my-repo/simple --outdated tinydb\=\=2.4 returned 2
[2015-10-08T09:20:56-04:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

No candidate version available for setuptools

Seems like something is breaking this cookbook?

I'm getting the same issue that is shown by the TravisCI Failed build

Error executing action `install` on resource 'python_package[setuptools]'

https://travis-ci.org/poise/poise-python/jobs/103535517

The same commit over the past 4 days was passing so I don't think its directly related to poise-python but something in chef or ubuntu?

I'm using Ubuntu 14.04.2 LTS
Chef 12.5.1-1

Any ideas what could be causing the issue?

Wrong python version in virtual env with custom python packages

Hello,

i'm trying to setup virtual env based on IUS packages in CentOS 6.7 x86_64.

I use next recipe for virtual env creation:

include_recipe 'yum-ius'

python_runtime '2.7' do
  provider :system
  options :package_name => 'python27'
end

python_virtualenv '/opt/python2_ve' do
  action :create
  python '2.7'
end

Converge log:

       Recipe: mycookbook::default
         * python_runtime[2.7] action install
           * poise_languages_system[python27] action install
             - install version 2.7.11-1.ius.centos6 of package python27
           - install version 2.7.11-1.ius.centos6 of package python27-devel

       * python_runtime_pip[2.7] action install

       * python_package[setuptools] action install
         - install version 20.2.2 of package setuptools
       * python_package[wheel] action install
         - install version 0.29.0 of package wheel
       * python_package[virtualenv] action install
         - install version 15.0.0 of package virtualenv

       * python_virtualenv[/opt/python2_ve] action create
         - Creating virtualenv at /opt/python2_ve
         * python_runtime_pip[/opt/python2_ve] action install

         * python_package[setuptools] action install
           - install version 20.2.2 of package setuptools
         * python_package[wheel] action install
           - install version 0.29.0 of package wheel

After converge i have:

[vagrant@ ~]$ /opt/python2_ve/bin/python -V
Python 2.6.6

but should be python 2.7.

RHEL system python exception: wheel

application_python("2") do
  provider PoisePython::PythonProviders::System
  action [:install]
  updated true
  updated_by_last_action true
  retries 0
  retry_delay 2
  default_guard_interpreter :default
  subresources [application_virtualenv[/our/graphite], python_runtime_pip[2], python_package[setuptools], python_package[wheel], python_package[virtualenv]]
  declared_type :application_python
  cookbook_name "r701-graphite"
  recipe_name "etc-metrics"
  parent application[/our/graphite]
  parent_python nil
  pip_version true
  setuptools_version true
  version "2"
  virtualenv_version true
  wheel_version true
end

Chef::Exceptions::Package
-------------------------
python_package[wheel] (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 151) had an error: Chef::Exceptions::Package: No candidate version available for wheel

I'm guessing the problem here is that https://github.com/poise/poise-python/blob/master/lib/poise_python/python_providers/base.rb#L151 hardcodes the package name as wheel and the desired RHEL system package is python-wheel

pip options don't work well with python_package

...because it uses them for more than install:

python_package 'graphite-web' do
  options '--install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp"'
  version node['r701-graphite']['server']['graphite_web_version']
end

  Mixlib::ShellOut::ShellCommandFailed
  ------------------------------------
  Expected process to exit with [0], but received '2'
  ---- Begin output of /etc-metrics.our.org/graphite/bin/python -m pip.__main__ list --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp"  ----
  STDOUT:
  STDERR: Usage:
    /etc-metrics.mitre.org/graphite/bin/python -m pip list [options]

  no such option: --install-option
  ---- End output of /etc-metrics.our.org/graphite/bin/python -m pip.__main__ list --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp"  ----
  Ran /etc-metrics.our.org/graphite/bin/python -m pip.__main__ list --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp"  returned 2

Please document how to restructure the repo to be a cookbook so we can fork and test changes

user specific virtualenv

Hi,

I am trying to use a user specific virtualenv:

include_recipe 'poise-python'

service = 's3cmd'

user service do
  system true
  supports manage_home: true
end

python_virtualenv "/home/#{service}/.env2" do
  user service
end

however I get the following error:

       Recipe: falcon-s3cmd::default
         * user[s3cmd] action create (up to date)
         * python_virtualenv[/home/s3cmd/.env2] action create

           ================================================================================
           Error executing action `create` on resource 'python_virtualenv[/home/s3cmd/.env2]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of ["/usr/bin/python2.7", "-m", "virtualenv", "/home/s3cmd/.env2"] ----
           STDOUT:
           STDERR: Traceback (most recent call last):
             File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
        "__main__", fname, loader, pkg_name)
             File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
        exec code in run_globals
             File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 2363, in <module>
        main()
             File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 832, in main
        symlink=options.symlink)
             File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 994, in create_environment
        site_packages=site_packages, clear=clear, symlink=symlink))
             File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 1170, in install_python
        mkdir(lib_dir)
             File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 448, in mkdir
        os.makedirs(path)
             File "/usr/lib/python2.7/os.py", line 150, in makedirs

             File "/usr/lib/python2.7/os.py", line 150, in makedirs
        makedirs(head, mode)
             File "/usr/lib/python2.7/os.py", line 157, in makedirs
        mkdir(name, mode)
           OSError: [Errno 13] Permission denied: '/home/s3cmd/.env2'
           ---- End output of ["/usr/bin/python2.7", "-m", "virtualenv", "/home/s3cmd/.env2"] ----
           Ran ["/usr/bin/python2.7", "-m", "virtualenv", "/home/s3cmd/.env2"] returned 1

Omitting the user works fine but then the virtualenv is owned by root which produces other problems within the virtualenv such as this:

The directory '/home/s3cmd/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

What am I doing wrong?

Cheers
Jannis

pip_requirements should support cwd attribute

For example, I have a requirements.txt with the following:

-e .

Which in turn pulls up setup.py for the list of requirements. When I try to use this requirements.txt, I get the following error:

STDERR: Directory '.' is not installable. File 'setup.py' not found.

Which is because the pip command is not being executed from the same directory as the requirements.txt file.

Default package names for Ubuntu 16.04 (xenial)

Hi!

Thanks for making this package, really useful. It isn't able to autodetect the package name for the newest Ubuntu LTS release, 16.04 (xenial). Adding the following after line 35 of lib/poise_python/python_providers/system.rb does the trick:
'16.04' => %w{python3.5 python2.7},

Best,

Adam

Unable to replace python cookbook with poise-python

We are trying to replace the depreciated python cookbook with the poise-python cookbook in our Chef 12 environment. When running chef-client(version 12.3.0) on a system that previously had the python cookbook on it, we receive the following error message:

Recipe: poise-python::default
  * python_runtime[2] action install

    ================================================================================
    Error executing action `install` on resource 'python_runtime[2]'
    ================================================================================

    Poise::Error
    ------------
    Unable to find cookbook for file "/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb"

    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/utils.rb:64:in `find_cookbook_name'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/defined_in.rb:69:in `poise_defined_in_cookbook'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:233:in `default_inversion_attributes'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:252:in `resolve_inversion_attribute'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:268:in `block in inversion_options'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:267:in `tap'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:267:in `inversion_options'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:307:in `resolve_inversion_provider'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/inversion.rb:350:in `provides?'

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/poise-python/recipes/default.rb

     20: python_runtime '2' if node['poise-python']['install_python2']

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/poise-python/recipes/default.rb:20:in `from_file'

    python_runtime("2") do
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      subresources [python_package[pymssql], python_package[pexpect]]
      declared_type :python_runtime
      cookbook_name "poise-python"
      recipe_name "default"
    end

However, we can see that cookbook files do exist:

$:~# ls -la /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/
total 28
drwxr-xr-x 2 root root 4096 Oct  2 16:58 .
drwxr-xr-x 5 root root 4096 Oct  2 16:58 ..
-rw------- 1 root root 5819 Oct  2 16:58 base.rb
-rw------- 1 root root 2646 Oct  2 16:58 portable_pypy.rb
-rw------- 1 root root 3029 Oct  2 16:58 scl.rb
-rw------- 1 root root 2647 Oct  2 16:58 system.rb

There are no issues running the poise-python cookbook on brand new nodes or nodes that have never ran the python cookbook before.

Can't install with librarian-chef

Trying to install via librarian-chef results in errors when it tries to resolve dependencies. It looks like it's failing because the cookbook does not contain a metadata file. Is there are reason there's no metadata.rb?

python_package fails to recognize that a package is already installed

Hello,

I'm trying to install several packages using the python_package resource as follow:

pip_dependencies = %w(
  argparse
  boto
  gevent
  six
)

python_runtime '2' do
  pip_version '8.0.0'
end

pip_dependencies.each do |pip|
  python_package pip do
    action :install
    python '2'
  end
end

Then chef-run breaks with:

  * python_runtime[2] action install
    * poise_languages_system[python2.7] action install
       (up to date)
     (up to date)
  * python_runtime_pip[2] action install[2016-01-21T12:27:30+00:00] WARN: pip does not support bootstrapping a specific version, see https://github.com/pypa/pip/issues/1087.


  * python_package[setuptools] action install (up to date)
  * python_package[wheel] action install
    * No candidate version available for wheel
    ================================================================================
    Error executing action `install` on resource 'python_package[wheel]'
    ================================================================================

    Chef::Exceptions::Package
    -------------------------
    No candidate version available for wheel

    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb

    151:         python_package 'wheel' do
    152:           parent_python new_resource
    153:           version wheel_version if wheel_version.is_a?(String)
    154:         end
    155:       end

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:151:in `install_wheel'

    python_package("wheel") do
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      package_name "wheel"
      declared_type :python_package
      cookbook_name "mycookbook-postgres"
      parent_python python_runtime[2]
    end


  ================================================================================
  Error executing action `install` on resource 'python_runtime[2]'
  ================================================================================

  Chef::Exceptions::Package
  -------------------------
  python_package[wheel] (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 151) had an error: Chef::Exceptions::Package: No candidate version available for wheel

  Cookbook Trace:
  ---------------
  /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
  /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

  Resource Declaration:
  ---------------------
  # In /var/chef/cache/cookbooks/mycookbook-postgres/recipes/_backups_common.rb

   34: python_runtime '2' do
   35:   pip_version '8.0.0'
   36: end
   37:

  Compiled Resource:
  ------------------
  # Declared in /var/chef/cache/cookbooks/mycookbook-postgres/recipes/_backups_common.rb:34:in `from_file'

  python_runtime("2") do
    action [:install]
    updated true
    updated_by_last_action true
    retries 0
    retry_delay 2
    default_guard_interpreter :default
    subresources [python_package[argparse], python_package[boto], python_package[gevent], python_package[six], python_package[wal-e], python_runtime_pip[2], python_package[setuptools], python_package[wheel], python_package[virtualenv]]
    declared_type :python_runtime
    cookbook_name "mycookbook-postgres"
    recipe_name "_backups_common"
    pip_version "8.0.0"
    setuptools_version true
    version "2"
    virtualenv_version true
    wheel_version true
  end

The underlying platform is Ubuntu 14.04 and I'm running chef-client 12.5.1. Even if I install all the packages manually with pip, subsequent chef runs fail with the same error. I can also list the packages with pip, but chef is not able to install them.

Even with my limited tests and knowledge about the python environment my conclusion is that is more a bug in the cookbook than anything else but I'd love to be proven wrong (getting a solution would be wonderful).

Thanks.

python_runtime resource option setting doc is unclear to me

This is not clear to me. I don't see what's different about the 2 examples really:

When setting options in the resource you can either set them for all providers:
python_runtime 'myapp' do
  version '2.7'
  options pip_version: false
end
or for a single provider:
python_runtime 'myapp' do
  version '2.7'
  options :system, dev_package: false
end

MemoryError during install package using pip

MemoryError occurs during install package using pip via 'python_package' directive.

I found pip's caching mechanism makes problem, see this article, so it would be fine if there is a kind of configuration not to use pip cache.

chef-test [2016-03-15T11:11:46-04:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out                                                                                                
chef-test [2016-03-15T11:11:46-04:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report                                                                         
chef-test [2016-03-15T11:11:46-04:00] ERROR: python_package[moin] (wiki::default line 19) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '2'    
chef-test ---- Begin output of ["/usr/bin/python2.7", "-m", "pip.__main__", "install", "moin==1.9.8"] ----                                                                                           
chef-test STDOUT: Collecting moin==1.9.8                                                                                                                                                             
chef-test   Downloading moin-1.9.8.tar.gz (37.1MB)                                                                                                                                                   
chef-test STDERR: Exception:                                                                                                                                                                         
chef-test Traceback (most recent call last):
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 209, in main
chef-test     status = self.run(options, args)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 310, in run
chef-test     wb.build(autobuilding=True)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 748, in build
chef-test     self.requirement_set.prepare_files(self.finder)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 360, in prepare_files
chef-test     ignore_dependencies=self.ignore_dependencies))
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 577, in _prepare_file
chef-test     session=self.session, hashes=hashes)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 810, in unpack_url
chef-test     hashes=hashes
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 649, in unpack_http_url
chef-test     hashes)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 871, in _download_http_url
chef-test     _download_url(resp, link, content_file, hashes)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 595, in _download_url
chef-test     hashes.check_against_chunks(downloaded_chunks)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
chef-test     for chunk in chunks:
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 563, in written_chunks
chef-test     for chunk in chunks:
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/utils/ui.py", line 139, in iter
chef-test     for x in it:
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 552, in resp_read
chef-test     decode_content=False):
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/response.py", line 344, in stream
chef-test     data = self.read(amt=amt, decode_content=decode_content)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/response.py", line 301, in read
chef-test     data = self._fp.read(amt)
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/filewrapper.py", line 54, in read
chef-test     self.__callback(self.__buf.getvalue())
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/controller.py", line 275, in cache_response
chef-test     self.serializer.dumps(request, response, body=body),
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/serialize.py", line 55, in dumps
chef-test     "body": _b64_encode_bytes(body),
chef-test   File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/serialize.py", line 12, in _b64_encode_bytes
chef-test     return base64.b64encode(b).decode("ascii")
chef-test MemoryError

I tried pip install matplotlib directly on the machine via ssh it fails, but when I try pip --no-cache-dir install matplotlib instead, it worked fine.

Error executing action `install` on resource 'poise_languages_system[python3.4]'

Hello,
I have been trying to work out this problem for a while now. I have been testing this cookbook in vagrant and docker ubuntu 14.04. All tests passed in vagrant and docker, but when I try it out on a lab server I get the following error. Any ideas would be appriciated. I am trying to upgrade our cookbooks from python to poise-python.

Thank you!

* python_runtime[python_runtime] action install
    * poise_languages_system[python3.4] action install

      ================================================================================
      Error executing action `install` on resource 'poise_languages_system[python3.4]'
      ================================================================================

      Chef::Exceptions::ValidationFailed
      ----------------------------------
      Option version must be a kind of [String]!  You passed [nil, nil].

      Cookbook Trace:
      ---------------
      /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:147:in `block in package_resources'
      /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:146:in `tap'
      /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:146:in `package_resources'
      /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `run_package_action'
      /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:100:in `action_install'
      /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
      /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

      Resource Declaration:
      ---------------------
      # In /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb

       32:         poise_languages_system options['package_name'] || system_package_name do
       33:           # Otherwise use the default install action.
       34:           action(:upgrade) if options['package_upgrade']
       35:           parent new_resource
       36:           # Don't pass true because we want the default computed behavior for that.
       37:           dev_package options['dev_package'] unless options['dev_package'] == true
       38:           dev_package_overrides dev_package_overrides
       39:           package_version options['package_version'] if options['package_version']
       40:           version options['version']
       41:         end
       42:       end

      Compiled Resource:
      ------------------
      # Declared in /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb:32:in `install_system_packages'

      poise_languages_system("python3.4") do
        action [:install]
        retries 0
        retry_delay 2
        default_guard_interpreter :default
        declared_type :poise_languages_system
        cookbook_name "krs-log-archive"
        parent # Declared in /var/chef/cache/cookbooks/krs-log-archive/recipes/python_setup.rb:10:in `from_file'

      python_runtime("python_runtime") do
        provider PoisePython::PythonProviders::System
        action [:install]
        retries 0
        retry_delay 2
        default_guard_interpreter :default
        subresources [python_virtualenv[.env]]
        declared_type :python_runtime
        cookbook_name "krs-log-archive"
        recipe_name "python_setup"
        version "3"
        pip_version true
        setuptools_version true
        virtualenv_version true
        wheel_version true
      end

        version "3"
        package_name "python3.4"
        dev_package "python3.4-dev"
      end


    ================================================================================
    Error executing action `install` on resource 'python_runtime[python_runtime]'
    ================================================================================

    Chef::Exceptions::ValidationFailed
    ----------------------------------
    poise_languages_system[python3.4] (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32) had an error: Chef::Exceptions::ValidationFailed: Option version must be a kind of [String]!  You passed [nil, nil].

    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:147:in `block in package_resources'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:146:in `tap'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:146:in `package_resources'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `run_package_action'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:100:in `action_install'
    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/krs-log-archive/recipes/python_setup.rb

     10: python_runtime 'python_runtime' do
     11:   provider :system
     12:    version '3'
     13:    pip_version true
     14:    setuptools_version true
     15:    virtualenv_version true
     16:    wheel_version true
     17: end
     18:  

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/krs-log-archive/recipes/python_setup.rb:10:in `from_file'

    python_runtime("python_runtime") do
      provider PoisePython::PythonProviders::System
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      subresources [python_virtualenv[.env]]
      declared_type :python_runtime
      cookbook_name "krs-log-archive"
      recipe_name "python_setup"
      version "3"
      pip_version true
      setuptools_version true
      virtualenv_version true
      wheel_version true
    end


Running handlers:
[2015-12-08T06:39:06-07:00] ERROR: Running exception handlers
Running handlers complete
[2015-12-08T06:39:06-07:00] ERROR: Exception handlers complete
[2015-12-08T06:39:06-07:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
Chef Client failed. 4 resources updated in 43.052575525 seconds
[2015-12-08T06:39:07-07:00] ERROR: python_runtime[python_runtime] (krs-log-archive::python_setup line 10) had an error: Chef::Exceptions::ValidationFailed: poise_languages_system[python3.4] (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32) had an error: Chef::Exceptions::ValidationFailed: Option version must be a kind of [String]!  You passed [nil, nil].
[2015-12-08T06:39:08-07:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Error defining private method

Build currently fail

TypeError
---------
#<Proc:0x007f952784bd40@/Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:139 (lambda)> is not a symbol

Cookbook Trace:
---------------
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:139:in `private'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:139:in `language_command_mixin'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:152:in `block in <module:Resource>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise/files/halite_gem/poise/utils.rb:148:in `instance_exec'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise/files/halite_gem/poise/utils.rb:148:in `block (3 levels) in parameterized_module'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/python_command_mixin.rb:30:in `include'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/python_command_mixin.rb:30:in `<module:Resource>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/python_command_mixin.rb:29:in `<module:PythonCommandMixin>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/python_command_mixin.rb:25:in `<module:PoisePython>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/python_command_mixin.rb:21:in `<top (required)>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/resources/pip_requirements.rb:21:in `<top (required)>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/resources.rb:17:in `<top (required)>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/files/halite_gem/poise_python/cheftie.rb:17:in `<top (required)>'
  /Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-python/libraries/default.rb:19:in `<top (required)>'

Relevant File Content:
----------------------
/Users/djimenez/Development/ForeFlight/web/.bundle/librarian-chef/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:

132:  
133:              # Create the main accessor for the parent/path.
134:              define_method(name) do |val=Poise::NOT_PASSED|
135:                language_command_runtime(name, runtime, val)
136:              end
137:  
138:              # Create the method to inherit settings from another resource.
139>>             private define_method(:"#{name}_from_parent") { |resource|
140:                language_command_runtime_from_parent(name, resource)
141:              }
142:            end
143:  
144:            # @api private
145:            def included(klass)
146:              super
147:              klass.extend(ClassMethods)
148:            end

centos6 default python binary not properly setting LD_LIBRARY_PATH

This is after a default install of a python virtualenv using the python_runtime and python_virtualenv resources from this cookbook.

[root@server ]# source venv/bin/activate
(venv)[root@server ]# python 
python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No     such file or directory
(venv)[root@server ]# which python
/opt/venv/bin/python
(venv)[root@server ]# updatedb
(venv)[root@server ]# locate libpython2.7.so.1.0
/opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
(venv)[root@server ]# export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64
(venv)[root@server ]# python
Python 2.7.8 (default, Jun  9 2015, 19:27:51)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()

Possibly broken pip install on Amazon linux

Forking this off from #35, I should find a way to test on Amazon Linux because it sounds like the way they package pip is causing issues. Unfortunately I don't know of any official way to test on it because they only release the OS as an AMI and my test options are either Docker images or Rackspace VMs.

python_virtualenv seizes using recent version

chef-12.6.0 on ubuntu-12.04 - only thing changed is the newer version.

         * python_runtime_pip[2.7] action install (up to date)
        (up to date)
        (up to date)
        (up to date)
          (up to date)
       * python_virtualenv[/opt/graphite] action create
       * python_runtime_pip[/opt/graphite] action install

           ================================================================================
           Error executing action `install` on resource 'python_runtime_pip[/opt/graphite]'
           ================================================================================

           Errno::ENOENT
           -------------
           No such file or directory - /opt/graphite/bin/python

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:155:in `pip_version'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:67:in `block in load_current_resource'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:65:in `tap'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:65:in `load_current_resource'
       /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb

           116:         python_runtime_pip new_resource.name do
       117:           parent new_resource
           118:           # If the version is `true`, don't pass it at all.
           119:           version pip_version if pip_version.is_a?(String)
           120:           get_pip_url pip_url if pip_url
           121:         end
           122:       end
           123:

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:116:in `install_pip'

           python_runtime_pip("/opt/graphite") do
             action [:install]
             retries 0
         retry_delay 2
             default_guard_interpreter :default
         declared_type :python_runtime_pip
             cookbook_name "graphite_ng"
         parent python_virtualenv[/opt/graphite]
           end


         ================================================================================
         Error executing action `create` on resource 'python_virtualenv[/opt/graphite]'
         ================================================================================

         Errno::ENOENT
         -------------
         python_runtime_pip[/opt/graphite] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 116) had an error: Errno::ENOENT: No such file or directory - /opt/graphite/bin/python

         Cookbook Trace:
         ---------------
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:155:in `pip_version'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:67:in `block in load_current_resource'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:65:in `tap'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:65:in `load_current_resource'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

         Resource Declaration:
         ---------------------
         # In /tmp/kitchen/cache/cookbooks/graphite_ng/recipes/_python.rb

          35: python_virtualenv node['graphite']['base_path'] do
        36:   user node['graphite']['carbon']['user']
          37:   group node['graphite']['carbon']['group']
          38:   system_site_packages true
        39: end

         Compiled Resource:
         ------------------
         # Declared in /tmp/kitchen/cache/cookbooks/graphite_ng/recipes/_python.rb:35:in `from_file'

       python_virtualenv("/opt/graphite") do
         provider PoisePython::Resources::PythonVirtualenv::Provider
         action [:create]
           retries 0
         retry_delay 2
         default_guard_interpreter :default
         subresources [python_execute[validate-storage-schemas], python_execute[carbon-mockup], python_execute[migrate-database], python_execute[create-superuser], python_execute[validate-django], python_runtime_pip[/opt/graphite], python_package[setuptools], python_package[wheel]]
         declared_type :python_virtualenv
           cookbook_name "graphite_ng"
           recipe_name "_python"
           user "graphite"
         group "graphite"
           system_site_packages true
           parent_python python_runtime[2.7]
           path "/opt/graphite"
           pip_version true
         setuptools_version true
         wheel_version true
       end

Can't install via Berkshelf

I got an error as following:

$ berks install
Resolving cookbook dependencies...
Fetching 'poise-python' from [email protected]:poise/poise-python.git (at master)
The resource at '/tmp/d20150813-27271-14qi8s8' does not appear to be a valid cookbook. Does it have a metadata.rb?

This application_python cookbook example is using this cookbook:
https://github.com/poise/application_python/blob/master/test/cookbooks/application_python_test/recipes/django.rb

How can I use it?

warnings during chef run: 'You declared a new resource ... for resource ..., but it comes alphabetically after ...'

       [2015-10-06T14:21:53+00:00] WARN: You declared a new resource PoisePython::PythonProviders::Scl for resource python_runtime, but it comes alphabetically after PoisePython::PythonProviders::PortablePyPy and has the same filters ({}), so it will not be used. Use override: true if you want to use it for python_runtime.
       [2015-10-06T14:21:53+00:00] WARN: You declared a new resource PoisePython::PythonProviders::System for resource python_runtime, but it comes alphabetically after PoisePython::PythonProviders::PortablePyPy and has the same filters ({}), so it will not be used. Use override: true if you want to use it for python_runtime.
       [2015-10-06T14:21:53+00:00] WARN: You declared a new resource PoisePython::PythonProviders::System for resource python_runtime, but it comes alphabetically after PoisePython::PythonProviders::Scl and has the same filters ({}), so it will not be used. Use override: true if you want to use it for python_runtime.
       [2015-10-06T14:21:53+00:00] WARN: You declared a new resource PoisePython::Resources::PythonVirtualenv::Provider for resource python_runtime, but it comes alphabetically after PoisePython::PythonProviders::PortablePyPy and has the same filters ({}), so it will not be used. Use override: true if you want to use it for python_runtime.
       [2015-10-06T14:21:53+00:00] WARN: You declared a new resource PoisePython::Resources::PythonVirtualenv::Provider for resource python_runtime, but it comes alphabetically after PoisePython::PythonProviders::Scl and has the same filters ({}), so it will not be used. Use override: true if you want to use it for python_runtime.
       [2015-10-06T14:21:53+00:00] WARN: You declared a new resource PoisePython::Resources::PythonVirtualenv::Provider for resource python_runtime, but it comes alphabetically after PoisePython::PythonProviders::System and has the same filters ({}), so it will not be used. Use override: true if you want to use it for python_runtime

seems to work fine, but in most chef runs these are the only warnings i get. regards

virtualenv installs with bad ownership?

I installed a virtualenv using this block:

python_virtualenv '/home/www/.env' do
    python 'py3'
    user 'www'
    group 'www'
end

Most of the virtualenv that results has the correct www/www permissoins, butt the pip that was subsequently installed was root/root, as are several items in site_packages, including __pycache__ for instance, which makes installing from the requirements file difficult.

I've tried following the source to see where that pip install happens but I've been losing track.

python_virtualenv attribute consistency

Hi there,

I'm attempting to use poise-python to replace the old python cookbook, so far not too bad but I'm running into issues in an upstream cookbook which uses the user variable on the python_virtualenv resource.

Since poise-python uses owner instead, this is causing issues which I don't know how to monkey patch.

Any ideas how this could be resolved?

Support for installing from private github repositories

With the python cookbook, we could install from private git repositories like so:

include_recipe 'python'
python_pip 'git+ssh://[email protected]/Asana/asana-kinesis-python.git'

Attempting the equivalent with python-poise:

python_package 'git+ssh://[email protected]/Asana/asana-kinesis-python.git'

yields

---- Begin output of /usr/bin/python2.7 - list asanakinesis --outdated git\+ssh://[email protected]/Asana/asana-kinesis-python.git ----
       STDOUT:
       STDERR: The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       /usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
         InsecurePlatformWarning
       You are using pip version 7.0.3, however version 8.0.0 is available.
       You should consider upgrading via the 'pip install --upgrade pip' command.
       Exception:
       Traceback (most recent call last):
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/basecommand.py", line 223, in main
           status = self.run(options, args)
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/commands/list.py", line 95, in run
           self.run_outdated(options)
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/commands/list.py", line 104, in run_outdated
           for dist, version, typ in self.find_packages_latest_versions(options):
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/commands/list.py", line 119, in find_packages_latest_versions
           user_only=options.user):
         File "<stdin>", line 18, in replacement
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/_vendor/pkg_resources/__init__.py", line 2960, in parse
           reqs = list(parse_requirements(s))
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/_vendor/pkg_resources/__init__.py", line 2904, in parse_requirements
           "version spec")
         File "/usr/local/lib/python2.7/dist-packages/pip-7.0.3-py2.7.egg/pip/_vendor/pkg_resources/__init__.py", line 2869, in scan_list
           raise ValueError(msg, line, "at", line[p:])
       ValueError: ('Expected version spec in', 'git+ssh://[email protected]/Asana/asana-kinesis-python.git', 'at', '+ssh://[email protected]/Asana/asana-kinesis-python.git')
       ---- End output of /usr/bin/python2.7 - list asanakinesis --outdated git\+ssh://[email protected]/Asana/asana-kinesis-python.git ----
       .7 - list asanakinesis --outdated git\+ssh://[email protected]/Asana/asana-kinesis-python.git returned 2
       [2016-01-20T22:28:42+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Is installing from private repositories supported in poise-python? Does it require different arguments?

Problem to install extra package

Hi,

I try to migrate from the python cookbook to poise-python, but I've a problem with the installation of a package (ex: airflow[mysql] )

python_package 'airflow[mysql]' do
  version node['airflow']['core']['version']
end

I follow the official documentation : http://pythonhosted.org/airflow/installation.html
It works if I run manually : pip install airflow[mysql]

But at converge :

           Error executing action `install` on resource 'python_package[airflow[mysql]]'
           ================================================================================

           Chef::Exceptions::Package
           -------------------------
           No candidate version available for airflow[mysql]

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/airflow/recipes/default.rb

            12: python_package 'airflow[mysql]' do
            13:   version node['airflow']['core']['version']
            14: end
            15: 

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/airflow/recipes/default.rb:12:in `from_file'

           python_package("airflow[mysql]") do
             action :install
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             package_name "airflow[mysql]"
             version "1.6.1"
             timeout 900
             declared_type :python_package
             cookbook_name "airflow"
             recipe_name "default"
             parent_python python_runtime[2]
           end

       [2015-11-16T14:25:51+00:00] ERROR: python_package[airflow[mysql]] (bbc_airflow::default line 12) had an error: Chef::Exceptions::Package: No candidate version available for airflow[mysql]
       [2015-11-16T14:25:51+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Thanks

Add a provider to support installing from source

If you are using ubuntu and the :system provider (i.e. apt-get), you will be relegated to the version that is in the apt-get repositories. And the apt-get repositories are usually behind the latest. For instance, right now apt-cache policy python2.7 says 2.7.6 is the latest version, but 2.7.11 is actually the latest.

Installing from apt-get also has the downside of not being great at installing older versions. I believe if you wanted 2.7.4, you would have to find a repo with it and add it. And this cookbook doesn't do that (which it probably shouldn't).

I think installing from source solves this problem. It does have the downside of taking more time, but it'll give the ability to support a wider range of versions on ubuntu, i.e. any version that is on the python.org page. (The :scl provider looks like it would support a wider range of versions, but it doesn't support ubuntu and I'm not familiar enough with softwarecollections.org to know if it's easy to support ubuntu.)

So, I'm thinking a :source provider would do the trick. The recipe for installing from source is pretty straightforward. It'll just need the version/checksums added as parameters/attributes. The more time-consuming part would likely be the testing (particularly for someone like me who isn't an expert on chef testing, but that's partially why I want to do this).

ark 'python' do
  version '2.7.11'
  url 'https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz'
  action :install_with_make
end

Thoughts? If agreed, I will attempt to start adding the provider some time next week when I get time.

Installing Python3 Not Functional on Ubuntu 15.10

Using hashicorp's ubuntu 15.10 box with Vagrant as the driver, I have been unable to get this cookbook to install Python 3. It seems to always fail on the install step.

Currently I have the following defined in poise-python.rb:

node.default['poise-python']['install_python2'] = false
node.default['poise-python']['install_python3'] = true

And I receive the following when trying to include the recipe:

       Recipe: poise-python::default
         * python_runtime[3] action install
           * poise_languages_system[python] action install

             ================================================================================
             Error executing action `install` on resource 'poise_languages_system[python]'
             ================================================================================

             PoiseLanguages::Error
             ---------------------
             Package python would install 2.7.9-1, which does not match 3. Please set the package_name or package_version provider options.

             Cookbook Trace:
             ---------------
             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:192:in `block (4 levels) in patch_load_current_resource!'
             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:190:in `block (3 levels) in patch_load_current_resource!'
             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:189:in `tap'
             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:189:in `block (2 levels) in patch_load_current_resource!'
             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:172:in `block in run_package_action'
             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `each'

             /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:100:in `action_install'
             /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
             /tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

             Resource Declaration:
             ---------------------
       # In /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb

       32:         poise_languages_system options['package_name'] || system_package_name do
       33:           # Otherwise use the default install action.
       34:           action(:upgrade) if options['package_upgrade']
       35:           parent new_resource
       36:           # Don't pass true because we want the default computed behavior for that.
        37:           dev_package options['dev_package'] unless options['dev_package'] == true
       38:           dev_package_overrides dev_package_overrides
       39:           package_version options['package_version'] if options['package_version']
       40:           version options['version']
       41:         end
       42:       end

             Compiled Resource:
             ------------------
             # Declared in /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb:32:in `install_system_packages'

             poise_languages_system("python") do
        action [:install]
        retries 0

        default_guard_interpreter :default
        declared_type :poise_languages_system
        cookbook_name :"poise-python"
        parent # Declared in /tmp/kitchen/cookbooks/poise-python/recipes/default.rb:19:in `from_file'


        action [:install]
        retries 0
        retry_delay 2
        default_guard_interpreter :default
        declared_type :python_runtime
        cookbook_name :"poise-python"
        recipe_name "default"
        pip_version true
        setuptools_version true
        version "3"
        virtualenv_version true
        wheel_version true
             end

         version "3"
        package_name "python"
        dev_package "python-dev"
             end


           ================================================================================
           Error executing action `install` on resource 'python_runtime[3]'
           ================================================================================

           PoiseLanguages::Error
           ---------------------
       poise_languages_system[python] (/tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32) had an error: PoiseLanguages::Error: Package python would install 2.7.9-1, which does not match 3. Please set the package_name or package_version provider options.

           Cookbook Trace:
           ---------------
       /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:192:in `block (4 levels) in patch_load_current_resource!'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:190:in `block (3 levels) in patch_load_current_resource!'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:189:in `tap'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:189:in `block (2 levels) in patch_load_current_resource!'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:172:in `block in run_package_action'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `each'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:162:in `run_package_action'
           /tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb:100:in `action_install'
           /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
           /tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/poise-python/recipes/default.rb

            19: python_runtime '3' if node['poise-python']['install_python3']
            20: python_runtime '2' if node['poise-python']['install_python2']

           Compiled Resource:
           ------------------
       # Declared in /tmp/kitchen/cookbooks/poise-python/recipes/default.rb:19:in `from_file'

           python_runtime("3") do
             action [:install]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :python_runtime
             cookbook_name :"poise-python"
             recipe_name "default"
             pip_version true
             setuptools_version true
             version "3"
             virtualenv_version true
             wheel_version true
           end


       Running handlers:
       [2016-01-27T16:41:35+00:00] ERROR: Running exception handlers
       Running handlers complete


       [2016-01-27T16:41:35+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2016-01-27T16:41:35+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2016-01-27T16:41:35+00:00] ERROR: python_runtime[3] (poise-python::default line 19) had an error: PoiseLanguages::Error: poise_languages_system[python] (/tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32) had an error: PoiseLanguages::Error: Package python would install 2.7.9-1, which does not match 3. Please set the package_name or package_version provider options.
       [2016-01-27T16:41:35+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

I have tried various combinations of specifying different runtime names, system package names, and versions, but have been unable to get it to work. Specifying a runtime with version 3.4 gives the same error message. Specifying the package name python-3.4 for provider :system yields:

poise_languages_system[python-3.4] (/tmp/kitchen/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32) had an error: PoiseLanguages::Error: Package python-3.4 would install , which does not match 3. Please set the package_name or package_version provider options.

ImportError: No module named _vendor

I'm using the default python2.7 installation on Ubuntu 14.04 LTS

python_package 'mopidy-moped' fails with:

[2016-01-22T11:57:10+00:00] ERROR: python_package[mopidy-moped] (jackdaw::spotify line 81) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '2'
---- Begin output of ["/usr/bin/python", "-", "list", "--outdated", "mopidy-moped"] ----
STDOUT: Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 74, in run
    self.run_outdated(options)
  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 83, in run_outdated
    for dist, remote_version_raw, remote_version_parsed in self.find_packages_latests_versions(options):
  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 108, in find_packages_latests_versions
    for dist in get_installed_distributions(local_only=options.local, skip=self.skip):
  File "<stdin>", line 13, in replacement
ImportError: No module named _vendor

Storing debug log for failure in /home/vagrant/.pip/pip.log
STDERR: 
---- End output of ["/usr/bin/python", "-", "list", "--outdated", "mopidy-moped"] ----
Ran ["/usr/bin/python", "-", "list", "--outdated", "mopidy-moped"] returned 2

Yet on the command line, all is fine:

vagrant@vagrant-ubuntu-trusty:~$ pip --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
vagrant@vagrant-ubuntu-trusty:~$ pip --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
vagrant@vagrant-ubuntu-trusty:~$ sudo pip --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
vagrant@vagrant-ubuntu-trusty:~$ sudo pip list --outdated mopidy-moped
argparse (Current: 1.2.1 Latest: 1.4.0)
setuptools (Current: 3.3 Latest: 19.4)
chardet (Current: 2.0.1 Latest: 2.3.0)
ssh-import-id (Current: 3.21 Latest: 4.5)
six (Current: 1.5.2 Latest: 1.10.0)
cffi (Current: 1.1.2 Latest: 1.5.0)
ply (Current: 3.4 Latest: 3.8)
requests (Current: 2.2.1 Latest: 2.9.1)
wheel (Current: 0.24.0 Latest: 0.26.0)
pycurl (Current: 7.19.3 Latest: 7.21.5)
pip (Current: 1.5.4 Latest: 8.0.2)
pycparser (Current: 2.10 Latest: 2.14)
html5lib (Current: 0.999 Latest: 0.9999999)
colorama (Current: 0.2.5 Latest: 0.3.6)
tornado (Current: 3.1.1 Latest: 4.3)
python-debian (Current: 0.1.21-nmu2ubuntu2 Latest: 0.1.23)
Could not find any downloads that satisfy the requirement apt-xapian-index
MySQL-python (Current: 1.2.3 Latest: 1.2.5)
urllib3 (Current: 1.7.1 Latest: 1.14)
vagrant@vagrant-ubuntu-trusty:~$

Am I misinterpreting what command is being run here? I feel like I'm missing what the resource is trying to do. I can provide my Vagrantfile and cookbook etc. if anybody would like more information.

python_package should support all of pip's flags

pip's --editable flag, especially, would be useful, but even more "esoteric" ones like --process-dependency-links would be good as well.

The quickest impl I can think of would be like a pip_flags ["--editable", "--process-dependency-links"], but I guess you could make each one a proper attribute.

poise / poise-python Goto Github PK

poise-python's Introduction

Poise-Python Cookbook

Quick Start

Installing a Package From a URI

Supported Python Versions

Windows Support

Requirements

Attributes

Recipes

default

Resources

python_runtime

Actions

Properties

Provider Options

python_runtime_options

Actions

Properties

python_execute

Actions

Properties

python_package

Actions

Properties

python_virtualenv

Actions

Properties

pip_requirements

Actions

Properties

Python Providers

Common Options

system

Options

scl

portable_pypy

portable_pypy3

Options

deadsnakes

python-build

Upgrading from the python Cookbook

Sponsors

License

poise-python's People

Contributors

Stargazers

Watchers

Forkers

poise-python's Issues

Recommend Projects

Recommend Topics

Recommend Org

`default`

`python_runtime`

`python_runtime_options`

`python_execute`

`python_package`

`python_virtualenv`

`pip_requirements`

`system`

`scl`

`portable_pypy`

`portable_pypy3`

`deadsnakes`

`python-build`

Upgrading from the `python` Cookbook