Giter VIP home page Giter VIP logo

voxpupuli / puppet-python Goto Github PK

View Code? Open in Web Editor NEW
198.0 51.0 369.0 1.02 MB

Puppet module for installing and managing Python, pip, virtualenvs and Gunicorn virtual hosts.

Home Page: https://forge.puppetlabs.com/puppet/python

License: Apache License 2.0

Ruby 59.40% Puppet 38.72% HTML 1.54% Pascal 0.34%
linux-puppet-module puppet hacktoberfest bsd-puppet-module centos-puppet-module debian-puppet-module freebsd-puppet-module gentoo-puppet-module sles-puppet-module ubuntu-puppet-module

puppet-python's Introduction

puppet-python

Build Status Release Puppet Forge Puppet Forge - downloads Puppet Forge - endorsement Puppet Forge - scores puppetmodule.info docs License

Puppet module for installing and managing python, pip, virtualenvs and Gunicorn virtual hosts.

Please note: The module stankevich/python has been deprecated and is now available under Vox Pupuli: puppet/python.

Usage

For class usage refer to the Reference. If contributing, this is updated with

bundle exec rake strings:generate\[',,,,false,true']

hiera configuration

This module supports configuration through hiera. The following example creates two python3 virtualenvs. The configuration also pip installs a package into each environment.

python::python_pyvenvs:
  "/opt/env1":
    version: "system"
  "/opt/env2":
    version: "system"
python::python_pips:
  "nose":
    virtualenv: "/opt/env1"
  "coverage":
    virtualenv: "/opt/env2"
python::python_dotfiles:
  "/var/lib/jenkins/.pip/pip.conf":
    config:
      global:
        index-url: "https://mypypi.acme.com/simple/"
        extra-index-url: "https://pypi.risedev.at/simple/"

Using SCL packages from RedHat or CentOS

To use this module with Linux distributions in the Red Hat family and python distributions from softwarecollections.org, set python::provider to 'rhscl' and python::version to the name of the collection you want to use (e.g., 'python27', 'python33', or 'rh-python34').

Release Notes

See Changelog

Contributors

Check out Github contributors.

puppet-python's People

Contributors

acullenn avatar alexjfisher avatar bastelfreak avatar baurmatt avatar bodgit avatar brabiega avatar codesplicer avatar daniellawrence avatar dannygoulder avatar danquack avatar ekohl avatar feltra avatar fgimian avatar gene1wood avatar ghoneycutt avatar igalic avatar jradmacher avatar kenyon avatar mbmilligan avatar petems avatar root-expert avatar saz avatar shivapoudel avatar smortex avatar sorreltree avatar stankevich avatar steverecio avatar treydock avatar zilchms avatar zipkid avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppet-python's Issues

Invalid parameter allow_virtual

Version 1.7.14 has breaking changes for Puppet version 3.4.3.

These are the errors:
Warning: Could not load fact file /etc/puppet/modules/python/lib/facter/pip_version.rb: Invalid parameter allow_virtual
Warning: Could not load fact file /etc/puppet/modules/python/lib/facter/python_version.rb: Invalid parameter allow_virtual

These appear to be the offending commits:
52f56f6
c5d94b7

Reverting to 1.7.13 resolves the issue.

python::pip doesn't find pip command

When I try to use python::pip without any parameter, I have this error :

Error: Parameter unless failed on Exec[pip_install_collective.zopeedit]: 'pip freeze | grep -i -e ^collective.zopeedit==' is not qualified and no path was specified. Please qualify the command or specify a path. at /etc/puppet/modules/python/manifests/pip.pp:87
Wrapped exception:
'pip freeze | grep -i -e ^collective.zopeedit==' is not qualified and no path was specified. Please qualify the command or specify a path.

If I change line 61 of pip.pp by :

'system' => '/usr/bin/pip',

It works.
Do you have an idea about why puppet doesn't find pip ?

pip => true, but no python-pip installed on CentOS 6.5

I'm not 100% sure I'm doing this right, but I am getting no errors, just python-pip is not getting installed and I am not sure why.

In Puppetfile

mod 'epel',
  :git => 'https://github.com/stahnma/puppet-module-epel.git',
  :ref => '1.0.0'
mod 'python',
  :git => 'https://github.com/stankevich/puppet-python.git',
  :ref => '1.7.9'

In my module's pp file...

include epel

class { 'python' :
    version    => 'system',
    pip        => true,
    dev        => false,
    virtualenv => false,
    gunicorn   => false,
    require    => Class['epel'],
}

It's probably something dumb I'm doing.

python::pip - empty pkgname is silently ignored

Trying this module out for the first time I wanted to use the simplest definition possible to install a module with pip.
So I used:
python::pip { 'requests':
ensure => latest,
}
The catalog compiled without complaining, but the result wasn't as expected.
Then I noticed that "pip install --upgrade" was in fact executed, but with empty package name, e.g.:

debug: Executing 'pip wheel --help > /dev/null 2>&1 && { pip wheel --version > /dev/null 2>&1 || wheel_support_flag='--no-use-wheel'; } ; pip --log //pip.log install --upgrade $wheel_support_flag '

So I realized the right way to do it was:
python::pip { 'requests':
pkgname => 'requests',
ensure => latest,
}
which works.

Shouldn't $pkgname be evaluated and forced to be a string? e.g. with something like stdlib::validate_string()?
Thanks.

Wheel support?

Hi,

Just curious - what's the Wheel support like on puppet-python?

Should we just patch our pip, and do it via that? Or is there a more recommended way?

Cheers,
Victor

virtualenv / systempkgs and fact precedence

On Centos 6, in a master/agent environment with this manifest

class { '::python':
        dev         => true,
        virtualenv  => true,
        gunicorn    => false
    }

    ->

    python::virtualenv { '/opt/python-xxx':
        ensure       => 'present',
        requirements => '/etc/xxx/requirements.txt',
        systempkgs   => true,
        owner        => 'root',
        group        => '0',
        cwd          => '/opt/python-xxx',
    }

executed on a machine without any previous installation of virtualenv, it seems that the fact virtualenv_version is empty (no way to get the virtualenv version since virtualenv is not yet installed when the fact is evaluated).

On Centos 6, this leads to the creation of a virtualenv without systempkgs. Since the virtualenv is created on the first puppet pass, it will never create the virtualenv with systempkgs.

Can not install the same packages in different virtualenvs

If I create two virtualenvs, say A and B, and use python::pip to install gunicorn in both it will fail with this error:
Error 400 on SERVER: Duplicate declaration: Python::Pip[gunicorn] is already declared in file [.....]

This somewhat defeats the purpose of virtualenvs and it would be really nice if the same package could be installed in more than one virtualenv.

virtualenv broken for python3

The ppa:fkrull/deadsnakes repository provides new python packages, I am using it to install python3.3

The python version in virtualenv definition does not seem to have any effect, as it is always created with the default python for the 'virtualenv' command. pip is not available for python3 either.

In addition, python3+ provides its own virtualenv โ€”pyvenvโ€”, which should be used.

Resource failed with ArgumentError

I'm trying to user this module (which seems very promising) in my project.
the module is installed with git submodules in modules/puppet-python folder.
but when I try to use it, even with a configuration exactly like /test/init.pp manifest:

class{'python':
    version => 'system',
    dev => true,
    virtualenv => true  
}

And getting this error:

Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find de
clared class python at D:/alon/workspace/Oknesset-vagrant/manifests/test.pp:5 on..

any clues on what I'm doing wrong?

IOError: [Errno 26] Text file busy

I am using this class in my manifest:

python::virtualenv { $virtualenv_dir:
ensure => present,
version => 'system',
systempkgs => false,
distribute => false,
owner => $user,
group => $group,
}

It works great the first time.

The problem is I have several daemonized celery workers launched which are using the venv/bin/python file.

When I apply the manifest second time, I get:

Already using interpreter /usr/bin/python
Traceback (most recent call last):
File "/usr/bin/virtualenv", line 3, in
virtualenv.main()
File "/usr/lib/python2.7/dist-packages/virtualenv.py", line 938, in main
never_download=options.never_download)
File "/usr/lib/python2.7/dist-packages/virtualenv.py", line 1039, in create_environment
site_packages=site_packages, clear=clear))
File "/usr/lib/python2.7/dist-packages/virtualenv.py", line 1274, in install_python
shutil.copyfile(executable, py_executable)
File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 26] Text file busy: '/usr/local/myapp/venv/bin/python'

Is it somehow possible to only run virtualenv command if the venv directory has not been created yet?

Or what would you suggest? I don't want to stop my celery workers while applying the puppet manifest.

Need a better way to deal with wheels when using pip

We have some problems when dealing with wheels and pip.

Pip does not support wheels prior to version 1.4.0
Pip wheels require setuptools/distribute > 0.8
Python 2.6 and older does not support setuptools/distribute > 0.8
Pip >= 1.5.0 tries to use wheels by default, even if wheel package is not
installed, in this case the --no-use-wheel flag needs to be passed
Versions prior to 1.4 don't support the --no-use-wheel flag

So depending on the version of pip is installed and if wheel support is available we now have to:

< 1.4.0 no flag

= 1.4.0 and < 1.5 ideally pass the use wheel flag if wheel support is installed but no flag is acceptable
= 1.5.0 pass the --no-use-wheel flag if wheel support isn't available, otherwise pass no flag

I started working on this on January 2nd when 1.5 was released but I'm still running into issues for some pip versions/cases.

Right now I'm using the following command but there are some cases where it does not work as it should, we need a better way of dealing with this.

"${pip_env} wheel --help > /dev/null 2>&1 && { ${pip_env} wheel --version > /dev/null 2>&1 || wheel_support_flag='--no-use-wheel'; } ; ${pip_env} --log ${cwd}/pip.log install ${install_args} $wheel_support_flag ${proxy_flag} ${source}

Manifests using the module aren't testable in windows

When I run puppet test on my manifests using puppet-python:

puppet apply --noop --modulepath=modules manifests\default.pp

I just need to verify that puppet builds the correct catalog etc. but instead I get a Module is not compatible with windows error.

This is a common scenerio when using vagrant+puppet on windows to provision a virtual linux machine.
The problem comes from this code in init.pp:

  $compatible = [ 'Debian', 'Ubuntu', 'CentOS', 'RedHat' ]
  if ! ($::operatingsystem in $compatible) {
    fail("Module is not compatible with ${::operatingsystem}")
  }

Let me make my self clear This is a complete valid failure when running puppet apply on windows without the --noop flag.
But this shouldn't fail on a test scenerio. Maybe when --noop is on puppet arguments the module should notify but not fail.

Dupilicate declaration for requirement file

I try to do below steps:

  1. Get the requirement file from master server, and save to /tmp/ directory
  2. Create the virtualenv and install package with the requirement file which has been saved in /tmp directory.

But when I try to so, there is always an error appeared, it shows as below:
Error: Failed to apply catalog: Duplicate declaration: File[/tmp/requirement.txt] is already declared in file /etc/puppet/modules/general/manifests/init.pp:7; cannot redeclare at /etc/puppet/modules/python/manifests/requirements.pp:98

it means "path parameter" of file resource and "requirements parameter" of python::virtualenv cannot be same, but this is not make sense, please fix it.

Manage compilers?

Hi there,

when you use virtualenvs, it might be necessary to compile native extensions for python inside it (e.g. numpy/matplotlib).

Would it be of interest to manage installation of gcc/g++/gfortran for the virtualenv?

I would implement such thing, as I need it myself... ;)

Gunicorn does not allow passing in a list of environment variables

The gunicorn module does not allow passing in a list of environment variables. It would be great to pass in an array or hash with custom environment variables that the gunicorn user will run with. This can allow some python apps to be configured without settings files or allow custom settings files to be passed in.

Ex. https://github.com/puppet-community/puppetboard#settings will read the os.env var PUPPETBOARD_SETTINGS=/some/file/path/settings.py to allow setting an alternait settings file. This allows for customization of the puppetboard service with out editing the built in settings file or managing a settings.py file in the directory where pip installed puppetboard.

Specify package versions

I'd like to be able to install specific versions of pip packages via your module. I'm not finding any documentation on how exactly to do this. Is this no possible, or is this something you have thought about for future revisions?

If virtualenv isn't installed, it isn't properly detected or installed.

A few bugs from the recent changes that use Facter to locate versions...

Could not retrieve virtualenv_version: can't convert Symbol into String
Could not retrieve virtualenv_version: can't convert Symbol into String
Python 2.7.4
Could not retrieve python_version: undefined method `[]' for nil:NilClass
Python 2.7.4
Could not retrieve python_version: undefined method `[]' for nil:NilClass

Error: /Stage[main]/Lint/Python::Virtualenv[/srv/lint/]/Exec[python_virtualenv_/srv/lint/]/returns: change from notrun to 0 failed: sh: 1: virtualenv: not found

Manage Python related .dotfiles (for a user)

There are a handful of Python related dotfiles, which don't seem to have a global option in /etc (or if they do, it's not documented)

In our very old fork of this module, we have a defined type python::dotfiles, which manages:

  • .pypirc
  • .pydistutils.cfg
  • .pip
  • .pip/pip.conf

The main reason why we need this is because so we can direct our jenkins builds against our internal PyPy repository.

Is there any interest for such an extension to this module?

Gunicorn timeout hardcoded in template

The python::gunicorn timeout variable is hardcoded into the ERB template. It would be nice to be able to set the default to the hardcoded "30" and allow it to be overridden.

Facter scripts raise warnings

On nodes that have python, virtualenv or pip not installed, the newly added facter ruby scripts give rise to warnings like:

Could not retrieve pip_version: can't convert Symbol into String
Could not retrieve virtualenv_version: can't convert Symbol into String

Upload to the forge

Maybe I'm missing something but I can't seem to find this on the puppet forge, which means I can't depend on it from within my module.

Could not match $(ensure at pip.pp:104

class { 'python' :
    version    => 'system',
    pip        => true,
    dev        => false,
    virtualenv => false,
    gunicorn   => false,
    provider   => 'pip',
    require    => Class['epel'],
}

Trying the above I get the below

puppet agent -t --environment ec2tools --debug
...
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not match $(ensure at /etc/puppetlabs/puppet/environments/ec2tools/modules/python/manifests/pip.pp:104

Remove redundant pkgname

Pkgname appears to be redundant to the normal name (python::pip { 'pkg': pkgname => 'pkg' }).

Also, requiring a pkgname means dropping support for syntax like python::pip { [ 'pkg', 'pkg', 'pkg']: ensure => present }.

Could not retrieve pip_version: undefined method

Hi and thank you for the great module! I am experiencing a warning across all nodes after installing the latest version of your module. What I am seeing is the following:

On a puppet run:

[root@hostname ~]# puppet agent --test
info: Retrieving plugin
info: Loading facts in /var/lib/puppet/lib/facter/rvm_installed.rb
info: Loading facts in /var/lib/puppet/lib/facter/pip_version.rb
info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
info: Loading facts in /var/lib/puppet/lib/facter/virtualenv_version.rb
info: Loading facts in /var/lib/puppet/lib/facter/python_version.rb
info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Could not retrieve pip_version: undefined method `[]' for nil:NilClass
Could not retrieve pip_version: undefined method `[]' for nil:NilClass
info: Caching catalog for hostname.example.com
info: Applying configuration version '1393955494'
notice: /Stage[main]/Motd/Exec[update-motd]/returns: executed successfully
notice: Finished catalog run in 11.01 seconds

Running facter manually:

[root@hostname ~]# facter -p virtualenv_version pip_version
Could not retrieve pip_version: undefined method `[]' for nil:NilClass
pip_version => 
virtualenv_version => 1.10.1

What I have in the node manifest:

  class { 'python':
    version     => 'system',
    dev         => true,
    virtualenv  => true,
  }

Information about the system:

[root@hostname ~]# cat /etc/issue
Scientific Linux release 6.5 (Carbon)
Kernel \r on an \m

[root@hostname ~]# rpm -qa | grep python
python-gudev-147.1-4.el6_0.1.x86_64
python-ldap-2.3.10-1.el6.x86_64
python-lxml-2.2.3-1.1.el6.x86_64
python-dmidecode-3.10.13-3.el6_4.x86_64
python-hwdata-1.7.3-1.el6.noarch
uwsgi-plugin-python-2.0-1.el6.x86_64
python-devel-2.6.6-52.el6.x86_64
python-setuptools-0.6.10-3.el6.noarch
python-virtualenv-1.10.1-1.el6.noarch
python-virtualenvwrapper-3.5-2.el6.noarch
newt-python-0.52.11-3.el6.x86_64
python-simplejson-2.0.9-3.1.el6.x86_64
python-iniparse-0.3.1-2.1.el6.noarch
python-paste-1.7.4-2.el6.noarch
python-kerberos-1.1-6.2.el6.x86_64
python-pycurl-7.19.0-8.el6.x86_64
python-krbV-1.0.90-3.el6.x86_64
python-netaddr-0.7.5-4.el6.noarch
python-2.6.6-52.el6.x86_64
dbus-python-0.83.0-6.1.el6.x86_64
libipa_hbac-python-1.9.2-129.el6_5.4.x86_64
ipa-python-3.0.0-37.el6.x86_64
libxml2-python-2.7.6-14.el6.x86_64
rpm-python-4.8.0-37.el6.x86_64
python-imaging-1.1.6-19.el6.x86_64
python-virtualenv-clone-0.2.4-1.el6.noarch
python-nss-0.13-1.el6.x86_64
python-urlgrabber-3.9.1-9.el6.noarch
python-ethtool-0.6-5.el6.x86_64
libselinux-python-2.0.94-5.3.el6_4.1.x86_64
python-libs-2.6.6-52.el6.x86_64

[root@hostname ~]# puppet --version
2.7.23

Please let me know if there is any further information I can provide.

Unable to customize `APP_MODULE` variable in gunicorn template

Some python projects do not use the application module name of app:app that is hard coded into the template. Should add a variable that defaults to app:app but allows people to pass in a different module name if needed.

Ex: https://github.com/nedap/puppetboard#nginx-gunicorn which uses puppetboard.app:app

Per the gunicorn docs (http://gunicorn-docs.readthedocs.org/en/latest/run.html), this follows the format of $(MODULE_NAME):$(VARIABLE_NAME).

Could not start Service[gunicorn]

Getting an gunicorn error with the following puppet script. Using Precise Ubuntu 12.04 LTS with via vagrant.

class { 'python':
  version    => 'system',
  pip => true,
  virtualenv => true,
  gunicorn   => true,
}

Here is the verbose log output.

bcapa55-208-dhcp:puppet-techstack cevaris$ vagrant provision
==> alpha: Installing Puppet modules with Librarian-Puppet...
==> alpha: Running provisioner: puppet...
Running Puppet with web.pp...
stdin: is not a tty
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/concat/lib/facter/concat_basedir.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/firewall/lib/facter/ip6tables_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/firewall/lib/facter/iptables_persistent_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/firewall/lib/facter/iptables_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/python/lib/facter/pip_version.rb
debug: Puppet::Type::Package::ProviderNim: file /usr/sbin/nimclient does not exist
debug: Puppet::Type::Package::ProviderPortage: file /usr/bin/emerge does not exist
debug: Puppet::Type::Package::ProviderAptrpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderUrpmi: file urpmi does not exist
debug: Puppet::Type::Package::ProviderFreebsd: file /usr/sbin/pkg_info does not exist
debug: Puppet::Type::Package::ProviderPkg: file /usr/bin/pkg does not exist
debug: Puppet::Type::Package::ProviderFink: file /sw/bin/fink does not exist
debug: Puppet::Type::Package::ProviderAix: file /usr/bin/lslpp does not exist
debug: Puppet::Type::Package::ProviderHpux: file /usr/sbin/swinstall does not exist
debug: Puppet::Type::Package::ProviderSun: file /usr/bin/pkginfo does not exist
debug: Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-nox does not exist
debug: Puppet::Type::Package::ProviderSunfreeware: file pkg-get does not exist
debug: Puppet::Type::Package::ProviderYum: file yum does not exist
debug: Puppet::Type::Package::ProviderPorts: file /usr/local/sbin/pkg_deinstall does not exist
debug: Puppet::Type::Package::ProviderZypper: file /usr/bin/zypper does not exist
debug: Puppet::Type::Package::ProviderRug: file /usr/bin/rug does not exist
debug: Puppet::Type::Package::ProviderPortupgrade: file /usr/local/sbin/pkg_deinstall does not exist
debug: Puppet::Type::Package::ProviderRpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderPacman: file /usr/bin/pacman does not exist
debug: Puppet::Type::Package::ProviderOpenbsd: file pkg_info does not exist
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/python/lib/facter/python_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/python/lib/facter/virtualenv_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/facter_dot_d.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/pe_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/puppet_vardir.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/root_home.rb
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat ${Status} ${Package} ${Version}\n python'
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/init.pp' in environment production
debug: Automatically imported python from python into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/install.pp' in environment production
debug: Automatically imported python::install from python/install into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/config.pp' in environment production
debug: Automatically imported python::config from python/config into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/pip.pp' in environment production
debug: Automatically imported python::pip from python/pip into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/requirements.pp' in environment production
debug: Automatically imported python::requirements from python/requirements into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/virtualenv.pp' in environment production
debug: Automatically imported python::virtualenv from python/virtualenv into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/gunicorn.pp' in environment production
debug: Automatically imported python::gunicorn from python/gunicorn into production
debug: Adding relationship from Class[Python::Install] to Class[Python::Config] with 'before'
debug: Puppet::Type::Service::ProviderSystemd: file /bin/systemctl does not exist
debug: Puppet::Type::Service::ProviderLaunchd: file /bin/launchctl does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
debug: Puppet::Type::Service::ProviderOpenrc: file /sbin/rc-service does not exist
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not exist
debug: Creating default schedules
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Failed to load library 'shadow' for feature 'libshadow'
debug: Puppet::Type::User::ProviderPw: file pw does not exist
debug: Puppet::Type::User::ProviderLdap: true value when expecting false
debug: Puppet::Type::User::ProviderUser_role_add: file roleadd does not exist
debug: Puppet::Type::User::ProviderDirectoryservice: file /usr/bin/dscl does not exist
debug: Failed to load library 'ldap' for feature 'ldap'
debug: /File[/etc/puppet/ssl/private_keys]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/run]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl/private]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/state/last_run_summary.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/clientbucket]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl/certs]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/client_yaml]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/state/state.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/state/graphs]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/state/last_run_report.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/etc/puppet/ssl/certificate_requests]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/client_data]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/log]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl/public_keys]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl]: Autorequiring File[/etc/puppet]
debug: Finishing transaction 70026358272520
debug: Loaded state in 0.00 seconds
debug: Loaded state in 0.00 seconds
info: Applying configuration version '1398793070'
debug: /Stage[main]/Python::Install/before: requires Class[Python::Config]
debug: /Schedule[daily]: Skipping device resources because running on a host
debug: /Schedule[monthly]: Skipping device resources because running on a host
debug: /Schedule[hourly]: Skipping device resources because running on a host
debug: /Schedule[never]: Skipping device resources because running on a host
debug: /Schedule[weekly]: Skipping device resources because running on a host
debug: Prefetching apt resources for package
debug: Executing '/usr/bin/dpkg-query -W --showformat '${Status} ${Package} ${Version}\n''
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat '${Status} ${Package} ${Version}\n''
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat ${Status} ${Package} ${Version}\n python-dev'
debug: Service[gunicorn](provider=upstart): Could not find gunicorn.conf in /etc/init
debug: Service[gunicorn](provider=upstart): Could not find gunicorn.conf in /etc/init.d
debug: Service[gunicorn](provider=upstart): Could not find gunicorn in /etc/init
debug: Service[gunicorn](provider=upstart): Executing 'ps -ef'
debug: Service[gunicorn](provider=upstart): Executing '/etc/init.d/gunicorn start'
err: /Stage[main]/Python::Config/Service[gunicorn]/ensure: change from stopped to running failed: Could not start Service[gunicorn]: Execution of '/etc/init.d/gunicorn start' returned 1:  at /tmp/vagrant-puppet-2/modules-0/python/manifests/config.pp:36
debug: /Schedule[puppet]: Skipping device resources because running on a host
debug: Finishing transaction 70026357405600
debug: Storing state
debug: Stored state in 0.01 seconds
notice: Finished catalog run in 0.34 seconds
debug: Finishing transaction 70026358894500
debug: Received report to process from alpha.int.colorado.edu
debug: Processing report from alpha.int.colorado.edu with processor Puppet::Reports::Store
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

puppet apply --verbose --debug --modulepath '/tmp/vagrant-puppet-2/modules-0:/etc/puppet/modules' --manifestdir /tmp/vagrant-puppet-2/manifests --detailed-exitcodes /tmp/vagrant-puppet-2/manifests/web.pp || [ $? -eq 2 ]

Stdout from the command:

info: Loading facts in /tmp/vagrant-puppet-2/modules-0/concat/lib/facter/concat_basedir.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/firewall/lib/facter/ip6tables_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/firewall/lib/facter/iptables_persistent_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/firewall/lib/facter/iptables_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/python/lib/facter/pip_version.rb
debug: Puppet::Type::Package::ProviderNim: file /usr/sbin/nimclient does not exist
debug: Puppet::Type::Package::ProviderPortage: file /usr/bin/emerge does not exist
debug: Puppet::Type::Package::ProviderAptrpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderUrpmi: file urpmi does not exist
debug: Puppet::Type::Package::ProviderFreebsd: file /usr/sbin/pkg_info does not exist
debug: Puppet::Type::Package::ProviderPkg: file /usr/bin/pkg does not exist
debug: Puppet::Type::Package::ProviderFink: file /sw/bin/fink does not exist
debug: Puppet::Type::Package::ProviderAix: file /usr/bin/lslpp does not exist
debug: Puppet::Type::Package::ProviderHpux: file /usr/sbin/swinstall does not exist
debug: Puppet::Type::Package::ProviderSun: file /usr/bin/pkginfo does not exist
debug: Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-nox does not exist
debug: Puppet::Type::Package::ProviderSunfreeware: file pkg-get does not exist
debug: Puppet::Type::Package::ProviderYum: file yum does not exist
debug: Puppet::Type::Package::ProviderPorts: file /usr/local/sbin/pkg_deinstall does not exist
debug: Puppet::Type::Package::ProviderZypper: file /usr/bin/zypper does not exist
debug: Puppet::Type::Package::ProviderRug: file /usr/bin/rug does not exist
debug: Puppet::Type::Package::ProviderPortupgrade: file /usr/local/sbin/pkg_deinstall does not exist
debug: Puppet::Type::Package::ProviderRpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderPacman: file /usr/bin/pacman does not exist
debug: Puppet::Type::Package::ProviderOpenbsd: file pkg_info does not exist
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/python/lib/facter/python_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/python/lib/facter/virtualenv_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/facter_dot_d.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/pe_version.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/puppet_vardir.rb
info: Loading facts in /tmp/vagrant-puppet-2/modules-0/stdlib/lib/facter/root_home.rb
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat ${Status} ${Package} ${Version}\n python'
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/init.pp' in environment production
debug: Automatically imported python from python into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/install.pp' in environment production
debug: Automatically imported python::install from python/install into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/config.pp' in environment production
debug: Automatically imported python::config from python/config into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/pip.pp' in environment production
debug: Automatically imported python::pip from python/pip into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/requirements.pp' in environment production
debug: Automatically imported python::requirements from python/requirements into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/virtualenv.pp' in environment production
debug: Automatically imported python::virtualenv from python/virtualenv into production
debug: importing '/tmp/vagrant-puppet-2/modules-0/python/manifests/gunicorn.pp' in environment production
debug: Automatically imported python::gunicorn from python/gunicorn into production
debug: Adding relationship from Class[Python::Install] to Class[Python::Config] with 'before'
debug: Puppet::Type::Service::ProviderSystemd: file /bin/systemctl does not exist
debug: Puppet::Type::Service::ProviderLaunchd: file /bin/launchctl does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
debug: Puppet::Type::Service::ProviderOpenrc: file /sbin/rc-service does not exist
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not exist
debug: Creating default schedules
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Failed to load library 'shadow' for feature 'libshadow'
debug: Puppet::Type::User::ProviderPw: file pw does not exist
debug: Puppet::Type::User::ProviderLdap: true value when expecting false
debug: Puppet::Type::User::ProviderUser_role_add: file roleadd does not exist
debug: Puppet::Type::User::ProviderDirectoryservice: file /usr/bin/dscl does not exist
debug: Failed to load library 'ldap' for feature 'ldap'
debug: /File[/etc/puppet/ssl/private_keys]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/run]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl/private]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/state/last_run_summary.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/clientbucket]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl/certs]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/client_yaml]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/state/state.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/state/graphs]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/state/last_run_report.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/etc/puppet/ssl/certificate_requests]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/client_data]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/log]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl/public_keys]: Autorequiring File[/etc/puppet/ssl]
debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/puppet]
debug: /File[/etc/puppet/ssl]: Autorequiring File[/etc/puppet]
debug: Finishing transaction 70026358272520
debug: Loaded state in 0.00 seconds
debug: Loaded state in 0.00 seconds
info: Applying configuration version '1398793070'
debug: /Stage[main]/Python::Install/before: requires Class[Python::Config]
debug: /Schedule[daily]: Skipping device resources because running on a host
debug: /Schedule[monthly]: Skipping device resources because running on a host
debug: /Schedule[hourly]: Skipping device resources because running on a host
debug: /Schedule[never]: Skipping device resources because running on a host
debug: /Schedule[weekly]: Skipping device resources because running on a host
debug: Prefetching apt resources for package
debug: Executing '/usr/bin/dpkg-query -W --showformat '${Status} ${Package} ${Version}\n''
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat '${Status} ${Package} ${Version}\n''
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat ${Status} ${Package} ${Version}\n python-dev'
debug: Service[gunicorn](provider=upstart): Could not find gunicorn.conf in /etc/init
debug: Service[gunicorn](provider=upstart): Could not find gunicorn.conf in /etc/init.d
debug: Service[gunicorn](provider=upstart): Could not find gunicorn in /etc/init
debug: Service[gunicorn](provider=upstart): Executing 'ps -ef'
debug: Service[gunicorn](provider=upstart): Executing '/etc/init.d/gunicorn start'
err: /Stage[main]/Python::Config/Service[gunicorn]/ensure: change from stopped to running failed: Could not start Service[gunicorn]: Execution of '/etc/init.d/gunicorn start' returned 1:  at /tmp/vagrant-puppet-2/modules-0/python/manifests/config.pp:36
debug: /Schedule[puppet]: Skipping device resources because running on a host
debug: Finishing transaction 70026357405600
debug: Storing state
debug: Stored state in 0.01 seconds
notice: Finished catalog run in 0.34 seconds
debug: Finishing transaction 70026358894500
debug: Received report to process from alpha.int.colorado.edu
debug: Processing report from alpha.int.colorado.edu with processor Puppet::Reports::Store


Stderr from the command:

stdin: is not a tty

bcapa55-208-dhcp:puppet-techstack cevaris$ 

python::requirements interferes with managing requirements.txt if not explicitly in a file resource

Hello, thanks for this super-useful module. I'd like to make a change to this block here
https://github.com/stankevich/puppet-python/blob/0277b9c81f5838dce9353a0a87a7029b7ebcf397/manifests/requirements.pp#L92
...but have some questions about how to best solve it.

Here's the situation: we deploy a package ("coolstuff") that has its own requirements.txt, like so:

file { "/srv/coolstuff":
    ensure  => directory,
    recurse => true,
    source  => "puppet:///modules/coolstuff/coolstuff",
}

python::virtualenv { "/srv/coolstuff_venv":
    requirements => "/srv/coolstuff/requirements.txt",
    require      => File["/srv/coolstuff"],
}

Result: the block in requirements.pp, which as linked above has "replace => false", locks the requirements.txt down at whatever version first appeared. So if we update the file in the coolstuff module, that newer version never gets deployed.

The workaround is to make it explicit that we manage the file ourselves, ie we add:

file { "/srv/coolstuff/requirements.txt":
    source  => "puppet:///modules/coolstuff/coolstuff/requirements.txt",
}

but obviously that's a bit ugly.

Would there be any other way to detect that requirements.txt is managed elsewhere, or should I rather add an attribute "manage_requirements (true/false)" to python::requirements (and also to python::virtualenv) to prevent this manually?

(Frankly, I don't really see the use case where python::requirements should ever manage requirements.txt, so as far as I'm concerned, it should be fine if that whole block, L83-94, goes out altogether?)

Need sudo to install pip

I'm trying to test this module in Vagrant with CentOS6.5 provided by this box. It seems I'm unable to install pip via this module as it requires sudo.

err: /Stage[main]/Python::Install/Package[python-pip]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install python-pip' returned 1: Error: Nothing to do

Here is the entry in my node's definition.

class { 'python':
    version    => 'system',
    pip        => true,
    dev        => false,
    virtualenv => false,
    gunicorn   => false,
  }

It seems like the installation command just requires sudo prepended and could possibly be a configuration parameter? If I'm missing something obvious please let me know.

EDIT: I'm running puppet agent manually as the vagrant user.

Resource order restrictions?

Hi there,

I'm bootstrapping a VM with a precise32 box that will contain a django 1.6.2 project that's wrapped by virtualenv and running on gunicorn. I do this using Vagrant to boostrap the VM and then puppet to provision it with this module.

However, I'm getting the following error:

err: /Stage[main]//Python::Gunicorn[vhost]/File[/etc/gunicorn.d/vhost]/ensure: change from absent to present failed: Could not set 'present on ensure: No such file or directory - /etc/gunicorn.d/vhost.puppettmp_7493 at /tmp/vagrant-puppet-1/modules-0/python/manifests/gunicorn.pp:74
notice: /Stage[main]//Python::Requirements[/vagrant/project/requirements.txt]/File[/vagrant/project/requirements.txt]/content: audit change: newly-recorded value {md5}360d2304abc12e709a8c4bbbaf202434
err: /Stage[main]//Python::Virtualenv[/vagrant/project]/Exec[python_virtualenv_/vagrant/project]/returns: change from notrun to 0 failed: sh: 1: virtualenv: not found
sh: 1: /vagrant/project/bin/pip: not found
sh: 1: /vagrant/project/bin/pip: not found

Here is my manifest:

# --- Setup Python, VirtualEnv, Django, and Gunicorn -----------------------------------------------------------------
class { "python":
  version   => "2.7",
  dev       => "true",
  virtualenv  => "true",
  gunicorn    => "true",
  pip         => "true",
}

python::virtualenv { "/vagrant/project":
  ensure  => "present",
  version => "2.7",
  requirements => "/vagrant/project/requirements.txt",
  systempkgs => "true",
  owner => "vagrant",
  group => "vagrant"
}

python::requirements { "/vagrant/project/requirements.txt":
  virtualenv => "/vagrant/project",
  owner => "vagrant",
  group => "vagrant",
}

python::gunicorn { "vhost":
  ensure => "present",
  virtualenv => "/vagrant/project",
  mode       => "wsgi",
  dir        => "/vagrant/project",
  bind        => "unix:/tmp/gunicorn.socket",
  template    => "python/gunicorn.erb"
}

Not really sure what I'm doing wrong here. Any ideas?

Installing pip module via github

I'm new to this module and I'm trying to install fileconveyor.

This is the command I am trying to replace with the puppet module:

sudo pip install -e git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor

This is my puppet module:

class { 'python':
      pip   => true
    }

  python::pip { 'fileconveyor':
      pkgname       => 'fileconveyor',
      install_args  => ['-e git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor']
    }

But I am getting the following error. What am I doing wrong? Thanks

Notice: /Stage[main]/Fileconveyor::Package/Python::Pip[fileconveyor]/Exec[pip_install_fileconveyor]/returns: Double requirement given: fileconveyor from git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor (already in fileconveyor, name='fileconveyor')
Notice: /Stage[main]/Fileconveyor::Package/Python::Pip[fileconveyor]/Exec[pip_install_fileconveyor]/returns: Storing debug log for failure in /root/.pip/pip.log
Error: pip wheel --help > /dev/null 2>&1 && { pip wheel --version > /dev/null 2>&1 || wheel_support_flag='--no-use-wheel'; } ; pip --log //pip.log install -e git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor $wheel_support_flag  fileconveyor returned 1 instead of one of [0]
Error: /Stage[main]/Fileconveyor::Package/Python::Pip[fileconveyor]/Exec[pip_install_fileconveyor]/returns: change from notrun to 0 failed: pip wheel --help > /dev/null 2>&1 && { pip wheel --version > /dev/null 2>&1 || wheel_support_flag='--no-use-wheel'; } ; pip --log //pip.log install -e git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor $wheel_support_flag  fileconveyor returned 1 instead of one of [0]

Multiple python versions

Hi!

We have a mixed environment where python 2 and python 3 are used at the same time.

We did a try declaring the class twice, but we get this error:

Duplicate declaration: Class[Python] is already declared in file ...

And also we tried to declare the python class passing a list of versions:

   class { 'python':
        version => ['2.7', '3.4'],
        pip => true,
        dev => true,
        virtualenv => true,
        gunicorn => false,
    }

But it did not help.

So my question is: It is possible to use this module for provide multiple versions of python at the same time?

Thank you!

python::pip Specifying a local path in url fails

Assuming a python package is available locally at /path/to/package, then this attempt to install it with pip fails:

python::pip { 'package':
  url => '/path/to/package'
}

This snippet in pip.pp:

$source = $url ? {
  false => $pkgname,
  default => "${url}#egg=${egg_name}",
}

causes explicitly specified local urls to have #egg=... appended to them, which causes the install to fail.

It can be worked around by not specifying a url, e.g.:

python::pip{ '/path/to/package': }

but this isn't intuitive.

Problem with Package defaults warning

as of puppet 3.6.1 there is a deprecation warning.
https://docs.puppetlabs.com/puppet/3.6/reference/release_notes.html#changes-to-rpm-behavior-with-virtual-packages

When using this puppet-python module setting this does not remove the warning as it should.

This is caused by the creation of a package resource in the 3 facts included in the module.
https://github.com/stankevich/puppet-python/blob/master/lib/facter/pip_version.rb#L4
pkg = Puppet::Type.type(:package).new(:name => "python-pip")

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.