Giter VIP home page Giter VIP logo

puppet-agent's Introduction

The Puppet Agent

  • Overview
  • Runtime requirements
  • Building puppet-agent
  • Branches in puppet-agent
  • License
  • Code Owners
  • Running Tests

Overview

The puppet agent is a collection of software that is required for puppet and its dependencies to run. This includes puppet, facter, and other Puppet software, but also vendored dependencies like ruby, curl, openssl, and more.

This repository contains configuration to build puppet-agent and the facter gem for all of Puppet's supported platforms using vanagon, a packaging utility.

The full list of software components built into the puppet agent and the facter gem can be found in their project definitions, and each of the components has its own configuration in the components directory.

Components that are not developed by Puppet (like ruby, curl, or openssl) are built separately into a tarball and consumed here in the puppet-runtime component. See the puppet-runtime project for more information and a full list of the vendored dependencies it provides.

pxp-agent is built separately into a tarball and consumed here in the pxp-agent component. See the pxp-agent-vanagon project for more information.

Runtime Requirements

Ruby and bundler are required to build puppet-agent. The Gemfile specifies all of the necessary ruby libraries to build a puppet-agent package. Additionally, puppet-agent requires a VM to build within for each desired package.

Environment variables

VANAGON_LOCATION

The location of Vanagon in the Gemfile can be overridden with the environment variable VANAGON_LOCATION. Can be set prior to bundle install or updated with bundle update.

  • 0.3.14 - Specific tag from the Vanagon git repo
  • https://github.com/puppetlabs/vanagon#version - Remote git location and version (can be a ref, branch or tag)
  • file:///workspace/vanagon - Absolute file path
  • file://../vanagon - File path relative to the project directory

DEV_BUILD

By default, headers and other files that aren't needed in the final puppet-agent package will be removed as part of the cleanup component. If you'd like to keep these files in the finished package, set the DEV_BUILD environment variable to some non-empty value. Note that this will increase the size of the package considerably.

Building puppet-agent or the facter gem

If you wish to build puppet-agent yourself:

  1. First, build the puppet-runtime for your target platform and agent version.
  2. Run bundle install to install required ruby dependencies.
  3. Update the location and version in the puppet-runtime component json file as follows:
    • location should be a file URL to your local puppet-runtime output directory, for example: file:///home/you/puppet-runtime/output
    • version should be the version of puppet-runtime that you built; You can find this value at the top level of the json metadata file produced by the build in your puppet-runtime output directory.
  4. You can disable the packaging of pxp-agent by setting NO_PXP_AGENT ENV variable. If you want to build an agent package that also contains pxp-agent you need to update the location and version in the pxp-agent component json file as follows:
  • location should be a file URL to your local pxp-agent- output directory, for example: file:///home/you/pxp-agent-vanagon/output
  • version should be the version of pxp-agent that you built; You can find this value at the top level of the json metadata file produced by the build in your pxp-agent output directory.
  • You also may need to change the source URIs for some other components. We recognize this is less than ideal at this point, but we wanted to err on the side of getting this work out in public rather than having everything perfect. If you have your own mirror of the components of puppet-agent, you can also use a rewrite rule. See the Vanagon README for an example.
  1. Now use vanagon to build the puppet-agent. Run the following:

    bundle exec build <project-name> <platform> <vm-hostname>

    Where:

    • project name is a project from configs/projects (this can be puppet-agent),
    • platform is a platform supported by vanagon and defined in the configs/platforms directory (for example, el-7-x86_64), and
    • the vm hostname is the hostname of a vm matching the desired platform. The current user must be able to ssh into that vm as root (vanagon has facilities to provide an ssh key beyond what is listed in .ssh/config).

Branches in puppet-agent

Tracking branch (main + stable):

  • some components may reference tags if they’re slow moving (ruby, openssl)
  • some components reference SHAs promoted by a CI pipeline (generally puppet-agent#main pipelines track components' main branches, and likewise for stable)

Guidelines on Merging Between Branches

  • stable should be merged to main regularly (e.g. per commit), as is done for component repos; no PR needed
  • main should be merged to stable as-needed; typically this is done when a component merges its main to stable, and there are matching changes needed in puppet-agent

Generally, no PR is needed for routine merges from stable to main, but a PR is advised for other merges. Use your judgment of course, and put up a PR if you want review.

Note that for all merges from main or stable, the merge should pick up:

  • changes outside of config/components
  • changes that bumped to a tag inside config/components

But never:

  • changes that bumped to a SHA inside config/components

Here's a sample snippet used for a stable -> main merge:

git merge --no-commit --no-ff stable
for i in {hiera,facter,puppet,pxp-agent,cpp-pcp-client}; do git checkout main -- configs/components/$i.json;done
git commit -m "(maint) Restore promoted components refs after merge from stable"

Issues

File issues in the Puppet Agent (PA) project on the Puppet Labs Jira site. Issues with individual components should be filed in their respective projects.

License

Puppet agent is licensed under the Apache-2.0 license.

Code Owners

See CODEOWNERS

Running Tests

See Acceptance README

puppet-agent's People

Contributors

ariaxli avatar bobosilavictor avatar branan avatar caseywilliams avatar ciprianbadescu avatar cthorn42 avatar ekinanp avatar ferventcoder avatar gabrielnagy avatar geoffnichols avatar gimmyxd avatar haus avatar iristyle avatar joshcooper avatar luchihoratiu avatar magisus avatar mcdonaldseanp avatar mckern avatar melissa avatar mhashizume avatar mihaibuzgau avatar mikaelsmith avatar mwaggett avatar puppetlabs-jenkins avatar scottgarman avatar sebastian-miclea avatar shermdog avatar speedofdark avatar stahnma avatar tvpartytonight avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppet-agent's Issues

macOS Full Disk Access no longer works as described in installation documentation

Describe the Bug

For reference: https://www.youtube.com/watch?v=DcrfCGqqjkA

Video link is how Munki solved this issue.

Full Disk Access (FDA) changed somewhere in macOS12 and up so that the FDA is not inherited as in previous versions. So, if we use the current Puppet agent instructions for macOS, we would assign FDA to the Puppet wrapper script. However, that script calls /bin/sh immediately which does not inherit FDA so Puppet in unable to update files. We don't want to grant /bin/sh FDA as that would open a huge security hole.

This also affects the LaunchDaemon which starts up the same wrapper script so it is also unable to perform updates.

As described in the video, a signed binary executable can be used to fix this - the video explains why much better than I can here.

See also https://www.qt.io/blog/the-curious-case-of-the-responsible-process

Asking for Puppet to produce a similar signed binary for the macOS installer.

Expected Behavior

Expected FDA granted to Puppet would allow it to manage the system, however this is no longer the case in macOS12 and newer.

Steps to Reproduce

# ps -ef |grep puppet |grep -v grep             
    0   143     1   0 Mon12PM ??         0:06.12 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent --verbose --no-daemonize --logdest console
# sudo launchctl procinfo 143 | grep responsible
responsible pid = 143
responsible unique pid = 143
responsible path = /bin/sh  <<< That doesn't have FDA

Environment

macOS12 or newer
Puppet 7.24

puppet-agent fails on Solaris 11 with the SRU update #57

A new software repository update, SRU, for Solaris 11 has broken the puppet-agent.
We've gotten a report confirming this for the 7.x branch, waiting for confirmation for 8.x/main branch.
The error is around libffi which comes packaged with Solaris, we currently rely on the 7.x stream of puppet-agent the Solaris system library of libffi.
On Solaris 11 with a SRU 57 or newer the failure will look something like this:

puppet agent --version
Error: Could not initialize global default settings: ld.so.1: ruby: libffi.so.5: open failed: No such file or directory - /opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/ffi-1.15.5/lib/ffi_c.so

We need to fix this in the puppet-runtime, this will require a new puppet-agent release in order to fix.

Puppet-Agent running in docker can't manage host systemd services

Describe the Bug

The puppet-agent ( https://hub.docker.com/r/puppet/puppet-agent ), running in a docker-container, can't manage the hosts systemd services like a native puppet-agent can.

I'm running the puppet-agent like the Overview Page on Docker Hub
docker run --rm --privileged --hostname agent -v /tmp:/tmp -v /etc:/etc -v /var:/var -v /usr:/usr -v /lib64:/lib64 puppet/puppet-agent-ubuntu

Hope there is a way to get this working.

Expected Behavior

Puppet-Agent running in container behaves like the native running binary.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Start the puppet-agent with the command from the hub.docker.com page
    docker run --rm --privileged --hostname agent -v /tmp:/tmp -v /etc:/etc -v /var:/var -v /usr:/usr -v /lib64:/lib64 puppet/puppet-agent-ubuntu
  2. Jump imto the container
    docker exec -ti <name_of_container> bash
  3. Create a small test .pp ( vi )
    service { 'chrony': ensure => 'stopped', enable => true, provider => 'systemd', }
  4. Apply the .pp file
    puppet apply ./init.pp ... Notice: Compiled catalog for agent in environment production in 0.01 seconds Error: /Stage[main]/Main/Service[chrony]: Provider systemd is not functional on this host

Environment

Additional Context

  • Docker Container systemctl output:
    System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down
  • Puppet-Agent Error Output:
    Error: /Stage[main]/Main/Service[chrony]: Provider systemd is not functional on this host

Is there an alternative to building from github source to create a puppet-agent rpm?

Use Case

Are there source rpms available on https://yum.puppetlabs.com/ for the puppet-agent package from which the .spec file could be generated and combined with a puppet-agent tarball from https://downloads.puppetabs.com/ to more easily generate an rpm?

Describe the Solution You Would Like

I would like to produce/re-package my own puppet-agent rpm.
With much older (pre all in one package) builds I could use the method described above to generate my own rpm and avoid having to build the rpm from the source on github.

Describe Alternatives You've Considered

Trying to build the puppet-agent from source as described on github.
Currently i'm unable to build the puppet-runtime project from source, which is a pre-requisite to building the puppet-agent.
Ticket raised on that: puppetlabs/puppet-runtime#784

Rake not in Gemfile.

~/code/puppetlabs/puppet-agent [master %] 11:17:01 $ bundle exec rake -T
/opt/boxen/rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/bundler-1.6.3/lib/bundler/rubygems_integration.rb:252:in block in replace_gem': rake is not part of the bundle. Add it to Gemfile. (Gem::LoadError) from /opt/boxen/rbenv/versions/2.0.0/bin/rake:22:in

'

puppet agent works interactively but refuses to start service

After upgrading, either by running an 8.x install either by first uninstalling a 7.x install or by running it over the top, puppet installs successfully on windows 2019/2022 however I receive errors on the service. Seems to happen moreso on core, but it occurs on both desktop experience and regular. Sometimes after 5-10 starts it will actually start, but usually if stopped, it refuses to start again.

Failed to transition the puppet service to the SERVICE_RUNNING state. Detail: Failed to start the service: The service did not respond to the start or control request in a timely fashion.

Running puppet agent interactively works perfectly fine. rolling back to 7.x (even 7.27.0) works, as does starting the 7.x service.
The issue is exclusively on the service starting cleanly after installation. A reboot does not fix it. Nor does removing C:\programdata\puppetlabs, or refreshing to certs.

Any other additional information needed, let me know.

Puppet agent on Debian 12 don't find installed modules

The default install of puppet-agent v7 on Debian 12 lack builtin support for stuff we're used to, like augeas, sshkeys, etc.

And when you manually install these core packets (apt install puppet-module-puppetlabs-augeas-core; apt install puppet-module-puppetlabs-sshkeys-core, etc) they do list as installed with puppet module list, but they are not found.

Workaround: For some reason, adding a symlink to the module(s) in /usr/share/puppet/vendor_modules/ make puppet-agent find them.


When running puppet agent -t on a Debian 12 machine, I get errors like this:

root@baz:~# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Failed to apply catalog: Resource type 'Augeas' was not found

or

Error: Failed to apply catalog: Resource type 'Ssh_authorized_key' was not found

Steps to Reproduce

Installed a brand new Debian 12 (bookworm) machine
Since the puppet-agent .deb package (v7) now seem to be located in the official Debian APT repo, I have not added any custom APT source.
(comparison: on my Debian 11 machines I have /etc/apt/sources.list.d/puppet7.list : deb http://apt.puppetlabs.com bullseye puppet7)

echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01_no_recommends
apt update && apt -y full-upgrade
apt -y install puppet-agent

mkdir /var/lib/puppet
echo -e 'vardir = /var/lib/puppet\nssldir = /var/lib/puppet/ssl\nserver = puppet-master2.foo.bar' > /etc/puppet/puppet.conf
puppet ssl bootstrap 
  <I now get a signed cerificate>

I declare the node baz.foo.bar in nodes.pp and assign a profile.
As long as the profile and manifest don't contain any augeas calls (or ssh-keys, etc), everything is working fine:

root@baz:~# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for baz.foo.bar
Info: Applying configuration version 'ea40e02a'
<...doing my changes...>
Notice: Applied catalog in 3.05 seconds
root@baz:~#

But if I add a augeas call - for example:

  augeas { '/etc/default/puppet: enable puppet daemon':
    changes => [
      'set /files/etc/default/puppet/START yes',
      ],
  }

...then puppet-agent immediately bail with "Error: Failed to apply catalog: Resource type 'Augeas' was not found".

Additional Context

Package: puppet-agent
Version: 7.23.0-1


The homepage for augeas_core (https://forge.puppet.com/modules/puppetlabs/augeas_core/readme) says:

Setup Requirements:
The augeas libraries and ruby bindings must be installed in order to use this module.
If you are using puppet-agent packages, then those prerequisites are already satisfied for most platforms.

So for Debian 12, that last sentence don't really apply right now.

To make puppet-agent work the way I'm used to, I currently have to run:

apt install puppet-module-puppetlabs-augeas-core
ln -s /usr/share/puppet/modules.available/puppetlabs-augeas-core /usr/share/puppet/vendor_modules/
apt install puppet-module-puppetlabs-sshkeys-core
ln -s /usr/share/puppet/modules.available/puppetlabs-sshkeys-core /usr/share/puppet/vendor_modules/
...and so on...

I guess you should do something like the following to solve this issue:

1
In the puppet-agent .deb package, perhaps you should add those core modules (like puppet-module-puppetlabs-augeas-core) as Depends.

Ok, now the module is installed, but this is not enough.
Because even though puppet module list show that it is installed...

root@baz:~# puppet module list
/usr/share/puppet/modules
└── puppetlabs-augeas_core (v1.1.2)

...apparently it is not found since we get the "'Augeas' was not found" message.

2
I assume the module "registration" is done correctly when the above package is installed (it is located here: /usr/share/puppet/modules/augeas_core -> /etc/alternatives/puppet-module-augeas-core -> /usr/share/puppet/modules.available/puppetlabs-augeas-core).

So the bug must be in 'puppet-agent'. It need to be fixed to find and use the installed module somehow.
Exactly how is up to you. Using the vendor_modules dir is just my workaround.

Unsatisfiable dependency on RedHat 9 package.

Describe the Bug

Puppet-agent package can't be installed on fully upgraded RedHat 9.3
It has requirement '/bin/mkdir', and nothing provides '/bin/mkdir' on RedHat 9.3.
Everything is moved to /usr/bin, and there is a symlink:

[root@localhost ~]# ls -altr /bin
lrwxrwxrwx. 1 root root 7 Aug  9  2021 /bin -> usr/bin

Steps to Reproduce

[root@localhost ~]# dnf install puppet-agent
Updating Subscription Management repositories.
Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)                                                                                                                                                             142 kB/s | 4.5 kB     00:00    
Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)                                                                                                                                                                136 kB/s | 4.1 kB     00:00    
Error: 
 Problem: cannot install the best candidate for the job
  - nothing provides /bin/mkdir needed by puppet-agent-7.30.0-1.el8.x86_64 from puppet7
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
[root@localhost ~]#

Environment

  • puppet-agent 7.30.0
  • Platform RedHat 9.3

Additional Context

[root@localhost ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.3 (Plow)
[root@localhost ~]# dnf upgrade
warning: Signature not supported. Hash algorithm SHA1 not available.
Updating Subscription Management repositories.
Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)                                                                                                                                                              92 kB/s | 4.5 kB     00:00    
Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)                                                                                                                                                                 92 kB/s | 4.1 kB     00:00    
Dependencies resolved.
Nothing to do.
Complete!
[root@localhost ~]# dnf info puppet-agent
warning: Signature not supported. Hash algorithm SHA1 not available.
Updating Subscription Management repositories.
Choria Orchestrator Releases                                                                                                                                                                                         1.1 kB/s | 858  B     00:00    
Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)                                                                                                                                                              89 kB/s | 4.5 kB     00:00    
Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)                                                                                                                                                                135 kB/s | 4.1 kB     00:00    
Available Packages
Name         : puppet-agent
Version      : 7.30.0
Release      : 1.el8
Architecture : x86_64
Size         : 22 M
Source       : puppet-agent-7.30.0-1.el8.src.rpm
Repository   : puppet7
Summary      : The Puppet Agent package contains all of the elements needed to run puppet, including ruby, facter, and hiera.
URL          : https://www.puppetlabs.com
License      : See components
Description  : The Puppet Agent package contains all of the elements needed to run puppet, including ruby, facter, and hiera.
             : 
             : Contains the following components:
             : cleanup
             : facter 4.7.0
             : hiera 3.12.0
             : module-puppetlabs-augeas_core v1.5.0
             : module-puppetlabs-cron_core v1.3.0
             : module-puppetlabs-host_core v1.3.0
             : module-puppetlabs-mount_core v1.3.0
             : module-puppetlabs-scheduled_task v3.2.0
             : module-puppetlabs-selinux_core v1.4.0
             : module-puppetlabs-sshkeys_core v2.5.0
             : module-puppetlabs-yumrepo_core v1.2.0
             : module-puppetlabs-zfs_core v1.5.0
             : module-puppetlabs-zone_core v1.2.0
             : puppet 7.30.0
             : puppet-resource_api 1.9.0
             : puppet-runtime 202404030
             : pxp-agent 202404041
             : shellpath 2023-02-15
             : wrapper-script

[root@localhost ~]# dnf provides /bin/mkdir
warning: Signature not supported. Hash algorithm SHA1 not available.
Updating Subscription Management repositories.
Last metadata expiration check: 0:00:15 ago on Fri 19 Apr 2024 08:50:59 AM EDT.
coreutils-8.32-31.el9.x86_64 : A set of basic GNU tools commonly used in shell scripts
Repo        : rhel-9-for-x86_64-baseos-rpms
Matched from:
Filename    : /usr/bin/mkdir

coreutils-8.32-32.el9.x86_64 : A set of basic GNU tools commonly used in shell scripts
Repo        : rhel-9-for-x86_64-baseos-rpms
Matched from:
Filename    : /usr/bin/mkdir

coreutils-8.32-34.el9.x86_64 : A set of basic GNU tools commonly used in shell scripts
Repo        : @System
Matched from:
Filename    : /usr/bin/mkdir

coreutils-8.32-34.el9.x86_64 : A set of basic GNU tools commonly used in shell scripts
Repo        : rhel-9-for-x86_64-baseos-rpms
Matched from:
Filename    : /usr/bin/mkdir

coreutils-single-8.32-31.el9.x86_64 : coreutils multicall binary
Repo        : rhel-9-for-x86_64-baseos-rpms
Matched from:
Filename    : /usr/bin/mkdir

coreutils-single-8.32-32.el9.x86_64 : coreutils multicall binary
Repo        : rhel-9-for-x86_64-baseos-rpms
Matched from:
Filename    : /usr/bin/mkdir

coreutils-single-8.32-34.el9.x86_64 : coreutils multicall binary
Repo        : rhel-9-for-x86_64-baseos-rpms
Matched from:
Filename    : /usr/bin/mkdir

[root@localhost ~]#

Invalid entropy given for user XXXXXX on macOS Sonoma and M2/M3 systems

Describe the Bug

We have been transitioning from x86 Mac systems to the newer Apple M2/M3 systems and have started getting this error during Puppet runs:

Error: Could not prefetch user provider 'directoryservice': Invalid entropy given for user scanuser

"scanuser" is an account we create via Puppet - it is created normally on x86 systems, this error is new for the M2/M3 builds on Sonoma.

The same Puppet 'user' call is used for all macOS systems so it is confusing as to why these new systems are throwing this error.

I've got a debug run from one of the systems that I'm happy to share direct with someone from Puppet, I don't want to post it publicly here.

Expected Behavior

User gets created as normal

Steps to Reproduce

Run puppet agent

Environment

  • Puppet agent 7.29.1
  • macOS Sonoma 14.4.1 on Apple M2 Pro

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.