saltstack-formulas / apt-formula Goto Github PK

Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html

License: Other

SaltStack 23.67% Ruby 30.16% JavaScript 13.53% Shell 4.10% Jinja 28.55%

apt-formula's Issues

Handle apt.conf.d configuration with pillars like preferences.d

the preferences.d files shipped via the pillars are quite nice, it would be nice to be able to do the same with apt.conf.d

Less an issue -- more a question

Why have update and upgrade only run when sources.list or sources.list.d are changed?

This means it can't be run as a maintenance job. I'm new to Saltstack formulas, so am I missing something obvious?

I modified and tested the formula to use additional pillar data, "always_update" and "always_upgrade" to enable the maintenance behavior, e.g.

{% from "apt/map.jinja" import apt as apt_map with context %}
{% set apt = pillar.get('apt', {}) %}
{% set always_update = apt.get('always_update', apt_map.always_update) %}

apt-get -y update:
  {% if always_update %}
  cmd.run
  {% else %}
  cmd.wait:
    - watch:
      - file: /etc/apt/sources.list
      - file: /etc/apt/sources.list.d
  {% endif %}

Any interesting in a pull request?

[WARNING ] could not open file '/etc/apt/sources.list'

Hello,

I'm using this formula to manage my apt repositories, I only include apt.repositories in my states, and my pillar looks like

{% set oscodename = grains['oscodename'] %}

apt:
  remove_sources_list: true
  clean_sources_list_d: true

  remove_preferences: true
  clean_preferences_d: true

  repositories:
    {{ oscodename }}:
      distro: {{ oscodename }}
      url: http://ftp.debian.org/debian
      comps: [main, contrib, non-free]
      type: [binary]
[...]

When I run a highstate or only the apt.repositories state, I got this error

[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://apt/repositories.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://apt/map.jinja'
[INFO    ] Running state [debian-archive-keyring] at time 15:26:43.653378
[INFO    ] Executing state pkg.installed for debian-archive-keyring
[INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}\n', '-W'] in directory '/root'
[INFO    ] Package debian-archive-keyring is already installed.
[INFO    ] Completed state [debian-archive-keyring] at time 15:26:43.866063
[INFO    ] Running state [/etc/apt/sources.list] at time 15:26:43.868531
[INFO    ] Executing state file.absent for /etc/apt/sources.list
[INFO    ] File /etc/apt/sources.list is not present
[INFO    ] Completed state [/etc/apt/sources.list] at time 15:26:43.870025
[INFO    ] Running state [/etc/apt/sources.list.d] at time 15:26:43.870463
[INFO    ] Executing state file.directory for /etc/apt/sources.list.d
[INFO    ] {'removed': ['/etc/apt/sources.list.d/wheezy-binary.list']}
[INFO    ] Completed state [/etc/apt/sources.list.d] at time 15:26:43.872370
[INFO    ] Running state [deb  http://ftp.debian.org/debian wheezy main contrib non-free] at time 15:26:43.873656
[INFO    ] Executing state pkgrepo.managed for deb  http://ftp.fr.debian.org/debian wheezy main contrib non-free
[WARNING ] could not open file '/etc/apt/sources.list'

[WARNING ] could not open file '/etc/apt/sources.list'

[INFO    ] Executing command 'apt-get -q update' in directory '/root'
[WARNING ] could not open file '/etc/apt/sources.list'
[...]

The error/warning (could not open file '/etc/apt/sources.list') appears several times, at least twice for each apt repository.

Do you have any idea of what can be the reason for this warning?

[FEATURE] apt-daily-upgrade.timer configuration for apt.unattended

Is your feature request related to a problem?

On debian (at least buster), the systemd service "apt-daily-upgrade.service", which install package upgrades is controlled by apt-daily-upgrade.timer.

Currently the service is triggered at 6:00 plus a random delay of 0-60 minutes :

# cat /lib/systemd/system/apt-daily-upgrade.timer

[Unit]
Description=Daily apt upgrade and clean activities
After=apt-daily.timer

[Timer]
OnCalendar=*-*-* 6:00
RandomizedDelaySec=60m
Persistent=true

[Install]
WantedBy=timers.target

Would it be possible to have a way of controlling the OnCalendar and RandomizedDelaySec parameters ?

Thanks

Describe the solution you'd like

an example configuration would be :

apt:
  unattended:
    service:
      upgrade:
        calendar: "*-*-* 6:00"
        delay: "60m"

Describe alternatives you've considered

Additional context

Not all Apt::Periodic options are supported

The file /etc/apt/apt.conf.d/10periodic is managed via the apt:unattended pillar, but not all APT::Periodic options are supported.

Those missing include, but are not limited to, MaxAge, MaxSize, and CleanInterval.

[BUG] `refresh_db` is deprecated for `pkgrepo.managed`

Your setup

Formula commit hash / release tag

Versions reports (master & minion)

Pillar / config used

Bug details

Describe the bug

As reported in the Slack #testing channel (which will disappear after a while).

https://saltstackcommunity.slack.com/archives/C7KKN9P6W/p1588231385039600

Rebuilt our pre-salted images at https://github.com/netmanagers/salt-image-builder after the CVE release and now beginning to run the tests of the master branch across the SaltStack Formulas. Got our first failures in the apt-formula, with a neon warning that's been reintroduced by a develop => master port 8 days ago (https://github.com/saltstack/salt/blame/master/salt/states/pkgrepo.py#L313-L318).

https://travis-ci.org/github/myii/apt-formula/jobs/681357767#L1517-L1530
RuntimeError: The warning triggered on filename '/usr/local/lib/python3.7/dist-packages/salt/states/pkgrepo.py', line number 315, is supposed to be shown until version 3000 is released. Current version is now 3000. Please remove the warning.
Out of interest, why is a salt.utils.versions.warn_until() significant enough to trigger a RuntimeError?

Steps to reproduce the bug

Expected behaviour

Attempts to fix the bug

This line will need to be removed/adjusted:

apt-formula/apt/repositories.sls

Line 81 in baa83f2

- refresh_db: False

Additional context

state `apt.update` can't run standalone

I thought I'd be able to use this apt.update state to update the apt db on servers, so I tried

salt 'minion' state.sls apt.update

It fails with

          ID: apt-get update
    Function: cmd.wait
      Result: False
     Comment: The following requisites were not found:
                                 watch:
                                     file: /etc/apt/sources.list
                                     file: /etc/apt/sources.list.d
     Started: 13:42:21.398096
    Duration: 0.061 ms
     Changes:

README lists this state so it looks like it's callable standalone (not just for internal use) but it seems not.

update formula to be compliant with ```fluorine``` release

Warnings:
The following package(s) are "virtual package" names: apt-transport-https. These will no longer be supported as of the Fluorine release. Please update your SLS file(s) to use the actual package name.

apt-transport-https:
    pkg.installed

should be:

apt-transport-https:
    pkg.installed:
      - name: apt-transport-https

[BUG] Helm Repo cannot use GPG key .asc

Your setup

Formula commit hash / release tag

v0.10.3

Versions reports (master & minion)

Saltmaster:

Salt Version:
          Salt: 3002.6
Dependency Versions:
          cffi: 1.14.5
      cherrypy: Not Installed
      dateutil: 2.7.3
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10
       libgit2: 0.27.7
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.5.6
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.19
      pycrypto: Not Installed
  pycryptodome: 3.6.1
        pygit2: 0.27.4
        Python: 3.7.3 (default, Jul 25 2020, 13:03:44)
  python-gnupg: Not Installed
        PyYAML: 3.13
         PyZMQ: 17.1.2
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.1

System Versions:
          dist: debian 10 buster
        locale: utf-8
       machine: x86_64
       release: 4.19.0-14-amd64
        system: Linux
       version: Debian GNU/Linux 10 buster

Saltminion:

Salt Version:
          Salt: 3003
Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.7.3
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.5.6
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.6.1
        pygit2: Not Installed
        Python: 3.7.3 (default, Jan 22 2021, 20:04:44)
  python-gnupg: Not Installed
        PyYAML: 3.13
         PyZMQ: 17.1.2
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.1
 
System Versions:
          dist: debian 10 buster
        locale: utf-8
       machine: x86_64
       release: 4.19.0-16-amd64
        system: Linux
       version: Debian GNU/Linux 10 buster

Pillar / config used

{#- URLs fuer Repositories und GnuPG-Keys #}
{%- set repo_url = 'https://baltocdn.com/helm/stable/debian/' %}
{%- set key_url = 'https://baltocdn.com/helm/signing.asc' %}

# Name der Debian-Version ermitteln
{%- if salt['grains.get']('osmajorrelease') == 10 %}
{%-  set distro = 'buster' %}
{%- elif salt['grains.get']('osmajorrelease') == 11 %}
{%-  set distro = 'bullseye' %}
{%- else %}
{%-  set distro = 'buster' %}
{%- endif %}

# Architektur des Systems erkennen und in Variable festhalten
{%- set osarch = salt['grains.get']('osarch') %}
### Major-Release-Nummer in Variable speichern
{%- set release = salt['grains.get']('osmajorrelease') %}

apt:
  repositories:
# Repo von Saltstack fuer stretch und buster Hosts
    helm:
      distro: "all main"
      #comps: [all, main]
      url: {{ repo_url }}
      arch: [amd64]
      type: [binary]
      keyurl: {{ key_url }}

Bug details

Describe the bug

The signing key from the helm docs isn't working for trusting the repo.
I also tried to forced to trust it with the option "trusted: yes" but this does not do the trick.

Err:5 https://baltocdn.com/helm/stable/debian all InRelease                
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 294AC4827C1A168A
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://baltocdn.com/helm/stable/debian all InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 294AC4827C1A168A
W: Failed to fetch https://baltocdn.com/helm/stable/debian/dists/all/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 294AC4827C1A168A
W: Some index files failed to download. They have been ignored, or old ones used instead.

Steps to reproduce the bug

Minion debug log:
 1 local:                                                                                                                                                    
  2 ----------
  3           ID: debian-archive-keyring
  4     Function: pkg.installed
  5       Result: True
  6      Comment: All specified packages are already installed
  7      Started: 10:17:23.416126
  8     Duration: 34.819 ms
  9      Changes:   
 10 ----------
 11           ID: /etc/apt/sources.list
 12     Function: file.managed
 13       Result: True
 14      Comment: File /etc/apt/sources.list exists with proper permissions. No changes made.
 15      Started: 10:17:23.455036
 16     Duration: 2.422 ms
 17      Changes:   
 18 ----------
 19           ID: /etc/apt/sources.list.d
 20     Function: file.directory
 21       Result: True
 22      Comment: The directory /etc/apt/sources.list.d is in the correct state
 23      Started: 10:17:23.457792
 24     Duration: 1.846 ms
 25      Changes:   
 26 ----------
 27           ID: deb debian10-saltstack
 28     Function: pkgrepo.managed
 29         Name: deb [arch=amd64  ] https://repo.saltproject.io/py3/debian/10/amd64/3003/ buster main
 30       Result: True
 31      Comment: Configured package repo 'deb [arch=amd64  ] https://repo.saltproject.io/py3/debian/10/amd64/3003/ buster main'
 32      Started: 10:17:23.460649
 33     Duration: 783.444 ms
 34      Changes:   
 35               ----------
 36               repo:
 37                   deb [arch=amd64 ] https://repo.saltproject.io/py3/debian/10/amd64/3003 buster main
 38 ----------
 39           ID: deb helm
 40     Function: pkgrepo.managed
 41         Name: deb [arch=amd64  ] https://baltocdn.com/helm/stable/debian/ all main main
 42       Result: True
 43      Comment: Configured package repo 'deb [arch=amd64  ] https://baltocdn.com/helm/stable/debian/ all main main'
 44      Started: 10:17:24.244553
 45     Duration: 211.369 ms
 46      Changes:   
 47               ----------
 48               repo:
 49                   deb [arch=amd64 ] https://baltocdn.com/helm/stable/debian all main main
 50 ----------
 51           ID: apt.refresh_db
 52     Function: module.run
 53         Name: pkg.refresh_db
 54       Result: True
 55      Comment: Module function pkg.refresh_db executed
 56      Started: 10:17:24.457909
 57     Duration: 2357.477 ms
 58      Changes:   
 59               ----------
 60               ret:
 61                   ----------
 62                   http://deb.debian.org/debian buster InRelease:
 63                       None
 64                   http://deb.debian.org/debian buster-updates InRelease:
 65                       None
 66                   http://security.debian.org/debian-security buster/updates InRelease:
 67                       None
 68                   https://baltocdn.com/helm/stable/debian all InRelease:
 69                       False
 70                   https://download.docker.com/linux/debian buster InRelease:
 71                       None
 72                   https://packages.gitlab.com/runner/gitlab-runner/debian buster InRelease:
 73                       None
 74                   https://repo.saltproject.io/py3/debian/10/amd64/3003 buster InRelease:
 75                       None
 76 
 77 Summary for local
 78 ------------
 79 Succeeded: 6 (changed=3)
 80 Failed:    0
 81 ------------
 82 Total states run:     6
 83 Total run time:   3.391 s

Expected behaviour

The Helm repo should be trusted.

Attempts to fix the bug

don't know

Why force installation of debian-archive-keyring regardless of OS ?

Ubuntu has ubuntu-keyring, Mint has linuxmint-keyring. Forcing debian-archive-keyring just adds a useless package on those systems.

Upgrade.sls aborts with error

When I run update.sls and upgrade.sls through highstate run.
`----------
ID: apt-get update
Function: cmd.run
Result: True
Comment: Command "apt-get update" run
Started: 14:55:03.296087
Duration: 7082.109 ms
Changes:
----------
pid:
21886
retcode:
0
stderr:
stdout:
Hit http://security.debian.org jessie/updates InRelease
Hit http://security.debian.org jessie/updates/main amd64 Packages
Hit http://security.debian.org jessie/updates/main Translation-en
Ign http://http.debian.net jessie InRelease
Hit http://http.debian.net jessie-backports InRelease
Hit http://repo.saltstack.com jessie InRelease
Ign http://httpredir.debian.org jessie InRelease
Hit http://http.debian.net jessie Release.gpg
Hit http://repo.saltstack.com jessie/main amd64 Packages
Get:1 http://http.debian.net jessie-backports/main Sources/DiffIndex [27.8 kB]
Hit http://httpredir.debian.org jessie-updates InRelease
Get:2 http://http.debian.net jessie-backports/main amd64 Packages/DiffIndex [27.8 kB]
Hit http://httpredir.debian.org jessie-backports InRelease
Get:3 http://http.debian.net jessie-backports/main Translation-en/DiffIndex [27.4 kB]
Hit http://httpredir.debian.org jessie Release.gpg
Ign http://repo.saltstack.com jessie/main Translation-en
Hit http://http.debian.net jessie Release
Get:4 http://httpredir.debian.org jessie-updates/main amd64 Packages/DiffIndex [1012 B]
Hit http://http.debian.net jessie/main Sources
Get:5 http://httpredir.debian.org jessie-updates/main Translation-en/DiffIndex [736 B]
Hit http://http.debian.net jessie/main amd64 Packages
Hit http://http.debian.net jessie/main Translation-en
Get:6 http://httpredir.debian.org jessie-backports/main Sources/DiffIndex [27.8 kB]
Get:7 http://httpredir.debian.org jessie-backports/main amd64 Packages/DiffIndex [27.8 kB]
Get:8 http://httpredir.debian.org jessie-backports/main Translation-en/DiffIndex [27.4 kB]
Hit http://httpredir.debian.org jessie Release
Hit http://httpredir.debian.org jessie/main amd64 Packages
Hit http://httpredir.debian.org jessie/main Translation-en
Fetched 168 kB in 3s (45.7 kB/s)

Reading package lists...

      ID: apt-get upgrade
Function: cmd.run
  Result: False
 Comment: Command "apt-get upgrade" run
 Started: 14:55:10.378360
Duration: 268.491 ms
 Changes:   
          ----------
          pid:
              21941
          retcode:
              1
          stderr:
          stdout:
              Reading package lists...
              Building dependency tree...
              Reading state information...
              The following packages will be upgraded:
                libdns-export100 libirs-export91 libisc-export95 libisccfg-export90
                linux-image-3.16.0-4-amd64 python-requests python-urllib3
              7 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
              Need to get 34.6 MB of archives.
              After this operation, 204 kB disk space will be freed.
              Do you want to continue? [Y/n] Abort.

Summary

Succeeded: 1 (changed=2)

Failed: 1

Total states run: 2
ERROR: Minions returned with non-zero exit code
`

Use deb.debian.org instead of httpredir.debian.org

httpredir is now deprecated (eventough it works by redirecting to deb.debian.org) see this message on the mailing list for more info https://lists.debian.org/debian-mirrors/2017/02/msg00000.html

Handle prefix options such as [trusted=yes]

The formula handles the following syntax

    - name: {{ r_type }} {{ r_arch }} {{ r_url }} {{ r_distro }} {{ r_comps }}

It would useful to have another freestyle option for things such as [trusted=yes]

[BUG] cannot find a way to set pkgrepo.managed aptkey to False

Your setup

Formula commit hash / release tag

I'm using the most recent master bea590b

Versions reports (master & minion)

Salt Version:
          Salt: 3002.6
 
Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: 1.1.0
      M2Crypto: 0.37.1
          Mako: Not Installed
       msgpack: 1.0.0
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.9.7
        pygit2: 1.4.0
        Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
  python-gnupg: 0.4.6
        PyYAML: 5.3.1
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-22-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye

master and minion is the same machine.

Pillar / config used

apt:
  remove_sources_list: true
  repositories:
    salt:
      distro: bullseye
      url: https://repo.saltproject.io/salt/py3/debian/11/amd64/3007
      key_url: https://repo.saltproject.io/salt/py3/debian/11/amd64/SALT-PROJECT-GPG-PUBKEY-2023.gpg
      opts:
        signed-by: /etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg

Bug details

Describe the bug

This role does not seem to know the aptkey false from pkgrepo.managed and does not download the key into /etc/apt/keyrings. apt-key itself is deprecated. Looking through the source cannot seem to find the option to set - aptkey: False in this role.

Steps to reproduce the bug

Use above pillar. See that it errors on the salt repo:

local:
----------
          ID: deb salt
    Function: pkgrepo.managed
        Name: deb [ signed-by=/etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg ] https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye main
      Result: True
     Comment: Configured package repo 'deb [ signed-by=/etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg ] https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye main'
     Started: 19:23:01.724969
    Duration: 813.624 ms
     Changes:   
              ----------
              repo:
                  deb [ signed-by=/etc/apt/keyrings/SALT-PROJECT-GPG-PUBKEY-2023.gpg ] https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye main
----------
          ID: apt.refresh_db
    Function: module.run
        Name: pkg.refresh_db
      Result: False
     Comment: Module function pkg.refresh_db threw an exception. Exception: W: GPG error: https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 64CBBC8173D76B3F
              E: The repository 'https://repo.saltproject.io/salt/py3/debian/11/amd64/3007 bullseye InRelease' is not signed.
     Started: 19:23:02.541283
    Duration: 4257.558 ms
     Changes:   

Summary for local
--------------
Succeeded: 130 (changed=1)
Failed:      1
--------------
Total states run:     131
Total run time:     6.392 s

Expected behaviour

I'd like to use aptkey: False.

Attempts to fix the bug

Look through the documentation (pillar) and source code.

saltstack-formulas / apt-formula Goto Github PK

apt-formula's Issues

Is your feature request related to a problem?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Your setup

Formula commit hash / release tag

Versions reports (master & minion)

Pillar / config used

Bug details

Describe the bug

Steps to reproduce the bug

Expected behaviour

Attempts to fix the bug

Additional context

Your setup

Formula commit hash / release tag

Versions reports (master & minion)

Pillar / config used

Bug details

Describe the bug

Steps to reproduce the bug

Expected behaviour

Attempts to fix the bug

Reading package lists...

Summary

Failed: 1

Your setup

Formula commit hash / release tag

Versions reports (master & minion)

Pillar / config used

Bug details

Describe the bug

Steps to reproduce the bug

Expected behaviour

Attempts to fix the bug

Additional context

Recommend Projects

Recommend Topics

Recommend Org