Investigate issue with package installation about network HOT 15 CLOSED

I think the whole when: condition should be omitted. The package module will take care of not installing packages that are already installed.

from network.

The when: condition is there to speed up the role on RHEL7 because subscription-manager is slowing yum down a lot even if nothing needs to be installed.

from network.

This must be a common issue to all the package: invocations though and other roles do not do that. Don't such optimizations belong to the yum or package modules?

from network.

It would be great if the modules would optimize this. However they have the problem that they do not only support installing package by name but also by provides AFAIK. Also without subscription manager yum/dnf is not so slow if nothing needs to be installed, so it might not be worth it for the general case.

from network.

Could the role(s) provide a wrapper action plugin which would handle this optimization? It does not make sense to clutter all the package installation tasks with such conditions.

from network.

Hello. I think I just hit this:

TASK [linux-system-roles.network : Install packages] *****************************************************************************************************************************************************************************************
Wednesday 02 March 2022  11:12:06 +0100 (0:00:00.064)       0:00:12.621 ******* 
fatal: [f12-h32-b04-5039ms.rdu2.scalelab.redhat.com]: FAILED! => {"msg": "The conditional check 'not network_packages is subset(ansible_facts.packages.keys())' failed. The error was: error while evaluating conditional (not network_packages is subset(ansible_facts.packages.keys())): {{ __network_provider_setup[network_provider]['packages'] }}: {'nm': {'service_name': '{{ __network_service_name_default_nm }}', 'packages': '{{ __network_packages_default_nm }}'}, 'initscripts': {'service_name': '{{ __network_service_name_default_initscripts }}', 'packages': '{{ __network_packages_default_initscripts }}'}}: {{['NetworkManager'] + __network_packages_default_gobject_packages|select()|list() + __network_packages_default_wpa_supplicant|select()|list() + __network_packages_default_wireless|select()|list() + __network_packages_default_team|select()|list()}}: [\"python{{ ansible_python['version']['major'] | replace('2', '')}}-gobject-base\"]: 'ansible_python' is undefined\n\nThe error appears to be in '/home/jhutar/.ansible/roles/linux-system-roles.network/tasks/main.yml': line 22, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n    var: \"ansible_facts.packages.keys()\"\n- name: Install packages\n  ^ here\n"}

I have tracked it down to skipping setup at the header of my playbook with gather_facts: no that makes ansible_python variable undefined. That variable is used when rendering __network_packages_default_gobject_packages and that is used when rendering __network_packages_default_nm. This chain causes above issue.

Not sure if running setup: if the variable is not defined would help, but maybe just asserting for all required variables at the beginning and failing with helpful message (and documenting need for setup:) could be a solution?

If I'm commenting in incorrect issue, please let me know and I'll create a new one.

from network.

Hello. I think I just hit this:

TASK [linux-system-roles.network : Install packages] *****************************************************************************************************************************************************************************************
Wednesday 02 March 2022  11:12:06 +0100 (0:00:00.064)       0:00:12.621 ******* 
fatal: [f12-h32-b04-5039ms.rdu2.scalelab.redhat.com]: FAILED! => {"msg": "The conditional check 'not network_packages is subset(ansible_facts.packages.keys())' failed. The error was: error while evaluating conditional (not network_packages is subset(ansible_facts.packages.keys())): {{ __network_provider_setup[network_provider]['packages'] }}: {'nm': {'service_name': '{{ __network_service_name_default_nm }}', 'packages': '{{ __network_packages_default_nm }}'}, 'initscripts': {'service_name': '{{ __network_service_name_default_initscripts }}', 'packages': '{{ __network_packages_default_initscripts }}'}}: {{['NetworkManager'] + __network_packages_default_gobject_packages|select()|list() + __network_packages_default_wpa_supplicant|select()|list() + __network_packages_default_wireless|select()|list() + __network_packages_default_team|select()|list()}}: [\"python{{ ansible_python['version']['major'] | replace('2', '')}}-gobject-base\"]: 'ansible_python' is undefined\n\nThe error appears to be in '/home/jhutar/.ansible/roles/linux-system-roles.network/tasks/main.yml': line 22, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n    var: \"ansible_facts.packages.keys()\"\n- name: Install packages\n  ^ here\n"}

I have tracked it down to skipping setup at the header of my playbook with gather_facts: no that makes ansible_python variable undefined. That variable is used when rendering __network_packages_default_gobject_packages and that is used when rendering __network_packages_default_nm. This chain causes above issue.

Not sure if running setup: if the variable is not defined would help, but maybe just asserting for all required variables at the beginning and failing with helpful message

That might be difficult, and probably unhelpful, if there are a couple of dozen variables required that are usually provided by setup or gather_facts: true.

(and documenting need for setup:) could be a solution?

Is it common to use roles and gather_facts: false? If so, can you point me at such roles that work under those conditions?
What is the use case for using gather_facts: false?

If I'm commenting in incorrect issue, please let me know and I'll create a new one.

from network.

That might be difficult, and probably unhelpful, if there are a couple of dozen variables required that are usually provided by setup or gather_facts: true.

I guess we can do something like the following to ensure that facts are always gathered:

- name: Gather facts
  setup:
  when:
    - ansible_python is not defined
  no_log: true

from network.

That might be difficult, and probably unhelpful, if there are a couple of dozen variables required that are usually provided by setup or gather_facts: true.

I guess we can do something like the following to ensure that facts are always gathered:

- name: Gather facts
  setup:
  when:
    - ansible_python is not defined
  no_log: true

yes - but I'm really interested in the use case for not using gather_facts - I'm also concerned that several other system roles need to do something similar . . .

from network.

Hello @richm !

That might be difficult, and probably unhelpful, if there are a couple of dozen variables required that are usually provided by setup or gather_facts: true.

I think checking for that single variable would be better than nothing - would give you possibility to provide helpful error message instead of that error message I have pasted before.

Is it common to use roles and gather_facts: false? If so, can you point me at such roles that work under those conditions?

Sorry, no idea. Anyway it is a valid option for Ansible playbook. Maybe it is just me who tends to write roles like that. linux-system-roles.timesync have the same issue, have not tried others.

What is the use case for using gather_facts: false?

It just saves few seconds of run-time (and some memory I assume).

Anyway it is up to you - I do not insist on having this fixed. Existence of this issue is IMHO a +- prove that I'm not the only one hitting it.

from network.

Every role has (or should have) a preamble like this in the tasks/main.yml - https://github.com/linux-system-roles/kernel_settings/blob/master/tasks/main.yml#L2-L15 for example

- name: Set version specific variables
  include_vars: "{{ lookup('first_found', ffparams) }}"
  vars:
    ffparams:
      files:
        - "{{ ansible_facts['distribution'] }}_\
          {{ ansible_facts['distribution_version'] }}.yml"
        - "{{ ansible_facts['distribution'] }}_\
          {{ ansible_facts['distribution_major_version'] }}.yml"
        - "{{ ansible_facts['distribution'] }}.yml"
        - "{{ ansible_facts['os_family'] }}.yml"
        - "default.yml"
      paths:
        - "{{ role_path }}/vars"

so at a minimum, every role would need to add something before this like

- name: Gather system facts necessary for role
  setup:
    gather_subset: # maybe there is a subset like "basic system facts"?
    filter:
      - ansible_distribution
      - ansible_os_family
      - ansible_distribution_version
      - ansible_distribution_major_version
      - etc. # role specific ones
    # or use role variables e.g.
    filter: "{{ __system_role_common_facts + __role_specific_facts }}"

from network.

There can be a reason why user set gather_facts: false, so we cannot force doing this. I'd rather roles to fail with an appropriate message, like `This role does not work with 'gather_facts: false'.

from network.

so at a minimum, every role would need to add something before this like

- name: Gather system facts necessary for role
  setup:
    gather_subset: # maybe there is a subset like "basic system facts"?
    filter:
      - ansible_distribution
      - ansible_os_family
      - ansible_distribution_version
      - ansible_distribution_major_version
      - etc. # role specific ones
    # or use role variables e.g.
    filter: "{{ __system_role_common_facts + __role_specific_facts }}"

It seems to me that the filter is not needed and it will probably also not speed up the fact gathering. Maybe the gather_subset makes sense. To me it seems to be most important to have a "when:" statement to ensure it is not done for each role again and again, if a playbook contains multiple roles.

from network.

Does any other role try to support running with gather_facts: false? I think Ansible generally does not recommend running roles with gather_facts: false, it can be used as a workaround when you run private playbooks that you know do not need any facts. I think we must be consistent with Ansible best practices here.
The issue I see here is that the role fails with an unclear error message when user attempts to run it with gather_facts: no. I suppose that would happen with many other roles in AH too.

from network.

#494 addresses this

from network.