redhat-cop / aap_configuration_template Goto Github PK

As per tower_configuration, the default branch should be named devel.

A new branch devel should be taken from master, set as the new default, then master deleted

There's been similar content before and was used by our team and others in the last few years. Maybe there's something in here we can reuse or get ideas from.

https://github.com/dfederlein/vagrant-tower

https://github.com/dfederlein/ansible-tower-demo

These roles will be in the tower_utilities collection once redhat-cop/aap_utilities#5 is merged and an initial release has been made to galaxy

I'm trying to use hub_config.yml to add 3 collection_remote's, "rh-certified", "validated", "community".
In my case, these 3 remotes already exist.

This task fails for me nearly every time because Automation Hub returns HTTP Error 504: Gateway Time-out.
Automation Hub Task Manager shows no recent tasks.

AAP Job Output

...
TASK [infra.ah_configuration.collection_remote : Validating arguments against arg spec 'main']
ok: [localhost] ...

TASK [infra.ah_configuration.collection_remote : Add Automation Hub Collection Remote repository]
ok: [localhost] ...

TASK [infra.ah_configuration.collection_remote : Create Repository | Wait for the repository creation]
FAILED - RETRYING: [localhost]: Create Repository | Wait for the repository creation (15 retries left)
FAILED - RETRYING: [localhost]: Create Repository | Wait for the repository creation (14 retries left)
failed: [localhost] ... "msg": "Error while getting server version: The host sent back a server error: /api/galaxy/: HTTP Error 504: Gateway Time-out. ..."
...

Pulp logs (from journalctl)

...
Mar 29 15:41:01 dev-automationhub1  gunicorn[579418]: [2024-03-29 15:41:01 -0400] [579418] [CRITICAL] WORKER TIMEOUT (pid:579422)
...

Nginx Access Logs

==>/var/log/nginx/automationhub.access.log<==
10.1.2.3 - - [29/Mar/2024:15:27:05 -0400] "POST /api/galaxy/_ui/v1/auth/login/ HTTP/1.1" 504 160 https://dev-automationhub.example.com "Python-urllib/3.6" "-"
10.1.2.3 - admin [29/Mar/2024:15:28:00 -0400] "GET /api/galaxy/_ui/v1/me/ HTTP/1.1" 504 160 https://dev-automationhub.example.com "Python-urllib/3.6" "-"
10.1.2.3  - - [29/Mar/2024:15:37:01 -0400] "POST /api/galaxy/_ui/v1/auth/login/ HTTP/1.1" 504 160 https://dev-automationhub.example.com "Python-urllib/3.6" "-"
10.1.2.3  - admin [29/Mar/2024:15:38:06 -0400] "GET /api/galaxy/ HTTP/1.1" 504 160 https://dev-automationhub.example.com "Python-urllib/3.6" "-"
10.1.2.3  - admin [29/Mar/2024:15:39:01 -0400] "POST /api/galaxy/_ui/v1/auth/logout/ HTTP/1.1" 504 160 https://dev-automationhub.example.com "Python-urllib/3.6" "-"

Nginx Error Logs

==> /var/log/nginx/automationhub.error.log <==
2024/03/29 15:27:00 [error] 1722181#0: *109935 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.1.2.3, server: dev-automationhub1, request: "POST /api/galaxy/_ui/v1/auth/login/ HTTP/1.1", upstream: http://unix:/var/run/pulpcore-api/pulpcore-api.sock/api/galaxy/_ui/v1/auth/login/, host: "dev-automationhub.example.com", referrer: https://dev-automationhub.example.com
2024/03/29 15:28:00 [error] 1722176#0: *109943 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.1.2.3, server: dev-automationhub1, request: "GET /api/galaxy/_ui/v1/me/ HTTP/1.1", upstream: http://unix:/var/run/pulpcore-api/pulpcore-api.sock/api/galaxy/_ui/v1/me/, host: "dev-automationhub.example.com", referrer: https://dev-automationhub.example.com
2024/03/29 15:37:00 [error] 579325#0: *811 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.1.2.3, server: dev-automationhub1, request: "POST /api/galaxy/_ui/v1/auth/login/ HTTP/1.1", upstream: http://unix:/var/run/pulpcore-api/pulpcore-api.sock/api/galaxy/_ui/v1/auth/login/, host: "dev-automationhub.example.com", referrer: https://dev-automationhub.example.com
2024/03/29 15:38:01 [error] 579326#0: *821 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.1.2.3, server: dev-automationhub1, request: "GET /api/galaxy/ HTTP/1.1", upstream: http://unix:/var/run/pulpcore-api/pulpcore-api.sock/api/galaxy/, host: "dev-automationhub.example.com", referrer: https://dev-automationhub.example.com
2024/03/29 15:39:01 [error] 579326#0: *829 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.1.2.3, server: dev-automationhub1, request: "POST /api/galaxy/_ui/v1/auth/logout/ HTTP/1.1", upstream: http://unix:/var/run/pulpcore-api/pulpcore-api.sock/api/galaxy/_ui/v1/auth/logout/, host: "dev-automationhub.example.com", referrer: https://dev-automationhub.example.com

add tags to all the playbooks so you can configure just specific parts instead of the whole to speed up post configuration updates

Issue: controller_config playbook does not configure Notifications

related files:

update template to use encrypted strings instead of vault file

I can see at least notifications is failing in the pipeline. We should check everything still matches up

I believe there might be an issue with the install_configure playbook that if you provide a different password then the default that it isn't actually added where it needs to be and so the password doesn't work. I ran into this during AnsibleFest 2022 but haven't gotten around to re-testing and verifying the issue and what the cause is

Hello ,

i have this error message during execution of the task ' Run the Ansible Automation Platform setup program':

fatal: [XXXXXXXX]: FAILED! => {
"ansible_job_id": "666487507563.1103234",
"changed": false,
"cmd": "./setup.sh -i inventory -e upgrade_ansible_with_tower=1 -e web_server_ssl_cert=/root/tower-cert/tower.cert -e web_server_ssl_key=/root/tower-cert/tower.key -e automationhub_ssl_cert=/root/tower-cert/tower.cert -e automationhub_ssl_key=/root/tower-cert/tower.key -e automationhub_api_token=xxxxxxxxxxxxxxxxxx",
"finished": 1,
"invocation": {
"module_args": {
"_raw_params": "\n./setup.sh -i "inventory" -e upgrade_ansible_with_tower=1 -e web_server_ssl_cert=/root/tower-cert/tower.cert -e web_server_ssl_key=/root/tower-cert/tower.key -e automationhub_ssl_cert=/root/tower-cert/tower.cert -e automationhub_ssl_key=/root/tower-cert/tower.key -e automationhub_api_token=xxxxxxx\n",
"_uses_shell": false,
"argv": null,
"chdir": "/tmp/ansible-automation-platform-setup-bundle-2.3-2/",
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"stdin_add_newline": true,
"strip_empty_ends": true
}
},
"msg": "[Errno 13] Permission denied: b'./setup.sh'",
"rc": 13,

When installing the project "config_as_code", a task fetch the collections defined in collections/requirements.yml.
In particular, those:

  - name: infra.controller_configuration
  - name: infra.ah_configuration
  - name: infra.ee_utilities
  - name: infra.aap_utilities

I've removed those collections from the 'community' repository because they were colliding with the same collections installed from the validatedrepository. The uniqueness constraint bug is present in AAP 2.4-6 (AH 4.9.1) and a fix is yet to be released.

In the meantime, the task is failing with this error: ERROR! Failed to resolve the requested dependencies map. Could not satisfy the following requirements:\n* infra.controller_configuration:2.6.0 (direct request).

This is because I need to have access to the validated repository that hosts those infra collections.

So in conclusion, I think that the validated repository should be enabled in the controller to cover this scenario.

create some automated testing to validate this works for latest AAP/AWX builds

I want to configure my Public Automation Hub token and add it to an organization.
For this, the file group_vars/all/credentials.yml seems a good candidate.
I didn't find where to set the ah_token variable. Is it an omission?

I found in vaults/dev.yml the variable offline_token with the cryptic comment "this is the one linked below about api token".
This offline_token variable in only used inside group_vars/all/aap_install.yml to fill the variable aap_setup_down_offline_token which serve to download aap here. So it doesn't seem to be the token I'm looking for. And it is not explained how to generate such token.

Also in vaults/dev.yml there is the cloud_token, which is used in group_vars/all/ah_repositories.yml to configure ah_collection_remotes. So this seems to be a better candidate. But the comment says: 'this is the one from console.redhat.com'. A better comment would have been: "To generate an AH token, got to the url: https://console.redhat.com/ansible/automation-hub/token and click on «Load Token» in the "Offline token" chapter".

But wait, I have to load an «offline token» to generate my cloud_token. But then, what's the purpose/difference with offline_token from above`?

Is ah_token and cloud_token the same?

Hi and thanks to all the contributors for this template! It is exactly what I needed.

I thinks the documentation for the installation of AAP could be more explanatory. I'm having trouble using this template to install Ansible Automation Platform and think additional explanations would help. I may offer a pull request once I've sorted this out but first I need your help to understand how this repo works please.

1. It's unclear where the playbooks are meant to be run from - the server hosting one of the controllers or a workstation? I tried running them from my workstation but now have AAP uncompressed in /var/tmp. Not what I wanted.

2. The inventory_env.yml files are new to me. I previously configured the setup.sh inventory for AAP/AWX node types and placements. How do these files work with bundle installation or the Technical Preview containerized installer? I see I need to add aap_setup_down_type: setup-bundle but details would help.

3. Without step-by-step SSH key generation guidance, getting all AAP nodes to communicate is difficult. Do I create a key on a controller node, copy the private key to other controllers, and add the public key to all nodes? More details would help ensure proper setup. As I'm getting permission denied even though the key are in place and work properly. Do I need to mount my .ssh folder inside the EE?

4. I've added a role preparing requirements like firewall and NTP - would you be interested in that contribution once I have the SSH keys working? I can then document the full process. Or is it intentionally left out of the scope of this template?

add remote repos for hub

The GitHub action for spinning up AWX and running playbooks seems to fail each time. I noticed it very occasionally being flaky in tower_configuration but it has never worked here. It seems to fail when waiting for AWX to come alive.

We need to diagnose and fix this issue

When I set controller_settings_all and controller_settings_dev, only the last value wins.
I was expended the variables to be merged.

For instance;

controller_settings_all:
  settings:
    GALAXY_IGNORE_CERTS: true
    AUTH_BASIC_ENABLED: true

controller_settings_dev:
  settings:
    TOWER_URL_BASE: https://aap-control-dev.example.com

When using: controller_settings: "{{ controller_settings_all | combine(controller_settings_dev, list_merge='append') }}"
Then control_settings become:

settings:
  TOWER_URL_BASE: https://aap-control-dev.example.com

I was expecting:

settings:
  GALAXY_IGNORE_CERTS: true
  AUTH_BASIC_ENABLED: true
  TOWER_URL_BASE: https://aap-control-dev.example.com