ansible-community / molecule-openstack Goto Github PK

View Code? Open in Web Editor NEW

15.0 20.0 11.0 64 KB

Molecule OpenStack Driver

Home Page: https://github.com/ansible-community/molecule-plugins/discussions

License: MIT License

Python 100.00%

molecule-driver molecule ansible hacktoberfest

molecule-openstack's Introduction

Molecule OpenStack Plugin

Molecule OpenStack is designed to allow use of OpenStack Clouds for provisioning test resources.

Please note that this driver is currently in its early stage of development.

Installation and Usage

Install molecule-openstack and pre-requisites:

pip install molecule-openstack ansible openstacksdk

Create a new role with molecule using the openstack driver:

molecule init role <role_name> -d openstack

Configure <role_name>/molecule/default/molecule.yaml with required parameters based on your openstack cloud. A simple config is:

dependency:
   name: galaxy
driver:
   name: openstack
platforms:
- name: molecule-foo
   image: "ubuntu"
   flavor: "m1.medium"
   network: "private"
   fip_pool: "public"
   ssh_user: "ubuntu"
provisioner:
   name: ansible
verifier:
   name: ansible

Argument fip_pool in only required when network is not an external network. Instead of configuring <role_name>/molecule/default/molecule.yaml the following environment variables can be exported:

$ export MOLECULE_OPENSTACK_IMAGE=ubuntu
$ export MOLECULE_OPENSTACK_FLAVOR=m1.medium
$ export MOLECULE_OPENSTACK_NETWORK=private
$ export MOLECULE_OPENSTACK_FIP_POOL=public
$ export MOLECULE_OPENSTACK_SSH_USER=ubuntu

After this molecule can be run from the base-dir of the role:

source ~/.openrc
molecule test

Functional Tests

Functional tests can be run with tox but require access to an openstack cluster. They are not part of ci yet. To run them locally:

$ export MOLECULE_OPENSTACK_IMAGE=<image_name>
$ export MOLECULE_OPENSTACK_FLAVOR=<flavor>
$ export MOLECULE_OPENSTACK_NETWORK=<network>
$ export MOLECULE_OPENSTACK_FIP_POOL=<fip_pool_if_required>
$ export MOLECULE_OPENSTACK_SSH_USER=<ssh_user>

$ source openstack_openrc.sh

$ tox -e py38-functional   # or 39,310

Get Involved

Join us in the #ansible-devtools channel on Libera.
Join the discussion in molecule-users Forum.
Join the community working group by checking the wiki.
Want to know about releases, subscribe to ansible-announce list.
For the full list of Ansible email Lists, IRC channels see the communication page.

License

The MIT License.

The logo is licensed under the Creative Commons NoDerivatives 4.0 License.

If you have some other use in mind, contact us.

molecule-openstack's People

Contributors

Stargazers

Watchers

Forkers

ssbarnea berendt alexandruivan swipswaps lportinari gstorme dingus9 mnaser ziegenberg amen619 zijlstra-it

molecule-openstack's Issues

Running testinfra tests with openstack delegated driver randomly fails

Issue Type

Bug report

Molecule and Ansible details

ansible --version && molecule --version
ansible 2.9.9
  config file = None
  configured module search path = ['/home/mat/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/ansible
  executable location = /home/mat/Virtualenv/ansible3/bin/ansible
  python version = 3.8.4rc1 (default, Jul  1 2020, 15:31:45) [GCC 9.3.0]
molecule 3.0.4

Molecule installation method (one of):

Ansible installation method (one of):

Detail any linters or test runners used:

Desired Behavior

Running testinfra tests should always return the same status when no change is made between the execution of 2 tests.

Actual Behaviour

When performing molecule verify command:

the first time all tests passed
the second time all tests failed du to an SSH issue with the openstack instances.

Firstly: all tests passed

molecule verify
--> Test matrix
    
└── default
    └── verify
    
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/mat/GIT/ansible_project/ansible_role/xymon-client/molecule/default/tests/...
    ============================= test session starts ==============================
    platform linux -- Python 3.8.4rc1, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
    rootdir: /home/mat/GIT/ansible_project/ansible_role/xymon-client/molecule/default
    plugins: testinfra-5.2.1
collected 8 items                                                              
    
    tests/test_default.py ..                                                 [ 25%]
    tests/test_common.py ......                                              [100%]
    
    ============================== 8 passed in 15.70s ==============================
Verifier completed successfully.

Secondly: all tests failed:

When retrying to perform molecule verify* command 5 seconds after the first try, all tests failed with the error:

RuntimeError: CommandResult(command=b'test -e /etc/default/xymon-client', exit_status=255, stdout=None, stderr=b'kex_exchange_identification: read: Connection reset by peer\r\n')

molecule verify
--> Test matrix
    
└── default
    └── verify
    
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/mat/GIT/ansible_project/ansible_role/xymon-client/molecule/default/tests/...
    ============================= test session starts ==============================
    platform linux -- Python 3.8.4rc1, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
    rootdir: /home/mat/GIT/ansible_project/ansible_role/xymon-client/molecule/default
    plugins: testinfra-5.2.1
collected 8 items                                                              
    
    tests/test_default.py FF                                                 [ 25%]
    tests/test_common.py FFFFFF                                              [100%]
    
    =================================== FAILURES ===================================
    _____________ test_xymon_client_default[ansible://xymon-debian10] ______________
    
    host = <testinfra.host.Host ansible://xymon-debian10>
    
        def test_xymon_client_default(host):
            f = host.file('/etc/default/xymon-client')
            ansible_vars = host.ansible.get_variables()
            hostname = ansible_vars['inventory_hostname']
    >       assert f.contains('^XYMONSERVERS="127.0.0.1"$')
    
    tests/test_default.py:13:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/modules/file.py:118: in contains
        return self.run_test("grep -qs -- %s %s", pattern, self.path).rc == 0
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/host.py:94: in run_test
        return self.run_expect([0, 1], command, *args, **kwargs)
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/host.py:75: in run
        return self.backend.run(command, *args, **kwargs)
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/backend/ansible.py:47: in run
        return host.run(command)
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/host.py:75: in run
        return self.backend.run(command, *args, **kwargs)
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/backend/ssh.py:34: in run
        return self.run_ssh(self.get_command(command, *args))
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    self = <testinfra.backend.ssh.SshBackend object at 0x7f908138c310>
    command = 'grep -qs -- \'^XYMONSERVERS="127.0.0.1"$\' /etc/default/xymon-client'
    
        def run_ssh(self, command):
            cmd, cmd_args = self._build_ssh_command(command)
            out = self.run_local(
                " ".join(cmd), *cmd_args)
            out.command = self.encode(command)
            if out.rc == 255:
                # ssh exits with the exit status of the remote command or with 255
                # if an error occurred.
    >           raise RuntimeError(out)
    E           RuntimeError: CommandResult(command=b'grep -qs -- \'^XYMONSERVERS="127.0.0.1"$\' /etc/default/xymon-client', exit_status=255, stdout=None, stderr=b'kex_exchange_identification: read: Connection reset by peer\r\n')
    
    /home/mat/Virtualenv/ansible3/lib/python3.8/site-packages/testinfra/backend/ssh.py:77: RuntimeError
    ______________ test_xymon_client_default[ansible://xymon-debian9] ______________
.....

I've rewritten the test in ansible instead of testinfra and with ansible verifier, there is no SSH issue.
So I guess something weird happens between testinfra and SSH when running molecule verify command.

Additionnal info

molecule.yml

dependency:
  name: galaxy
driver:
  name: delegated
  security_group: xymon-default-molecule
  network: molecule
  ssh_user: admin
  keypair: xymon-default
lint: |
  set -e
  yamllint .
  ansible-lint -x 503,303 molecule/default/*.yml
platforms:
  - name: xymon-debian10
    image: Debian_10
    flavor: clouds.small
  - name: xymon-debian9
    image: Debian_9
    flavor: clouds.small
provisioner:
  name: ansible
  config_options:
    defaults:
      ansible_managed: 'Molecule Tests'
verifier:
  name: testinfra

converge.yml

- name: Converge
  hosts: all
  become: true
  vars:
    ansible_python_interpreter: auto_silent
  pre_tasks:
    - name: update apt cache
      apt:
        update_cache: true
      changed_when: false
  roles:
    - role: xymon-client

testinfra

test_common.py

import os

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


def test_package(host):
    p = host.package('xymon-client')
    assert p.is_installed


def test_service(host):
    s = host.service('xymon-client')
    assert s.is_running
    assert s.is_enabled


def test_xymon_client_default(host):
    f = host.file('/etc/default/xymon-client')
    assert f.exists
    assert f.is_file
    assert f.user == 'root'
    assert f.group == 'root'
    assert f.mode == 0o644

test_default.py

import os

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


def test_xymon_client_default(host):
    f = host.file('/etc/default/xymon-client')
    ansible_vars = host.ansible.get_variables()
    hostname = ansible_vars['inventory_hostname']
    assert f.contains('^XYMONSERVERS="127.0.0.1"$')
    assert f.contains('^CLIENTHOSTNAME="%s"$' % hostname)

Update cookiecutter template with necessary changes

Remove with_items and use loops based on: https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#with-items
Remove molecule filters: Related: ansible/molecule#2872

Unable to override the ssh connection properties

I want to change this:
-o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey to this
-o PreferredAuthentications=publickey

Because the driver appends the options that I add under ssh_connection_options, I am not able to achieve that result.. Unfortunately, ssh works like this:

ssh(1) obtains configuration data from the following sources in the
     following order:

           1.   command-line options
           2.   user's configuration file (~/.ssh/config)
           3.   system-wide configuration file (/etc/ssh/ssh_config)

For each parameter, the first obtained value will be used.

How could I have those options changed?

creation new scenario with driver is missing manadatory options

At this moment these two options are missing from molecule.yml when creating a new scenario:

---
dependency:
  name: galaxy
driver:
  name: openstack
platforms:
  - name: instance
    image: foo  # <-- missing
    flavor: bar  # <-- missing
provisioner:
  name: ansible
verifier:
  name: ansible

The idea is to find a way to have something created there, something that is likely to work for the user.

molecule fails on lookup when instance_config.yml is not present

When running molecule on a role for the first time instance_config.yml is not present under $HOME/.cache/molecule/<role_name>/<scenario_name>/instance_config.yml therefore the destroy.yml fails due to lookup failing with:

$ molecule destroy -s test
INFO     test scenario test matrix: dependency, cleanup, destroy
INFO     Running test > dependency
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the requirements file.
INFO     Running test > cleanup
WARNING  Skipping, cleanup playbook not configured.
INFO     Running test > destroy

PLAY [Destroy] ************************************************************************

TASK [Destroy molecule instance(s)] ***************************************************
[WARNING]: Unable to find '/home/nikos/.cache/molecule/surfsara-
postfix/test/instance_config.yml' in expected paths (use -vvvvv to see paths)

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: /home/nikos/.cache/molecule/surfsara-postfix/test/instance_config.yml"}

PLAY RECAP ****************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/nikos/.cache/molecule/surfsara-postfix/test/inventory --skip-tags molecule-notest,notest /home/nikos/repo/os-provisioning/roles/surfsara-postfix/molecule/test/destroy.yml

Allow ssh_user to be defined per instance: This can be really useful when testing a role across different images/os etc. Also by defining it on molecule.yml users wont need to meddle with create.yml.
Allow the usage of fip_pools: It is quite often that openstack is set to not allow users to directly hook-up to external networks. a fip is required for external connectivity. The existing template does not allow to define this
Handle instance destruction by id: Currently destroy.tml tries to delete the vm by name. This will fail if there are multiple vm's with the same name. This can be often if multiple developers working on the same role of/and molecule is part of ci/cd.

Create fails with `Floating IP pool not found`

Working my way through the readme instructions.

I have a network which is apparently external - from Horizon:

Name	Subnets Associated	Shared	External
ilab	ilab 10.60.0.0/16     Yes	Yes

However if I try to run molecule create without specifying fip_pool I get an error:

"msg": "ResourceNotFound: 404: Client Error for url: http://10.60.253.1:8774/v2.1/os-floating-ips, Floating IP pool not found."}

This a Neutron-managed network so "floating IP pools" don't exist. I tried setting network name as the fip_pool but that fails with:

"msg": "Timeout waiting for the floating IP to be attached."