gluster / gluster-ansible Goto Github PK

ctdb roles include:

• ctdb_setup
• ctdb_enable
• ctdb_disable
• ctdb_stop
• ctdb_start

Comments, discussion (if any) , all things related to migration of gluster-ansible roles to ansible galaxy goes here,

Requesting you to give comments and vote if you agree or do not agree as well,

These are some points which i could think of :-

Advantages:-

1. Automate the complete flow (Tag, maintain, automate changelog) , for every release.

2. No further builds into copr repos required , since we latch onto ansible for installing roles (i.e no more yum install or dnf install we can use something like:- ansible-galaxy install role-name)

Disadvantages:-
Nothing i can think of.

Thanks,
Prajith.

Create a 3-way replicated or 3-way distributed-replicated volume with client side meta-data caching turned on.

• Write an example playbook for the above

Setup a trusted storage pool with nodes (bare metal or VM) that have been configured with bricks (RAID 6 or JBODs) depending on volume types. (bare metal or VM) that have been configured with bricks (RAID 6 or JBODs) depending on volume types.

It is likely that a clutch of issues around this topic exists. The basic idea is to ensure that prior to a configuration and deployment, a set of pre-flight health checks are conducted for system resources etc.

https://docs.openshift.com/container-platform/3.9/install_config/install/advanced_install.html#configuring-cluster-pre-install-checks and https://docs.openshift.com/container-platform/3.9/admin_guide/diagnostics_tool.html#ansible-based-tooling-health-checks are added here for reference purposes

I am attempting to use gluster-ansible for CentOS 8 and noticed that the gluster-ansible package which pulls in all the other related gluster-ansible-* packages as dependencies is not built for CentOS 8. It is available for CentOS 7 and other Fedora versions.

Is this an oversight or is this intentional?

Sachin Prabhu

The system roles include:

• backend_setup - A generic backend setup for GlusterFS volume.
• backend_reset - A generic backend reset for GlusterFS volume.
• glusterfs_pv - Create physical volume.
• glusterfs_vg - Create a volume group.
• glusterfs_lv - Create a logical volume.
• mount - mount a logical volume, or a GlusterFS volume.
• umount - umount a logical volume, or a GlusterFS volume.

something like

# for debian buster
---
- name: Add glusterfs key
  apt_key:
    url: https://download.gluster.org/pub/gluster/glusterfs/7/rsa.pub
    state: present

- name: Add glusterfs repo
  apt_repository:
    repo: 'deb [arch=amd64] https://download.gluster.org/pub/gluster/glusterfs/7/LATEST/Debian/buster/amd64/apt buster main'
    state: present

- name: Install deps
  apt:
    name:
      - lvm2
      - xfsprogs
    state: present

- name: Install gluster server
  apt:
    name: glusterfs-server

- name: Start and enable service
  service:
    name: glusterd
    state: restarted

geo-replication modules include the following:

• georep_create - Create a geo-replication session (Secure and root)
• georep_delete - Delete a geo-replication session
• georep_pause - Pause a geo-replication session
• georep_resume - Resume a geo-replication session
• georep_start - Start a geo-replication session
• georep_stop - Stop a geo-replication session

nfs-ganesha roles include:

• nfs_ganesha_create
• nfs_ganesha_destroy
• nfs_ganesha_add_node
• nfs_ganesha_del_node
• nfs_ganesha_export_vol
• nfs_ganesha_unexport_vol
• nfs_ganesha_refresh_conf

glusterfs_snapshot roles include:

• glusterfs_snapshot_create
• glusterfs_snapshot_delete
• glusterfs_snapshot_activate
• glusterfs_snapshot_deactivate
• glusterfs_snapshot_clone
• glusterfs_snapshot_restore
• glusterfs_snapshot_config

Create an arbitrated or distributed-replicated arbitrated volume

• Example playbook for the above

glusterfs_clients roles include:

• glusterfs_client_mount
• glusterfs_client_umount

One workflow of the execution of roles for the creation of a basic cluster can be worked upon.
Example: Install RHGS, peer-probe nodes, set up backend, create a distribute-replicate volume
using the Glusterfs.infra roles available.

Short term (3-6 months)

• configure SSL for the control and data path
• configure auth.allow to lock down client connections to a specified list of IP's
• configure glusterd to support gfapi (rhel 7.3/ovirt 4 timeframe)
• configure systemd slice, and glusterd.service overrides for CPU control

Medium term 6-9 months

• support gluster based tiering in addition to lvmcache
• automatically configure the lvmcache lv sizes instead of having them hard-coded by the admin in the conf file - still support current settings as an override, but the tool should make some sensible default choices, based on device size.

How would gadmin validate that all the necessary gluster-ansible roles and playbooks are installed on the system?

GlusterFS volume for all the supported configurations

• 4+2
• 8+4
• 8+3
• 8+2
  *16+4

Write example playbooks for the above configuration.

I am trying to create a Glsuter volume with 2 replica and 1 arbiter brick. Below is the ansible task i use as per the documentation. However i am getting the below error. Any advise on this?

• name: Configure Gluster volume.
  gluster_volume:
  state: present
  name: "{{ gluster_brick_name }}"
  brick: "{{ gluster_brick_dir }}"
  replicas: 2
  arbiters: 1
  cluster: "{{ gluster_cluster }}"
  host: "{{ inventory_hostname }}"
  force: yes
  run_once: true

Error:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: None fatal: [node1]: FAILED! => {"changed": false, "msg": "error running gluster (/sbin/gluster --mode=script volume create gluster replica 3 arbiter 1 transport tcp node1:/BK/gluster/brick node2:/BK/gluster/brick force) command (rc=1): number of bricks is not a multiple of replica count\n\nUsage:\nvolume create <NEW-VOLNAME> [stripe <COUNT>] [replica <COUNT> [arbiter <COUNT>]] [disperse [<COUNT>]] [disperse-data <COUNT>] [redundancy <COUNT>] [transport <tcp|rdma|tcp,rdma>] <NEW-BRICK>... [force]\n\n"}

Currently we use the following variable for setting up cache:

     gluster_infra_cache_vars:
       - vgname: vg_vdc
         cachedisk: '/dev/vdd'
         cachelvname: 'cachelv_thinpool_vg_vdc'
         cachethinpoolname: 'thinpool_vg_vdc'
         cachelvsize: '50G'
         cachemetalvsize: '2G'
         cachemetalvname: 'cache_thinpool_vg_vdc'
         cachemode: 'writethrough'

We currently mention the disk which has to be used for setting up the cache.
And internally use this device to extend the volume group vg_vdc. However since Ansible 2.8
this will not work. We have to also mention the disks which are currently part of the volume group vg_vdc.

For example if vg_vdc is made of /dev/vdc then the variable should look like:

     gluster_infra_cache_vars:
       - vgname: vg_vdc
         cachedisk: '/dev/vdc,/dev/vdd'
         cachelvname: 'cachelv_thinpool_vg_vdc'
         cachethinpoolname: 'thinpool_vg_vdc'
         cachelvsize: '50G'
         cachemetalvsize: '2G'
         cachemetalvname: 'cache_thinpool_vg_vdc'
         cachemode: 'writethrough'

We have to make this change while setting up cache for GlusterFS backend.

Quota roles include:

• quota_enable
• quota_disable
• quota_remove
• quota_remove_objects
• quota_default_soft_limits
• quota_limit_usage
• quota_limit_objects
• quota_alert_time
• quota_soft_timeout
• quota_hard_timeout

Title says it all :)

Reading all the issues and documentation i dont have an idea how to deploy glusterfs using these playbooks

Gadmin integration strategy is being discussed here: gluster/gadmin#4

Based on my understanding of ansible, it seems that writing a plugin would enable us to get the cleanest integration between gadmin and gluster-ansible. I'm considering the following overall criteria:

• The playbooks shipped by gluster-ansible should be directly usable via gadmin, without requiring any modifications. This means that a handler based functionality would not work, as gluster-ansible playbooks need to function in a normal ansible execution by the user. If the handlers are included in the playbooks that function specifically for gadmin, it would impair gluster-ansible's native functionality.
• Plugin selection is done at runtime via an environment variable. This means that the playbooks can simply be run with or without the plugin and hence serve both normal ansible and gadmin execution, without modifications.
• Plugins are the way ansible enables 'customisation' of the output from a play. Ansible itself ships with plugins for writing to a log file, JSON output etc. However, in the case of gadmin, we need a combination of the two.
• While ansible recommends that plugins be merged into ansible core itself, it does allow plugins to ship with the roles. This means that both gluster-ansible and gadmin can keep on with the development by shipping a plugin with gluster-ansible, till it is integrated in ansible core.

Required functionality

• Gadmin needs to supply an output file name. This could be done via an environment variable whose value is read by the plugin. If the variable is undefined, the output should be sent directly to stdout.
• The plugin needs to output in a well-defined parsable format such as JSON or YAML.
• A schema for the output needs to be maintained such that the parsing code can be implemented in gadmin. To start with, it would be necessary to define the following fields:
  • task name
  • success or failure
  • message
  • dictionary of variables and values that may be relevant to the end user

The actual output message and variables need to be decided per-playbook. However, the plugin itself needs to provide the infrastructure first.

GlusterFS core modules include:

• peer_probe --- Peer probe a node; add a node to cluster
• peer_detach --- Detach a node from the cluster
• volume_create --- Create a GlusterFS volume
• volume_delete --- Delete a GlusterFS volume

When trying this example, instead of a list of IPs, I'm getting a Python exception - it fails (I think) on the 'strip' that is missing - https://github.com/ansible/ansible/blob/363ec65062172f0c595a9f91a93be6ab2e26228f/lib/ansible/modules/storage/glusterfs/gluster_volume.py#L321

   node-1: task path: /etc/ansible/roles/gluster.cluster/roles/gluster_volume/tasks/main.yml:2
    node-1: An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'dict' object has no attribute 'strip'
    node-1: fatal: [node-1]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1546349804.09-227968537903750/AnsiballZ_gluster_volume.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1546349804.09-227968537903750/AnsiballZ_gluster_volume.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1546349804.09-227968537903750/AnsiballZ_gluster_volume.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_gluster_volume_payload_BDtslG/__main__.py\", line 527, in <module>\n  File \"/tmp/ansible_gluster_volume_payload_BDtslG/__main__.py\", line 451, in main\n  File \"/tmp/ansible_gluster_volume_payload_BDtslG/__main__.py\", line 298, in probe_all_peers\nAttributeError: 'dict' object has no attribute 'strip'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

If I'm using something like:

     gluster_cluster_hosts:
       - 192.168.250.11
       - 192.168.250.12
       - 192.168.250.13

Everything works fine.

Gluster doc directly leads to https://github.com/gluster/gluster-ansible as the preferred way to install and configure glusterfs for ansible users.

However this GitHub project doesn't clearly states how to use it from ansible :

• is it available at ansible Galaxy?
• it seems parts have been integrated into the official ansible modules but not all (?)
• is it a plain ansible module (I couldn't make it work as-is)
• does it require to download each submodules separately

It would be nice to clarify this, thanks!