gluster / glustercli-python Goto Github PK

View Code? Open in Web Editor NEW

19.0 10.0 33.0 132 KB

Python bindings for Gluster Commands and Metrics

License: GNU General Public License v2.0

Python 98.27% Makefile 1.73%

glustercli-python's Introduction

glustercli-python

Python bindings for GlusterFS CLI and Metrics.

Hello World

from glustercli.cli import volume

# Create a Volume
volume.create("gv1", ["fvm1:/bricks/b1", "fvm2:/bricks/b2"],
              force=True)

# Start Volume
volume.start("gv1")

# Read Volume Info
volume.info("gv1")

# Get GlusterFS version
from glustercli.cli import glusterfs_version
print(glusterfs_version())

Install

sudo pip3 install glustercli

Documentation

See Documentation

Install pydoc-markdown package to generate Documentation.

sudo pip3 install pydoc-markdown

glustercli-python's People

Contributors

Stargazers

Watchers

glustercli-python's Issues

`volume.vollist()` only shows one volume when more than one exist

Trying to debug an issue with the volume.vollist() function only returning a single volume although more than one volume exist and are started

some relevant info about this test setup:

The VMs are:

[root@gluster1 vagrant]# cat /etc/*release
CentOS Linux release 7.5.1804 (Core) 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.5.1804 (Core) 
CentOS Linux release 7.5.1804 (Core) 
[root@gluster1 vagrant]#

RPM's installed

[root@gluster1 vagrant]# rpm -qa | grep gluster
centos-release-gluster41-1.0-3.el7.centos.noarch
glusterfs-libs-4.1.2-1.el7.x86_64
glusterfs-client-xlators-4.1.2-1.el7.x86_64
glusterfs-cli-4.1.2-1.el7.x86_64
glusterfs-fuse-4.1.2-1.el7.x86_64
glusterfs-server-4.1.2-1.el7.x86_64
glusterfs-4.1.2-1.el7.x86_64
glusterfs-api-4.1.2-1.el7.x86_64

glustercli-python version

[root@gluster1 vagrant]# pip freeze | grep gluster
You are using pip version 8.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
glustercli==0.3

Demo of the Bug

[root@gluster1 vagrant]# python
Python 2.7.5 (default, Sep 15 2016, 22:37:39) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gluster.cli import volume
>>> volume.vollist()
['gv0']
>>>

but using cli

[root@gluster1 vagrant]# gluster vol list
gv0
gv1

Release 0.8.2 contains the wrong version internally

Running egg_info on the latest release produces this warning:

UserWarning: The version specified ('master') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.

[RFE] New wrapper: glusterfs_version

What do you think of adding glusterfs_version variable to glustercli?

i.e for example...

from glustercli import cli
cli.glusterfs_version

RFE from @sac

Package Rename to glustercli

Currently this package depends on GlusterFS installation and installs under the installed package. That means Glusterfs installs meta package gluster and this package will be installed under gluster/cli. This introduces issues when gluster package is uninstalled or reinstalled. Change the package name to glustercli and create it as independent package.

All existing users need to change from

from gluster.cli import volume

from glustercli.cli import volume

issue with volume_status parser with older versions of glusterfs (3.6.5)

Currently installed on Centos 7

Using the latest glustercli glustercli==0.5 with this commit: 442e256

# rpm -qa | grep gluster
glusterfs-geo-replication-3.6.5-1.el7.x86_64
glusterfs-3.6.5-1.el7.x86_64
glusterfs-libs-3.6.5-1.el7.x86_64
glusterfs-cli-3.6.5-1.el7.x86_64
glusterfs-fuse-3.6.5-1.el7.x86_64
glusterfs-debuginfo-3.6.5-1.el7.x86_64
glusterfs-api-3.6.5-1.el7.x86_64
glusterfs-server-3.6.5-1.el7.x86_64

Output of the volume status from an active volume

[root@server01 infra]# gluster vol status archive detail --xml                                      
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
  <volStatus>
    <volumes>
      <volume>
        <volName>archive</volName>
        <nodeCount>1</nodeCount>
        <node>
          <hostname>server03</hostname>
          <path>/export/raid/archive</path>
          <peerid>6a9da7fc-70ec-4302-8152-0e61929a7c8b</peerid>
          <status>1</status>
          <port>49152</port>
          <pid>26298</pid>
          <sizeTotal>10994207162368</sizeTotal>
          <sizeFree>2949546762240</sizeFree>
          <device>/dev/sdh1</device>
          <blockSize>4096</blockSize>
          <mntOptions>rw,relatime,attr2,inode64,noquota</mntOptions>
          <fsName>xfs</fsName>
        </node>
      </volume>
    </volumes>
  </volStatus>
</cliOutput>

Error Parsing volume status details

# python                                                                                                                                                                         [25/594]
Python 2.7.5 (default, Jun 24 2015, 00:41:19) 
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gluster.cli import volume
>>> vol_list = volume.vollist()
>>> volume.info('archive')                                                                               
[{'status': 'Started', 'bricks': [{'name': 'server03:/export/raid/archive', 'uuid': '6a9da7fc-70ec-4302-8152-0e61929a7c8b'}], 'distribute': 1, 'replica': 1, 'stripe': 1, 'num_bricks': 
1, 'transport': 'TCP', 'uuid': 'd7020552-60f5-4b06-b546-dea27815c63c', 'name': 'archive', 'type': 'DISTRIBUTE', 'options': [{'name': 'nfs.acl', 'value': 'off'}]}]
>>> volume.status_detail(volname='archive')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/infra/glustercli-python/gluster/cli/volume.py", line 176, in status_detail
    return parse_volume_status(volume_execute_xml(cmd), info(volname))
  File "/usr/infra/glustercli-python/gluster/cli/parsers.py", line 101, in parse_volume_status
    nodes_data = _parse_volume_status(status_data)
  File "/usr/infra/glustercli-python/gluster/cli/parsers.py", line 95, in _parse_volume_status
    raise GlusterCmdOutputParseError(e)
gluster.cli.parsers.GlusterCmdOutputParseError: 'NoneType' object has no attribute 'find'

Problem is not seen on a newer version of glusterfs with latest glustercli-python
An example of xml dump from a newer version of glusterfs

[root@gluster1 vagrant]# gluster vol status gv0 detail --xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
  <volStatus>
    <volumes>
      <volume>
        <volName>gv0</volName>
        <nodeCount>2</nodeCount>
        <node>
          <hostname>gluster1</hostname>
          <path>/bricks/brick1/gv0</path>
          <peerid>75c1ee2c-b0cb-455f-bcc1-957929a984ce</peerid>
          <status>1</status>
          <port>49152</port>
          <ports>
            <tcp>49152</tcp>
            <rdma>N/A</rdma>
          </ports>
          <pid>12681</pid>
          <sizeTotal>40446885888</sizeTotal>
          <sizeFree>39277514752</sizeFree>
          <device>/dev/mapper/VolGroup00-LogVol00</device>
          <blockSize>4096</blockSize>
          <mntOptions>rw,seclabel,relatime,attr2,inode64,noquota</mntOptions>
          <fsName>xfs</fsName>
          <inodeSize>xfs</inodeSize>
          <inodesTotal>39518208</inodesTotal>
          <inodesFree>39484411</inodesFree>
        </node>
        <node>
          <hostname>gluster2</hostname>
          <path>/bricks/brick1/gv0</path>
          <peerid>47111a73-8c59-4d2f-a2d2-295e558b91fc</peerid>
          <status>1</status>
          <port>49152</port>
          <ports>
            <tcp>49152</tcp>
            <rdma>N/A</rdma>
          </ports>
          <pid>12661</pid>
          <sizeTotal>40446885888</sizeTotal>
          <sizeFree>39429644288</sizeFree>
          <device>/dev/mapper/VolGroup00-LogVol00</device>
          <blockSize>4096</blockSize>
          <mntOptions>rw,seclabel,relatime,attr2,inode64,noquota</mntOptions>
          <fsName>xfs</fsName>
          <inodeSize>xfs</inodeSize>
          <inodesTotal>39518208</inodesTotal>
          <inodesFree>39489389</inodesFree>
        </node>
      </volume>
    </volumes>
  </volStatus>
</cliOutput>

(gluster packages installed)

[root@gluster1 vagrant]# rpm -qa | grep gluster
centos-release-gluster41-1.0-3.el7.centos.noarch
glusterfs-libs-4.1.2-1.el7.x86_64
glusterfs-client-xlators-4.1.2-1.el7.x86_64
glusterfs-cli-4.1.2-1.el7.x86_64
glusterfs-fuse-4.1.2-1.el7.x86_64
glusterfs-server-4.1.2-1.el7.x86_64
glusterfs-4.1.2-1.el7.x86_64
glusterfs-api-4.1.2-1.el7.x86_64

gluster volume profile needed.

parsers.py failing to lack of checks for value existence..

I'm running a GlusterFS cluster which is distributed across a 3 VMs and 1 Container (Which is running on a NAS, so we can snapshot the data). I've noticed that glustercli is not working correctly, I'm seeing the following error:

  File "/root/gdash-master/gdash/glustercli/cli/parsers.py", line 268, in _parse_volume_status
    nodes.append(_parse_a_node(node_el))
  File "/root/gdash-master/gdash/glustercli/cli/parsers.py", line 237, in _parse_a_node
    'inodes_total': int(node_el.find('inodesTotal').text),
AttributeError: 'NoneType' object has no attribute 'text'

It looks like there is an issue in the parsers.py > _parse_a_node function, there is no check for the values, its assuming all values are always present

    value = {
        'name': brick_path,
        'uuid': node_el.find('peerid').text,
        'online': True if node_el.find('status').text == "1" else False,
        'pid': node_el.find('pid').text,
        'size_total': int(node_el.find('sizeTotal').text),
        'size_free': int(node_el.find('sizeFree').text),
        'inodes_total': int(node_el.find('inodesTotal').text),
        'inodes_free': int(node_el.find('inodesFree').text),
        'device': node_el.find('device').text,
        'block_size': node_el.find('blockSize').text,
        'mnt_options': node_el.find('mntOptions').text,
        'fs_name': node_el.find('fsName').text,
    }

Which is incorrect, if you see below, there is no inodes_total or inodes_free values:

        <node>
          <hostname>syn-02-gfs</hostname>
          <path>/gfs</path>
          <peerid>49194ed8-1942-4bae-9efa-0deef473e714</peerid>
          <status>1</status>
          <port>49152</port>
          <ports>
            <tcp>49152</tcp>
            <rdma>N/A</rdma>
          </ports>
          <pid>43</pid>
          <sizeTotal>4123168604160</sizeTotal>
          <sizeFree>1940550733824</sizeFree>
          <device>/dev/mapper/cachedev_0</device>
          <blockSize>4096</blockSize>
          <mntOptions>rw,nodev,relatime,ssd,synoacl,space_cache=v2,auto_reclaim_space,metadata_ratio=50,subvolid=328,subvol=/@syno/CNTRS-02/data/glusterfs-k3/gfs</mntOptions>
          <fsName>btrfs</fsName>
          <inodeSize>btrfs</inodeSize>
        </node>