Giter VIP home page Giter VIP logo

cs's Introduction

CS

License

Python versions

A simple, yet powerful CloudStack API client for python and the command-line.

  • Async support.
  • All present and future CloudStack API calls and parameters are supported.
  • Syntax highlight in the command-line client if Pygments is installed.
  • BSD license.

Installation

pip install cs

# with the colored output
pip install cs[highlight]

# with the async support
pip install cs[async]

# with both
pip install cs[async,highlight]

Usage

In Python:

from cs import CloudStack

cs = CloudStack(endpoint='https://api.exoscale.ch/v1',
                key='cloudstack api key',
                secret='cloudstack api secret')

vms = cs.listVirtualMachines()

cs.createSecurityGroup(name='web', description='HTTP traffic')

From the command-line, this requires some configuration:

cat $HOME/.cloudstack.ini
[cloudstack]
endpoint = https://api.exoscale.ch/v1
key = cloudstack api key
secret = cloudstack api secret
# Optional ca authority certificate
verify = /path/to/certs/exoscale_ca.crt
# Optional client PEM certificate
cert = /path/to/client_exoscale.pem
# If you need to pass the certificate and key as separate files
cert_key = /path/to/client_key.pem

Then:

$ cs listVirtualMachines
{
  "count": 1,
  "virtualmachine": [
    {
      "account": "...",
      ...
    }
  ]
}
$ cs authorizeSecurityGroupIngress \
    cidrlist="0.0.0.0/0" endport=443 startport=443 \
    securitygroupname="blah blah" protocol=tcp

The command-line client polls when async results are returned. To disable polling, use the --async flag.

To find the list CloudStack API calls go to http://cloudstack.apache.org/api.html

Configuration

Configuration is read from several locations, in the following order:

  • The CLOUDSTACK_ENDPOINT, CLOUDSTACK_KEY, CLOUDSTACK_SECRET and CLOUDSTACK_METHOD environment variables,
  • A CLOUDSTACK_CONFIG environment variable pointing to an .ini file,
  • A CLOUDSTACK_VERIFY (optional) environment variable pointing to a CA authority cert file,
  • A CLOUDSTACK_CERT (optional) environment variable pointing to a client PEM cert file,
  • A CLOUDSTACK_CERT_KEY (optional) environment variable pointing to a client PEM certificate key file,
  • A cloudstack.ini file in the current working directory,
  • A .cloudstack.ini file in the home directory.

To use that configuration scheme from your Python code:

from cs import CloudStack, read_config

cs = CloudStack(**read_config())

Note that read_config() can raise SystemExit if no configuration is found.

CLOUDSTACK_METHOD or the method entry in the configuration file can be used to change the HTTP verb used to make CloudStack requests. By default, requests are made with the GET method but CloudStack supports POST requests. POST can be useful to overcome some length limits in the CloudStack API.

CLOUDSTACK_TIMEOUT or the timeout entry in the configuration file can be used to change the HTTP timeout when making CloudStack requests (in seconds). The default value is 10.

CLOUDSTACK_RETRY or the retry entry in the configuration file (integer) can be used to retry list and queryAsync requests on failure. The default value is 0, meaning no retry.

CLOUDSTACK_JOB_TIMEOUT or the job_timeout entry in the configuration file (float) can be used to set how long an async call is retried assuming fetch_result is set to true). The default value is None`, it waits forever.

CLOUDSTACK_POLL_INTERVAL or the poll_interval entry in the configuration file (number of seconds, float) can be used to set how frequently polling an async job result is done. The default value is 2.

CLOUDSTACK_EXPIRATION or the expiration entry in the configuration file (integer) can be used to set how long a signature is valid. By default, it picks 10 minutes but may be deactivated using any negative value, e.g. -1.

CLOUDSTACK_DANGEROUS_NO_TLS_VERIFY or the dangerous_no_tls_verify entry in the configuration file (boolean) can be used to deactivate the TLS verification made when using the HTTPS protocol.

Multiple credentials can be set in .cloudstack.ini. This allows selecting the credentials or endpoint to use with a command-line flag.

[cloudstack]
endpoint = https://some-host/api/v1
key = api key
secret = api secret

[exoscale]
endpoint = https://api.exoscale.ch/v1
key = api key
secret = api secret

Usage:

$ cs listVirtualMachines --region=exoscale

Optionally CLOUDSTACK_REGION can be used to overwrite the default region cloudstack.

For the power users that don't want to put any secrets on disk, CLOUDSTACK_OVERRIDES let you pick which key will be set from the environment even if present in the ini file.

Pagination

CloudStack paginates requests. cs is able to abstract away the pagination logic to allow fetching large result sets in one go. This is done with the fetch_list parameter:

$ cs listVirtualMachines fetch_list=true

Or in Python:

cs.listVirtualMachines(fetch_list=True)

Tracing HTTP requests

Once in a while, it could be useful to understand, see what HTTP calls are made under the hood. The trace flag (or CLOUDSTACK_TRACE) does just that:

$ cs --trace listVirtualMachines

$ cs -t listZones

Async client

cs provides the AIOCloudStack class for async/await calls in Python 3.5+.

import asyncio
from cs import AIOCloudStack, read_config

cs = AIOCloudStack(**read_config())

async def main():
   vms = await cs.listVirtualMachines(fetch_list=True)
   print(vms)

asyncio.run(main())

Async deployment of multiple VMs

import asyncio
from cs import AIOCloudStack, read_config

cs = AIOCloudStack(**read_config())

machine = {"zoneid": ..., "serviceofferingid": ..., "templateid": ...}

async def main():
   tasks = asyncio.gather(*(cs.deployVirtualMachine(name=f"vm-{i}",
                                                    **machine,
                                                    fetch_result=True)
                            for i in range(5)))

   results = await tasks

   # Destroy all of them, but skip waiting on the job results
   await asyncio.gather(*(cs.destroyVirtualMachine(id=result['virtualmachine']['id'])
                          for result in results))

asyncio.run(main())

Release Procedure

mktmpenv -p /usr/bin/python3
pip install -U twine wheel build
cd exoscale/cs
rm -rf build dist
python -m build
twine upload dist/*

cs's People

Contributors

atsaki avatar bcachet avatar brutasse avatar chrisglass avatar falzm avatar greut avatar igrishaev avatar marcaurele avatar phsm avatar pierre-emmanuelj avatar resmo avatar schu avatar ste-m avatar tgrondier avatar tooangel avatar ujjwalsh avatar vincentbernat avatar vojtechcima avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cs's Issues

A problem with dynamically generated Python scripts

Hi, is this a cs or ansible problem?

https://stackoverflow.com/questions/52740831/debugging-ansible-trace-for-possible-bug-in-python-cloud-stack-module

How to reproduce (before docker build, change cloudstack.ini.default to cloudstack.ini with your Cloud Stack credentials and your API URL):

git clone https://gitlab.cc-asp.fraunhofer.de/peter.muryshkin/ansible-beginner.git
cd ansible-beginner
docker build -f Dockerfile.ubuntu16.04LTS -t foo .

Relevant Docker build log output regarding the question whether the SSH key is there (Dockerfile: line 8. ls -alh ~/.ssh/). if it is not there, Ansbile throws an error that this file is not available.

drwxrwxr-x 2 root root 4.0K Oct 10 12:54 .
drwx------ 1 root root 4.0K Oct 11 10:52 ..
-rw-r--r-- 1 root root 1.8K Oct 10 12:54 id_rsa.pub

Then, to reproduce the actual error,

docker run -it foo bash -c 'ansible-playbook -C main.yml -vvv'

Error (locally, in other setup you need to check that the CloudStack API is available):

<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py", line 113, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py", line 105, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py", line 48, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py", line 267, in <module>
  File "/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py", line 258, in main
  File "/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py", line 133, in register_ssh_key
  File "/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py", line 211, in get_ssh_key
  File "/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py", line 227, in _get_ssh_fingerprint
  File "/usr/local/lib/python2.7/dist-packages/sshpubkeys/keys.py", line 157, in hash_md5
    fp_plain = hashlib.md5(self._decoded_key).hexdigest()
TypeError: md5() argument 1 must be string or buffer, not None

fatal: [localhost]: FAILED! => {
    "changed": false, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1539255360.27-44970784416337/AnsiballZ_cs_sshkeypair.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py\", line 267, in <module>\n  File \"/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py\", line 258, in main\n  File \"/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py\", line 133, in register_ssh_key\n  File \"/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py\", line 211, in get_ssh_key\n  File \"/tmp/ansible_cs_sshkeypair_payload_7Q2J_c/__main__.py\", line 227, in _get_ssh_fingerprint\n  File \"/usr/local/lib/python2.7/dist-packages/sshpubkeys/keys.py\", line 157, in hash_md5\n    fp_plain = hashlib.md5(self._decoded_key).hexdigest()\nTypeError: md5() argument 1 must be string or buffer, not None\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", 
    "rc": 1
}
	to retry, use: --limit @/work/ansible-beginner/main.retry

PLAY RECAP ***************************************************************************************************************************************************************************
localhost                  : ok=9    changed=0    unreachable=0    failed=1   

After this execution, if you would retain the test system (in the example above, the container gets thrown away), the temporary Python scripts seem to be cleaned up. How to retain them? For example, to check for the script contents AnsiballZ_cs_sshkeypair.py.

How does Ansible require key specification?

# register your existing local public key:
- cs_sshkeypair:
    name: [email protected]
    public_key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
  delegate_to: localhost

How is it specified in this setup?


- cs_sshkeypair:
    name: my_sshkey
    public_key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
  delegate_to: localhost


How to use createTags?

The API method createTags() requires a parameter "tags" as map of tags (key/value pairs) to be passed.

However for the following call I get an error as listed below.

tagsresult = self.cs.createTags(resourceids=mynetwork_id, resourcetype="Network", 
tags={"foo":"bar"})

530 - Value for the key null is either null or empty

Do I have to use some another notation to pass the tags?

L2 Networks do not appear to be supported

If I try to create a VM where one of the networks is an L2 network I get the error message:

Could not find all networks, networks list found: [u'<THE_OTHER_NETWORK>']"

Test environments for cs

How do you test this component also for cross-compatibility?

I've checked for community Docker images of Apache Cloud Stack but for example for 4.7 /4.9/4.10 there are no images available and the provided Dockerfile does not build.

See also:
apache/cloudstack#3083

Need a user-agent name defined, so it's more secure

We have some WAF Bot mitigation that is blocking and banning IPs that are based on the user-agent that is being passed
python-urllib3/1.26.5 Can you update the default agent to something else besides the none that is being used and hence using the default python-urllib3/1.26.5

Maybe there is a better place to put this, but in my testing i just modified this section of code and added a user agent here and was able to successfully use it without my WAF blocking it.

https://github.com/exoscale/cs/blob/master/cs/client.py#L214

Suggested default agent name: exoscale-cs

async behind an extra

aiohttp being a relatively big package (aiodns/cchardet and so on), what do you think of hiding the async side of the force behind an extra flag?

CS client does not finish if there is only one element and fetch_list is used

When using a list API call (for example listVPCs) and using for example the ID as parameter (id=), I'd expect CS to return that single entry. However, the script times out, CS never finishes. When stopping the script, the following traceback is shown:

Traceback (most recent call last):
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./cloud_access_tool", line 321, in <module>
    main()
  File "./cloud_access_tool", line 312, in main
    at.action_migrate(args.database)
  File "/Users/dherrman/PycharmProjects/cloud_access_tool/AccessTool.py", line 710, in action_migrate
    acl_list = self.cs.listNetworkACLLists(fetch_list=True, id=acl)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/cs/client.py", line 103, in handler
    return self._request(command, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/cs/client.py", line 140, in _request
    **{kwarg: kwargs})
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1002, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 865, in read
    return self._sslobj.read(len, buffer)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 625, in read
    v = self._sslobj.read(len, buffer)
KeyboardInterrupt

When omitting the fetch_list=True parameter, the expected result is shown:

{'count': 1, 'networkacllist': [{'id': 'eafcf9a3-8b01-4cda-b206-c3ebfed2543c', 'name': 'dhe-test-migrate', 'description': '', 'vpcid': 'bfdfa866-e533-4270-b284-063ff0e1321a'}]}

API response content type text/javascript from CS 4.5 not accepted

python-cs 2.5.3 no longer works with our Cloudstack installation due to the new response content type check introduced in 69077c5 .

Instead of application/json, Cloudstack 4.5 seems to return text/javascript. While this isn't completely right, there is no reason why text/javascript should be considered invalid in this context.

Please add text/javascript to the list of supported response content types.

Support POST for larger userdata in deployVirtual Machine

According to API documentation, userdata parameter can be allowed to be large with POST method.

In the existing clients, this has not been supported: CloudMonkey implements this only in the upcoming release 6.4.0.

Also, this Python client seems to user GET, also when userdata parameter is larger than 8K:

send: b'GET /client/api?projectid=0..1fa&userdata=zoK...wOQ&...
command=deployVirtualMachine&response=json&signatureVersion=3&
expires=2023-11-13T19%3A12%3A09%2B0000&signature=so..Q5n4%3D
HTTP/1.1\r\nHost: XXXXX\r\nAccept-Encoding: identity\r\nUser-Agent: python-urllib3/2.0.7\r\n\r\n'

Environment variables do not work

If you try to use only env vars to config it, you get:

Config file not found. Tried /Users/marina/.cloudstack.ini, /Users/marina/entuu
ra/cloudstack.ini

If you put an empty .cloudstack.ini there to shut it up, you get:

Error: region 'cloudstack' not in config

Tests fail on master

I get the following output while running tox:

===================================================================== test session starts =====================================================================
platform linux -- Python 3.8.12, pytest-7.3.2, pluggy-1.2.0
cachedir: .tox/py38/.pytest_cache
rootdir: /home/rominf/dev/cs
configfile: setup.cfg
plugins: cov-4.1.0
collected 15 items

tests.py .FFF...........                                                                                                                                [100%]

========================================================================== FAILURES ===========================================================================
_____________________________________________________________ ConfigTest.test_current_dir_config ______________________________________________________________

self = <tests.ConfigTest testMethod=test_current_dir_config>

    def test_current_dir_config(self):
        with open('/tmp/cloudstack.ini', 'w') as f:
            f.write('[cloudstack]\n'
                    'endpoint = https://api.example.com/from-file\n'
                    'key = test key from file\n'
                    'secret = test secret from file\n'
                    'dangerous_no_tls_verify = true\n'
                    'theme = monokai\n'
                    'other = please ignore me\n'
                    'header_x-custom-header1 = foo\n'
                    'header_x-custom-header2 = bar\n'
                    'timeout = 50')
            self.addCleanup(partial(os.remove, '/tmp/cloudstack.ini'))

        with cwd('/tmp'):
            conf = read_config()
>           self.assertEqual({
                'endpoint': 'https://api.example.com/from-file',
                'key': 'test key from file',
                'secret': 'test secret from file',
                'expiration': 600,
                'theme': 'monokai',
                'timeout': '50',
                'trace': None,
                'poll_interval': 2.0,
                'name': 'cloudstack',
                'poll_interval': 2.0,
                'verify': None,
                'dangerous_no_tls_verify': True,
                'retry': 0,
                'method': 'get',
                'cert': None,
                'headers': {
                    'x-custom-header1': 'foo',
                    'x-custom-header2': 'bar',
                },
            }, conf)
E           AssertionError: {'endpoint': 'https://api.example.com/from-[339 chars]ar'}} != {'timeout': '50', 'method': 'get', 'retry':[354 chars]ar'}}
E             {'cert': None,
E           +  'cert_key': None,
E           -  'dangerous_no_tls_verify': True,
E           ?                             ^^^^
E
E           +  'dangerous_no_tls_verify': 1,
E           ?                             ^
E
E              'endpoint': 'https://api.example.com/from-file',
E              'expiration': 600,
E              'headers': {'x-custom-header1': 'foo', 'x-custom-header2': 'bar'},
E              'key': 'test key from file',
E              'method': 'get',
E              'name': 'cloudstack',
E              'poll_interval': 2.0,
E              'retry': 0,
E              'secret': 'test secret from file',
E              'theme': 'monokai',
E              'timeout': '50',
E              'trace': None,
E              'verify': None}

tests.py:165: AssertionError
______________________________________________________ ConfigTest.test_env_var_combined_with_dir_config _______________________________________________________

self = <tests.ConfigTest testMethod=test_env_var_combined_with_dir_config>

    def test_env_var_combined_with_dir_config(self):
        with open('/tmp/cloudstack.ini', 'w') as f:
            f.write('[hanibal]\n'
                    'endpoint = https://api.example.com/from-file\n'
                    'key = test key from file\n'
                    'secret = secret from file\n'
                    'theme = monokai\n'
                    'other = please ignore me\n'
                    'timeout = 50')
            self.addCleanup(partial(os.remove, '/tmp/cloudstack.ini'))
        # Secret gets read from env var
        with env(CLOUDSTACK_ENDPOINT='https://api.example.com/from-env',
                 CLOUDSTACK_KEY='test key from env',
                 CLOUDSTACK_SECRET='test secret from env',
                 CLOUDSTACK_REGION='hanibal',
                 CLOUDSTACK_DANGEROUS_NO_TLS_VERIFY='1',
                 CLOUDSTACK_OVERRIDES='endpoint,secret'), cwd('/tmp'):
            conf = read_config()
>           self.assertEqual({
                'endpoint': 'https://api.example.com/from-env',
                'key': 'test key from file',
                'secret': 'test secret from env',
                'expiration': 600,
                'theme': 'monokai',
                'timeout': '50',
                'trace': None,
                'poll_interval': 2.0,
                'name': 'hanibal',
                'poll_interval': 2.0,
                'verify': None,
                'dangerous_no_tls_verify': True,
                'retry': 0,
                'method': 'get',
                'cert': None,
            }, conf)
E           AssertionError: {'endpoint': 'https://api.example.com/from-[267 chars]None} != {'timeout': '50', 'method': 'get', 'retry':[282 chars]kai'}
E             {'cert': None,
E           +  'cert_key': None,
E           -  'dangerous_no_tls_verify': True,
E           ?                             ^^^^
E
E           +  'dangerous_no_tls_verify': 1,
E           ?                             ^
E
E              'endpoint': 'https://api.example.com/from-env',
E              'expiration': 600,
E              'key': 'test key from file',
E              'method': 'get',
E              'name': 'hanibal',
E              'poll_interval': 2.0,
E              'retry': 0,
E              'secret': 'test secret from env',
E              'theme': 'monokai',
E              'timeout': '50',
E              'trace': None,
E              'verify': None}

tests.py:131: AssertionError
__________________________________________________________________ ConfigTest.test_env_vars ___________________________________________________________________

self = <tests.ConfigTest testMethod=test_env_vars>

    def test_env_vars(self):
        with env(CLOUDSTACK_KEY='test key from env',
                 CLOUDSTACK_SECRET='test secret from env',
                 CLOUDSTACK_ENDPOINT='https://api.example.com/from-env'):
            conf = read_config()
>           self.assertEqual({
                'key': 'test key from env',
                'secret': 'test secret from env',
                'endpoint': 'https://api.example.com/from-env',
                'expiration': 600,
                'method': 'get',
                'trace': None,
                'timeout': 10,
                'poll_interval': 2.0,
                'verify': None,
                'dangerous_no_tls_verify': False,
                'cert': None,
                'name': None,
                'retry': 0,
            }, conf)
E           AssertionError: {'key': 'test key from env', 'secret': 'tes[240 chars]': 0} != {'timeout': 10, 'method': 'get', 'retry': 0[258 chars]env'}
E             {'cert': None,
E           +  'cert_key': None,
E              'dangerous_no_tls_verify': False,
E              'endpoint': 'https://api.example.com/from-env',
E              'expiration': 600,
E              'key': 'test key from env',
E              'method': 'get',
E              'name': None,
E              'poll_interval': 2.0,
E              'retry': 0,
E              'secret': 'test secret from env',
E              'timeout': 10,
E              'trace': None,
E              'verify': None}

tests.py:72: AssertionError

This prevents me from packaging for Fedora.

security group with internal sg as source

Hey folks,

what's the syntax with cs if we want to create a sg rule where a security group is the source.
I know we can do it through your UI but never quite got it from the API.

thx

Add more verbosity to CloudStack exception error messages

Currently, errors do not include details. In an exception log you would read something of the sort:

cs.client.CloudStackApiException: HTTP 530 response from CloudStack

While if same error caused during a CloudMonkey call, you would find additionally the error code and message:

Error: (HTTP 530, error code 4250) Internal error executing command, please contact your system administrator

UPDATE. After a closer look, it seems that there might be a bug or not used feature in cs - a flag to use json as it seems - regarding JSON parsing of the Response.

Currently, response.json() could provide for example the following data if there is an error:

.{'listtemplatesresponse': {'cserrorcode': 9999,
                           'errorcode': 431,
                           'errortext': 'Unable to execute API command '
                                        'listtemplates due to missing '
                                        'parameter templatefilter',
                           'uuidList': []}}

In a test hack I've tried to change original code of [cs/client.py](https://github.com/exoscale/cs/blob/e8784588b3040750119f68eef5cd1ba3b602b4d2/cs/client.py#L352):

        if response.status_code != 200:
            raise CloudStackApiException(
                "HTTP {0} response from CloudStack".format(
                    response.status_code),
                error=data,
                response=response)

as the following:

        if response.status_code != 200:
                ddata = response.json()
                k,val = ddata.popitem()
                errmsg="HTTP %s response from CloudStack.\ncserrorcode %s: %s" % (response.status_code, val['cserrorcode'], val['errortext'])
             
             raise CloudStackApiException(
                 errmsg, 
                 error=data,
                 response=response)

This results in a more detailed output.

E           cs.client.CloudStackApiException: HTTP 431 response from CloudStack.
E           cserrorcode 9999: Unable to execute API command listtemplates due to missing parameter templatefilter

This hack does not follow the original logic though, because the method signature in this context containes the boolean function argument json, and the code in question is for the case "it's not json" as it seems.

For time being, ambigous error messages containing just HTTP error code are really a headache. Who knows what should be behind a 431 in terms which parameter exactly is missing, or which backend problem is a HTTP 530?

Encoding issue for not ASCII characters

Hi all,

for me it is not possible to add a comment to a server, when the comment include not pure standard ASCII letters. For example german "umlaute" (öäü). The following issue appear:

"'ascii' codec can't decode byte 0xc3 in position 9: ordinal not in range(128)"

The problem is near this command line

params[k] = text_type(v)

in this file: /usr/local/lib/python2.7/dist-packages/cs/client.py
I tried to fix it and set utf8 decoding before executing this code

v = v.decode("utf-8")

but this fit only for strings and not for all possible objects. So it solve the problem only in some cases not in all. For example I get this issue back:

"msg": "'bool' object has no attribute 'decode'"

This issue was recognized in version 2.5.8 and 2.7.1

The webfrontend of Cloudstack use the API and transform "Umlaute". (e.g.: ü = %C3%BC) in a UTF8 encoded form.

Sorry I don't have so much Python skills to fix this problem alone and send a PR. Could you please assist?

If you have any questions do not hesitate to ask.
Thank you in advanced!

Kind regards Muschranini

Boilerplate code for resource validation and handling asynchronous calls

I am not sure whether this project uses some generative/mapping technique to map to the CS API instead of implementing all calls?

If so, if possible, it would be really great to have some check functions, like:

...
    def zoneExists(self,zone):
        """Check if a specific zone is defined"""
        zones_list = self.cs.listZones()

        zones_list = zones_list['zone']
        zones=[]

        for z in zones_list:
            zones.append(z['name'])

        return zone in zones

Another example handling asynchronous calls:

class VirtualMachineManager():
    def __init__(self, cloudstack):
        self.cs = cloudstack

    def destroyVirtualMachine(self,**kwargs):
        result =  self.cs.destroyVirtualMachine(**kwargs)
        asj = AsyncJob(self.cs,result['jobid'])
        assert asj.isSuccessful() == True
        print("destroyed VM with ID: %s" % result['id'])
        return result['id']

    def deployVirtualMachine(self, **kwargs):
        result =  self.cs.deployVirtualMachine(**kwargs)
        asj = AsyncJob(self.cs,result['jobid'])
        assert asj.isSuccessful() == True
        print("created VM with ID: %s" % result['id'])
        return result['id']

    def expungeVirtualMachine(self, **kwargs):
        result =  self.cs.expungeVirtualMachine(**kwargs)
        asj = AsyncJob(self.cs,result['jobid'])
        assert asj.isSuccessful() == True

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.