Giter VIP home page Giter VIP logo

upcloud-python-api's Introduction

UpCloud's Python API Client

test PyPI version License

OOP-based API client for UpCloud's API. Includes most of the API functionality and some convenience functions that combine several API endpoints and logic.

Please test all of your use cases thoroughly before actual production use. Using a separate UpCloud account for testing / developing the client is recommended.

Installation

pip install upcloud-api

Alternatively, if you want the newest (possibly not yet released) stuff, clone the project and run:

python setup.py install

Supported Python versions in API v2.5.1

  • Python 3.7
  • Python 3.8
  • Python 3.9
  • PyPy3

Python 2 has been deprecated

  • Python 2.7 is no longer supported, but available in older API versions (< v2.0.0).

Changelog

Usage

More usage examples are available under [docs/]. If there's a specific thing you're interested in, but are not able to get working, please contact UpCloud support.

Defining and creating servers

import upcloud_api
from upcloud_api import Server, Storage, login_user_block

manager = upcloud_api.CloudManager('api_user', 'password')
manager.authenticate()


login_user = login_user_block(
    username='theuser',
    ssh_keys=['ssh-rsa AAAAB3NzaC1yc2EAA[...]ptshi44x [email protected]'],
    create_password=False
)

cluster = {
    'web1': Server(
        plan='2xCPU-4GB',
        hostname='web1.example.com',
        zone='uk-lon1', # All available zones with ids can be retrieved by using manager.get_zones()
        storage_devices=[
            # OS: template storage UUID, all available os templates can be retrieved by calling manager.get_templates()
            # Note: the storage os template uuid:s will change when OS is updated. So check that the UUID is correct
            # default tier: maxIOPS, the 100k IOPS storage backend
            Storage(os='01000000-0000-4000-8000-000030200200', size=10),
            # secondary storage, hdd for reduced speed & cost
            Storage(size=100, tier='hdd')
        ],
        login_user=login_user  # user and ssh-keys
    ),
    'web2': Server(
        plan='2xCPU-4GB',
        memory_amount=1024,
        hostname='web2.example.com',
        zone='uk-lon1',
        storage_devices=[
            Storage(os='01000000-0000-4000-8000-000030200200', size=10),
            Storage(size=100, tier='hdd'),
        ],
        login_user=login_user
    ),
    'db': Server(
        # use custom resources, instead of a plan
        core_number=12, # CPU cores
        memory_amount=49152, # RAM in MB
        hostname='db.example.com',
        zone='uk-lon1',
        storage_devices=[
            Storage(os='01000000-0000-4000-8000-000030200200', size=10),
            Storage(size=100),
        ],
        login_user=login_user
    ),
    'lb': Server(
        plan='2xCPU-4GB',
        hostname='balancer.example.com',
        zone='uk-lon1',
        storage_devices=[
            Storage(os='01000000-0000-4000-8000-000030200200', size=10)
        ],
        login_user=login_user
    )
}

for server in cluster:
    manager.create_server(cluster[server]) # creates all server objects defined in cluster

Servers can be defined as dicts without using Server or Storage classes. The syntax/attributes are exactly like above and under the hood they are converted to Server and Storage classes. This feature is mainly for easier usage of the module from Ansible, but may provide useful elsewhere.

Stop / Start / Destroy Servers

for server in cluster:
	server.shutdown()
	# OR:
	server.start()
	# OR:
	server.destroy()
	for storage in server.storage_devices:
	  storage.destroy()

As the success of server.start() or server.destroy() and storage.destroy() depend on the Server's state, new helpers have been added. The helpers may be called regardless of the server's current state.

# makes sure that the server is stopped (blocking wait) and then destroys the server and its storages
server.stop_and_destroy()

# makes sure that the server is started (blocking wait)
server.ensure_started()

CloudManager offers mode fine-grained deletion options for servers as you can choose to delete storages and choose what happens to their backups when deleting the server. By default the storage and their backups are always preserved.

Following example would delete all storages attached to a server, but would keep the latest backup of each storage if backups exist.

from upcloud_api.storage import BackupDeletionPolicy

manager.delete_server(uuid, delete_storages=True, backups=BackupDeletionPolicy.KEEP_LATEST)

Upgrade a Server

server = cluster['web1']
server.shutdown()
server.core_number = 4
server.memory_amount = 4096
server.save()
server.start()

Clone a new server from existing storage

Cloning is done by giving existing storage uuid to storage_devices. Note that size of the storage must be defined and must be at least the same size as the storage being cloned.

clone = Server(
    plan='2xCPU-4GB',
    hostname='cloned.server',
    zone='fi-hel1',
    storage_devices=[
        Storage(
            uuid='012bea57-0f70-4154-84d0-b3d25f4a018b',
            size=50  # size must be defined and it has to be at least same size than storage being cloned
        ),
    ]
)

manager.create_server(clone)

Easy access to servers and their information

# returns a public IPv4 (preferred) IPv6 (no public IPv4 was attached) address
server.get_public_ip()

# returns a JSON serializable dict with the server's information (storages and ip-addresses included)
server.to_dict()

Get resources

servers     = manager.get_servers()
server1     = manager.get_server(uuid) # e.g servers[0].uuid
storages    = manager.get_storages()
storage1    = manager.get_storage(uuid) # e.g server1.storage_devices[0].uuid
ip_addrs    = manager.get_ips()
ip_addr     = manager.get_ip(address) # e.g server1.ip_addresses[0].address

Testing

Set up environment and install dependencies:

# run at project root, python3 and virtualenv must be installed
virtualenv venv
source venv/bin/activate

Install the package in editable mode.

# run at project root
pip install -e .

Tests are located under test/. Run with:

py.test test/

To test against all supported python versions, run:

tox

The project also supplies a small test suite to test against the live API at test/live_test.py. This suite is NOT run with py.test as it will permanently remove all resources related to an account. It should only be run with a throwaway dev-only account when preparing for a new release. It is not shipped with PyPI releases. See source code on how to run the live test.

Bugs, Issues, Problems, Ideas

Please report issues and features requests through the issues page.

upcloud-python-api's People

Contributors

ajmyyra avatar akx avatar alienhaxor avatar cancerballs avatar darep avatar dependabot[bot] avatar eligit avatar elnygren avatar faustasm avatar iler avatar kangasta avatar mlackman avatar onnimonni avatar poposensei avatar scop avatar szepeviktor avatar t-tran avatar tophattom avatar tvarsis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

upcloud-python-api's Issues

Getting self.uuid error when cloning a server

Traceback (most recent call last):
File "buildServer.py", line 40, in
print(clone)
File "/Users/skip/Library/Python/2.7/lib/python/site-packages/upcloud_api/server.py", line 110, in str
return self.uuid

===== CODE =======
id = uuid.uuid4().hex[:10].lower()
cloneUUID = str(sys.argv[1])

manager = upcloud_api.CloudManager('username', 'password')
manager.authenticate()

if cloneZONE == 'Singapore' :
print('Singapore')
storage = manager.get_storage(cloneUUID)
print(storage.zone)
print(storage.uuid)
print(storage.size)

clone = Server(core_number=1, memory_account=1024, hostname='CloneServer.' + id, zone=storage.zone, storage_devices=[Storage(uuid=storage.uuid,size=storage.size),])

Creating server from custom template results in "upcloud_api.errors.UpCloudAPIError: UNKNOWN_ATTRIBUTE The request body contains an unknown attribute zone."

The call to storage.to_dict() in server.py line 332 results in a dictionary that includes a "zone" attribute. This dictionary is then used as a storage description for the server to be created.

This "zone" attribute inside the storage description seems to be the attribute that causes the error: removing it from the request causes the request to succeed. Note: I specified the server's zone separately.

The code that should reproduce the error:

import upcloud_api
import random

_manager = upcloud_api.CloudManager(login, password)
_manager.authenticate()
storage = _manager.get_storage(template_uuid)
srv = upcloud_api.Server(
    plan = "1xCPU-2GB",
    hostname = "testpackage{:06}".format(random.randrange(1000000)),
    zone = storage.zone,
    storage_devices = [storage],
)
_server = _manager.create_server(srv)

Announcement: major quality & consistency overhaul (v0.4)

I'm doing a quality & consistency overhaul with some breaking changes at: #29

WIP and preview of (breaking) changes to come at 0.4 that can be considered a somewhat major release.

Main goals:

  • professional quality (the points below elaborate on this)
  • improve consistency in code quality and naming conventions
  • PEP8ish style
  • UpCloudResource base class and interface for resource classes (Storage, Server, IPAddress...)
  • fix bad code, aim for simplicity. Especially Storage and Server are a bit of a mess.
  • improve tests and documentation
  • proper and consistent docstrings

Note: all breaking changes will be perfectly documented. They will be syntax things most of the time (e.g like IP_address -> IPAddress).

Creating Ubuntu 18.04 server errors out

manager = upcloud_api.CloudManager('api_user', 'password')
manager.authenticate()

login_user = login_user_block(username=api_user, ssh_keys=['ssh_key'], create_password=True)
server = Server(core_number=1, memory_amount=1024, zone=ZONE.London, hostname='hostname',
	        storage_devices=[Storage(os='Ubuntu 18.04', size=25)], login_user=login_user)

The above code errors out, the traceback being

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sumit/Coding/removeads.org/server/utils.py", line 29, in create_server
    manager.create_server(server)
  File "/home/sumit/.local/lib/python3.6/site-packages/upcloud_api/cloud_manager/server_mixin.py", line 114, in create_server
    body = server.prepare_post_body()
  File "/home/sumit/.local/lib/python3.6/site-packages/upcloud_api/server.py", line 347, in prepare_post_body
    storage_body['storage'] = OperatingSystems.get_OS_UUID(storage.os)
  File "/home/sumit/.local/lib/python3.6/site-packages/upcloud_api/constants.py", line 55, in get_OS_UUID
    "Invalid OS -- valid options are: 'CentOS 6.5', 'CentOS 7.0', "
Exception: Invalid OS -- valid options are: 'CentOS 6.5', 'CentOS 7.0', 'Debian 7.8', 'Debian 8.0' ,'Ubuntu 12.04', 'Ubuntu 14.04', 'Ubuntu 16.04', 'Windows 2008', 'Windows 2012'

Requirements are pinned too hard

The requirements are pinned to the exact version as it is, which makes it hard to use pip-tools or another dependency manager.

Instead of

future==0.14.3
mock==1.0.1
requests==2.6.0
responses==0.3.0
six==1.9.0
wheel==0.24.0

I suggest something like

future>=0.14.3
mock>=1.0.1
requests>=2.6.0
responses>=0.3.0
six
wheel

maybe.

simple_backup not working

In server.py 'simple_backup' needs to be added to updateable_fields and optional_fields.

When it's not mentioned in the fields one cannot set simple backups using Python API. At least not easily.

"online" tests with cloud credentials

The offline tests are lacking in numerous ways and can only reveal bad programming mistakes.

Automatic tests against the actual API would be a huge plus.

Deploy server using API

Hi

I have been having a play around with the API in python and been testing deploying a server but struggling. I'm using requests:

payload = {"server":
{"zone": "us-chi1",
"title": "Server",
"hostname":"example.upcloud.com",
"plan":"1xCPU-1GB",
"storage_devices":
{"storage_device":
[{"action": "clone",
"storage":"01000000-0000-4000-8000-000050010200",
"title": "example.upcloud.com-disk0",
"size": 25,
"tier": "maxiops"
}]
}
}
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)

It doesn't like the payload parameter, what have I done wrong?

Get a list of available plans - suggestion

Hello,

Is it possible to add the following to upcloud_api/cloud_manager/__init__.py please, so I can grab a list of available plans?
I did try a cheeky push, but it didn't work :)

    def get_plans(self):
        """
        Returns a list of available plans.
        """
        return self.api.get_request('/plan')

Or is there another way I could get a list of available plans?

Thanks,

David

CloudManager.create_server() ignores `Storage.address` attribute.

I expect that the code below would create a server with its only storage attached as a virtio device. Instead, the created server has the storage attached as an IDE device.

        _manager = upcloud_api.CloudManager(login, password)
        _manager.authenticate()
        storage = _manager.get_storage(uuid_of_the_template)
        storage.address = "virtio:0"
        srv = upcloud_api.Server(
            plan = "1xCPU-2GB",
            hostname = "testpackage{:06}".format(random.randrange(1000000)),
            zone = storage.zone,
            storage_devices = [storage],
        )
        _manager.create_server(srv)

Api returns 400 Bad Request when specifying the charset

Working:

POST https://api.upcloud.com/1.2/server/<server-id>/stop
Authorization: Basic {{username}} {{password}}
Content-Type: application/json

{"stop_server":{"stop_type":"soft","timeout":"60"}}

Not working:

Request

POST https://api.upcloud.com/1.2/server/<server-id>/stop
Authorization: Basic {{username}} {{password}}
Content-Type: application/json; charset=utf-8

{"stop_server":{"stop_type":"soft","timeout":"60"}}

Response

HTTP/1.1 400 Bad Request
Date: Fri, 02 Aug 2019 10:12:28 GMT
Server: Apache
Content-Length: 141
Connection: close
Content-Type: application/json; charset=UTF-8

{ "error": { "error_code": "CONTENT_TYPE_INVALID", "error_message": "The Content-Type header has an invalid value." }}

In my opinion, setting the charset shoud not result in an error here.

How to clone servers?

Hello,

I need to clone an already existing server, but I can't find anything related to cloning servers on the documentation. Can anyone guide me through it?

Test issue

Issue to test some things, can be disregarded.

Inaccurate error message when sending invalid `plan` value

This isn't exactly a bug in the Python API, but the REST API in general, but I suppose it'll end up in the right place...

Trying to create a server with the .plan attribute set to None (i.e. serialized to null as JSON) fails with

upcloud_api.errors.UpCloudAPIError: UNKNOWN_ATTRIBUTE The request body contains an unknown attribute server.

(A better error would point at a faulty plan attribute.)

When the plan attribute does not exist at all, things work as expected.

Unnecessary dependency on py.test

The API has a hard dependency on py.test -- consumers of the library will thus download py.test (and py, as a transitive dependency) whenever they pip install it.

Unless the library does actually need py.test to regularly function (which I doubt ๐Ÿ˜ธ) the dependency should moved away from those parsed by setup.py.

Add hard shutdown and hard restart APIs

Currently only soft shutdown and restart APIs are available.

I think this would be most fluently expressed by rephrasing

def shutdown(self):
def restart(self):

as

def shutdown(self, hard=False, timeout=30):
def restart(self, hard=False, timeout=30):

How to start many servers simultaneously?

Hi, currently we have an operation where we wish to start many servers (e.g. 30+ servers) at once. The stop() method seems to respond once the server is being stopped but the start() method seems to block until the server is fully started which means starting them one by one would take a while. We have code that waits for all of them to reach certain state but it is not as useful for the start() method as they would have most likely already been started.

Is there a way to have start() behave like the stop() method or something that achieves similar result?

Remove default zone for all resources

User should always be the one who gets to decide where their resources are going. Remove the default (fi-hel1) atleast from servers, and check if other resources (storages, networks, object storages atleast) contain anything similar.

AUTHENTICATION_FAILED Authentication failed using the given username and password

This might sound as a very stupid question, but I can't figure out a solution. Running the following code, generates the error message written in the title:

import upcloud_api
from upcloud_api import Server, Storage, ZONE, login_user_block
manager = upcloud_api.CloudManager("myusername", "mypsw")
manager.authenticate()

The username and password are correct, I've checked multiple times. Thank you in advance.

add comment support for firewall rules

It seems that Upcloud's API now has a new "comment" attribute for firewall rules:

โžœ  ~ curl https://api.upcloud.com/1.2/server/uuid/firewall_rule --user user:pass
{
   "firewall_rules" : {
      "firewall_rule" : [
         {
            "action" : "accept",
            "comment" : "",
            "destination_address_end" : "",
            "destination_address_start" : "",
            "destination_port_end" : "53",
            "destination_port_start" : "53",
            "direction" : "in",
            "family" : "IPv4",
            "icmp_type" : "",
            "position" : "1",
            "protocol" : "udp",
            "source_address_end" : "",
            "source_address_start" : "",
            "source_port_end" : "",
            "source_port_start" : ""
         },
         {
            "action" : "accept",
            "comment" : "",
            "destination_address_end" : "",
            "destination_address_start" : "",
            "destination_port_end" : "22",
            "destination_port_start" : "22",
            "direction" : "in",
            "family" : "IPv4",
            "icmp_type" : "",
            "position" : "2",
            "protocol" : "tcp",
            "source_address_end" : "",
            "source_address_start" : "",
            "source_port_end" : "",
            "source_port_start" : ""
         },
         {
            "action" : "reject",
            "comment" : "",
            "destination_address_end" : "",
            "destination_address_start" : "",
            "destination_port_end" : "",
            "destination_port_start" : "",
            "direction" : "in",
            "family" : "",
            "icmp_type" : "",
            "position" : "3",
            "protocol" : "",
            "source_address_end" : "",
            "source_address_start" : "",
            "source_port_end" : "",
            "source_port_start" : ""
         }
      ]
   }
}

However, the API client doesn't seem to have support for the new attribute yet. At the moment, the lack of support breaks the upcloud_firewall.py Ansible module you guys have made, since it wants to check the existing firewall rules via the API first.

A quick fix I found was to add a new "comment": None attribute to the attributes hash in upcloud_api/firewall.py. Obviously that doesn't qualify as support, but at least you can use the Ansible modules without errors.

Please make add a release tag for upcloud-api-0.4.5

Hi Wil,

I would like to use your integration in my Gentoo Overlay for Home Assistant. Therefor I need a release file in tar.gz format.
Home Assistant currently requires: upcloud-api-0.4.5

This one exists at Pypi, but only in wheel format, cannot use this.

Generally, a tar.gz release on Pypi (sdist) would be preferred, because this could take use of Gentoo's mirror system.

Most of the other integrations do both.

Thanks
\B.

Align tox rules with CI tests

Tox runs the tests, but leaves isort, black & flake8 out of it. Add these so it does the same testing for style and tests as the CI does for every PR.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.