divio / divio-cli Goto Github PK

View Code? Open in Web Editor NEW

21.0 13.0 18.0 10.98 MB

Command line interface to the Divio Cloud (formerly aldryn-client)

Home Page: http://control.divio.com

License: Other

Python 99.72% Makefile 0.28%

divio-cli's Introduction

Divio CLI - Command-line interface to the Divio Cloud

Installing

pip install divio-cli

⚠️ For Windows users: Make sure you have added Python to PATH during installation, otherwise you must add it manually for the divio-cli commands to function properly.

Using the CLI

See Divio developer handbook: How to use the Divio command-line interface

Support for docker-compose 1 and 2

The CLI supports version 1 (docker-compose) and version 2 (docker compose) of the command invocation and is using the new version 2 by default.

You can opt-in to still use the old version 1 command by updating your CLI global settings in ~/.aldryn and add a new setting called docker-compose with the value ["docker-compose"]:

{"update_check_timestamp": 1234567890, "docker-compose": ["docker-compose"]}

Testing

The test suite is split into two categories: unit tests and integration tests. Unit tests run completely locally, so no previous setup is required. Integration tests run against the actual Divio infrastructure, so setup of a Divio project is required. In order to run the test suite, you have to have GNU Make and GNU Bash on your system. The integration tests also require Docker Compose.

Both test categories take optional arguments like TOX_ARGS and PYTEST_ARGS:

make test TOX_ARGS="-e python.11" PYTEST_ARGS="-s"

To clear all local state run:

make clean

Unit tests

These do not require external communication and can be run with the following command:

make test

Integration tests

These do require a more involved setup and will trigger actions on a real project. It is recommended to use a project on control.dev.aldryn.net because it requires no control panel running locally, and is what the CI pipeline does.

The CI uses ci-test-project-do-not-delete project. DO NOT USE THIS PROJECT FOR YOUR LOCAL TESTING! The project is reserved for CI testing. Create your own project and replicate the CI project configuration.

To run the integration test suite, run:

make test_integration

The first run will fail, but will create an empty .env file. Configure your test project there. You can take a look at the variables tab in the CI Settings as a starting point.

Linting

To run the linter, run the following command:

make lint

Creating a release

Checkout a new branch for the new version - release-X.X.X
Update the changelog.
Merge the branch (after approval).
Tag master with the release number X.X.X and git push origin <tagname>

The pipeline will then take care of the release.

divio-cli's People

Contributors

Stargazers

Watchers

Forkers

ojii jrief suriyan stefanfoulis kinkerl wilfoa abdulniyaspm1 glasnt chescales bykof stuaxo mebdivio ajhalili2006 arsinam80m trellixvulnteam jamison413 paragonsean gdrosos

divio-cli's Issues

help text on forms

Changing network (or network DNS) breaks connection to docker.io

This morning I was able to run aldryn project workon for a project at home; when moved to my office later in the day I would consistently get an error

Network timed out while trying to connect to https://index.docker.io/v1/repositories/library/postgres/images. You may want to check your internet connection or if you are behind a proxy.

After finally deleting and recreating the Virtual Box VM, it worked again. Then on returning home, it failed with the same error, until I once more deleted and recreated the VM.

@GaretJax speculated this may be due to different network/DNS addresses at the two locations.

If this is the case, then the Docker configuration needs to be robust enough to handle this.

Cannot upload latest aldryn-boilerplate

I'm getting the following error when running aldryn boilerplate update on the latest aldryn-boilerplate:

➜  aldryn-boilerplate git:(develop) aldryn boilerplate upload
400 CSS compilation error:
LoadError on line ["36"] of /usr/lib/ruby/1.9.1/rubygems/custom_require.rb: cannot load such file --bootstrap-sass
Run with --trace to see the full backtrace

When using bootstrap-sass the config.rb needs an additional line: https://github.com/aldryn/aldryn-boilerplate/blob/master/private/config.rb#L29

maybe this is not yet configured on the uploader?

provide yaml examples for apps and boilerplates

name: 'Divio Boilerplate'
url: 'https://github.com/divio'
version: '0.0.1'
description: 'This is the standardsite Divio boilerplate.'
author:
    name: 'Angelo Dini'
    url: 'http://www.divio.ch'
license:
    name: 'BSD'
    text: 'literal-include LICENSE.txt'
templates:
- ['fullwidth.html', 'full width']
- ['sidebar_left.html', 'sidebar left']
- ['sidebar_right.html', 'sidebar right']

maybe you want also to hint to the yaml syntax as you are talking about "tuples" within the documentation.

[suggestion] implement argcomplete

Hello!

A minor suggestion - would be nice to have tab-completion in bash (and possibly other shells).

(something like this)

remove usage of unicode()

found here

divio-cli/divio_cli/api_requests.py

Line 92 in 675b07c

messages.NETWORK_ERROR_MESSAGE + unicode(e)

allow initial data and help text on config forms

Divio addon upload fails on python 3 system

Traceback (most recent call last):
File "/usr/local/bin/divio", line 11, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/divio_cli/cli.py", line 546, in addon_upload
ret = upload_addon(ctx.obj.client, ctx.parent.params['path'])
File "/usr/local/lib/python3.6/site-packages/divio_cli/upload/addon.py", line 64, in upload_addon
archive_obj = create_addon_archive(path)
File "/usr/local/lib/python3.6/site-packages/divio_cli/upload/addon.py", line 53, in create_addon_archive
add_addon_meta_files(tar, path)
File "/usr/local/lib/python3.6/site-packages/divio_cli/upload/addon.py", line 36, in add_addon_meta_files
tar_add_bytesio(tar, BytesIO(fobj.read()), 'aldryn_config.py')
TypeError: a bytes-like object is required, not 'str'

https://github.com/divio/divio-cli/blob/master/divio_cli/upload/addon.py#L35

Possibly other things fail as well

"aldryn" is used in user-facing message

This should be renamed?

divio-cli/divio_cli/messages.py

Line 15 in 4dc59fc

LOGIN_SUCCESSFUL = u'Welcome to Aldryn. You are now logged in as {greeting}'

aldryn project setup does not pull base.css

I am filing this here with aldryn-client since I encounter this problem while using aldryn-client but I have the feeling this problem is due to a bug at the Aldryn Cloud backend.

I am using aldryn-client on Linux since I don't really have a Windows machine at hand to run the Aldryn Desktop app.

After having run aldryn project setup myprojectname command, the project is successfully downloaded to my local Linux machine, apart of one file, the base.css file.

Due to this the website does not render locally. To fix this I have to run sass base.scss ../../static/css/base.css in order to generate the base.css file.

Note, I have only observed this problem while using the Aldryn Explorer Theme.

sync doesn't work with OSX 10.8 + Coda 2

On OSX (10.8) and using Coda 2, sync doesn't work.

It seems to "stop" after one event (per file).

Might be solved with a proper FSEvent implementation (Maybe MacFSEvents?).

Needs further investigation.

Other editors (Textwrangler) seem to work using kqueue.

Linux works fine with inotify (Pycharm, vim, Gedit)

add-on validation JSON parsing issues

The JSON parser that checks the JSON file when validating an add-on, does not handle // comments in a JSON file.

This happens when you try to validate an add-on with a comment in the JSON.

$ ~ aldryn addon validate
control.aldryn.com
ValueError('No JSON object could be decoded',)

And this is what happens after you remove the comment.

$ ~ aldryn addon validate
control.aldryn.com
Configuration file is valid

Show better warning if .aldryn file is missing

Warnings for missing .aldryn file should be more understandable to the user:

Traceback (most recent call last):
  File "/Users/joseph/.local/bin/divio", line 11, in <module>
    sys.exit(cli())
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/utils.py", line 189, in read_remote_id
    remote_id = get_aldryn_project_settings(silent=True)['id']
  File "/Users/joseph/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/utils.py", line 17, in get_aldryn_project_settings
    with open(os.path.join(project_home, settings.ALDRYN_DOT_FILE)) as fh:
  File "/Users/joseph/.local/venvs/divio-cli/bin/../lib/python2.7/posixpath.py", line 70, in join
    elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'```

installation / usability feedback for new divio user on ubuntu 16.04 / xenial

I've landed here from:
http://support.divio.com/local-development/setup/set-up-the-local-development-environment-linux

Some of my feedback relates to https://www.divio.com/en/downloads/divio-linux/ and some of it to the divio-cli. As there is no obvious place to deposit feedback about the gui, I'll leave all of it here and anticipate you might redirect some of it :)

lsb_release -crs

yields

16.04
xenial

Downloading the appimage worked with out issue
You may wish to reorder the documentation to mention the (possible? seems most likely) need for

chmod 700

Before mentioning running gui

In my case I got stuck at the screen requesting me to specify a local folder - it was impossible to click continue. For this reason I proceeded to attempt to install & use the cli...

pip install divio-cli

worked without issue. You may wish to reorder the documentation to mention

divio doctor

before suggesting other installation steps, since divio doctor may indicate only a small number of steps are needed.

I experienced the following with divio doctor:

At least on my platform, it incorrectly attempts to invoke docker: docker-machine is not a binary or symlink that is present after following the linked docker documentation. I was able to work around this issue with:

ln -s /usr/bin/docker /usr/bin/docker-machine

It might also be helpful to feedback to the user the appropriate / missing download (noting it doesn't exist in my case, but.. :) )

It could be helpful to attempt to detect membership of the docker group. You might also consider changing the language of "Following the installation, we strongly recommend you take Docker's advised post-installation steps." since divio-cli doesn't appear to work unless the user is a member of the docker group, or invoked using sudo

It could be helpful if you suggest to the user how to install docker-compose, or remind them the correct fm to rt. Indeed: This user should have rtfm :)

After

pip install divio-cli

I was surprised to find that
divio-cli was not executable. Perhaps there are strong reasons to invoke this as divio and not divio-cli

Having managed to make divio doctor report all checks successful / green, I found I was able to progress past the original blocking page in the gui...

I next became blocked during the "local server setup" step. In the gui I saw "Running command. This may take a while." In the terminal I launched the gui from I found a prompt:

The authenticity of host 'git.divio.com (5.148.186.169)' can't be established.
RSA key fingerprint is SHA256:oKtZtDEiR6KsH5Ghh2DlrnbtS5xe9gfL/qP80NQLc0A....

probably because many minutes elapsed between the prompt and my accepting the key finger print, I next encountered:

Warning: Permanently added 'git.divio.com,5.148.186.169' (RSA) to the list of known hosts.
Connection to git.divio.com closed by remote host.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
--------------------------------------------------------------------------------
There was an error trying to run a command. This is most likely
not an issue with divio-cli, but the called program itself.
Try checking the output of the command above.
The command was:
  git clone [email protected]:{redacted} /home/{redacted}

and an additional terminal window was launched:

There was a problem while checking your system. Please run 'divio doctor'.
user@user-Parrot:~/local-git-clones/reimagi-divio-app/here$

returning back to the gui and clicking the action "Setup" again returned me to "Running command. This may take a while"

Indeed it does. In my case the progress was visible to me because I invoked the gui from a terminal window. Had I invoked it by double clicking, I suppose I would not have had this visibility.

The next issue I faced was:

Removing intermediate container 7364eb7850c3
Successfully built fd86e57b66bd
creating new database container
 ===> Pulling database from reimagitest2 test server
 ---> Preparing downloadError: Network error. Please check your connection and try again.('Connection aborted.', error(104, 'Connection reset by peer'))

And no error message in the gui. http://support.divio.com/local-development/setup/set-up-the-local-development-environment-linux explains the linux gui client is in beta so I simply click "setup" again, and ignore the "proceed with caution" message, clicking "Setup Project"

At this point, I appear to be ready to proceed with http://support.divio.com/local-development/divio-app/using-the-divio-app

When using `remote-id` on commands that give WARNINGS, make note of it

If I'm making commands on projects that deal with live changes (e.g. commands that bring up warnings that include "This will replace ALL existing media"), I want the source project listed in this warning.

This would be particularly useful when working with remote-id commands that will be overriding settings of other projects.

This might be an edge case for power users, but more explicit warnings including scope would be helpful for users.

Permission Denied (publickey)

I've installed divio-cli, but I get this error:

Creating workspace
cloning project repository
Cloning into '/Users/joshsullivan/Documents/Divio/grace-reformed-church'...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
There was an error trying to run a command. This is most likely
not an issue with divio-cli, but the called program itself.
Try checking the output of the command above.
The command was:
git clone [email protected]:grace-reformed-church.git /Users/joshsullivan/Documents/Divio/grace-reformed-church

I made sure the SSH keys were setup correctly. Another other steps I can take?

Validation doesn't check the cmscloud_config file

The upload command requires cmscloud_config.py file but validation process doesn't crash without it.

zip() creates zip object in python 3 and list object in python 2...leads to TypeError

For some reason whenever I build aldryn-client it builds under the python3.4 environment. As such the code has been running under the 3.4 environment and I managed to find a small bug in utils.py

In utils.py the function "pretty_size(num)" utilizes a global object "unit_list" that is created as shown below:

unit_list = zip(
['bytes', 'kB', 'MB', 'GB', 'TB', 'PB'],
[0, 0, 1, 2, 2, 2],
)

The problem is that in python2 unit_list automatically becomes a list object and in python3 it remains a zip object. So when "len(unit_list)" is called within "def pretty_size(num)" on line 172 of utils.py:

172: exponent = min(int(log(num, 1024)), len(unit_list) - 1)

we get a TypeError under the python3 environment, zip objects have no len(). I solved this in my own copy by simply creating the list object with list() when unit_list is created globally, as shown below:

unit_list = list(zip(
['bytes', 'kB', 'MB', 'GB', 'TB', 'PB'],
[0, 0, 1, 2, 2, 2],
))

I'm not sure if this is the best way to solve this or even globally appropriate but it seems to have worked for me for now.

Separately can someone email me as to why when I run ./scripts/build_unix it buils with the python3.4 environment? Is this just how it is? or can I change something to force python2.7? My email address is [email protected] and an answer would be greatly appreciated.

Permission denied (publickey)

I'm getting the following error "Permission denied (publickey)" when I trying to clone a new project. I've generated new RSA and ECDSA keys, I uploaded this to my control panel, tried on different computers and I'm still getting the same error.. It only happens when trying to clone in aldryn, when I test the same key with github or bitbucket it works well:

ssh -vT [email protected]:

OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /home/urkh/.ssh/config
debug1: /home/urkh/.ssh/config line 1: Applying options for git.aldryn.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to git.aldryn.com [5.148.186.169] port 22.
debug1: Connection established.
debug1: identity file /home/urkh/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/urkh/.ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2-hpn14v4
debug1: match: OpenSSH_7.2p2-hpn14v4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to git.aldryn.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:oKtZtDEiR6KsH5Ghh2DlrnbtS5xe9gfL/qP80NQLc0A
debug1: Host 'git.aldryn.com' is known and matches the RSA host key.
debug1: Found key in /home/urkh/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/urkh/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).

ssh -vT [email protected]

OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /home/urkh/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to github.com [192.30.253.112] port 22.
debug1: Connection established.
debug1: identity file /home/urkh/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/urkh/.ssh/id_rsa-cert type -1
debug1: identity file /home/urkh/.ssh/id_dsa type 2
debug1: key_load_public: No such file or directory
debug1: identity file /home/urkh/.ssh/id_dsa-cert type -1
debug1: identity file /home/urkh/.ssh/id_ecdsa type 3
debug1: key_load_public: No such file or directory
debug1: identity file /home/urkh/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/urkh/.ssh/id_ed25519 type 4
debug1: key_load_public: No such file or directory
debug1: identity file /home/urkh/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: Remote protocol version 2.0, remote software version libssh-0.7.0
debug1: no match: libssh-0.7.0
debug1: Authenticating to github.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in /home/urkh/.ssh/known_hosts:4
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: Skipping ssh-dss key /home/urkh/.ssh/id_dsa - not in PubkeyAcceptedKeyTypes
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/urkh/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: Authentication succeeded (publickey).
Authenticated to github.com ([192.30.253.112]:22).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: pledge: network
debug1: Sending environment.
debug1: Sending env LC_PAPER = es_VE.UTF-8
debug1: Sending env LC_ADDRESS = es_VE.UTF-8
debug1: Sending env LC_MONETARY = es_VE.UTF-8
debug1: Sending env LC_NUMERIC = es_VE.UTF-8
debug1: Sending env LC_TELEPHONE = es_VE.UTF-8
debug1: Sending env LC_IDENTIFICATION = es_VE.UTF-8
debug1: Sending env LANG = en_US.UTF-8
debug1: Sending env LC_MEASUREMENT = es_VE.UTF-8
debug1: Sending env LC_TIME = es_VE.UTF-8
debug1: Sending env LC_NAME = es_VE.UTF-8
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
Hi urkh! You've successfully authenticated, but GitHub does not provide shell access.
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 3112, received 1768 bytes, in 0.4 seconds
Bytes per second: sent 7216.4, received 4099.8
debug1: Exit status 1

$ aldryn doctor

Verifying your system setup
 ✓  Login
 ✓  Git
 ✓  Docker Client
 ✓  Docker Machine
 ✓  Docker Compose
 ✓  Docker Engine Connectivity
 ✓  Docker Engine Internet Connectivity
 ✓  Docker Engine DNS Connectivity

Can't pull db on Windows

(i can provide the project)

Pulling db results in this error on windows:

tar: can't open '/app//data.tar.gz': No such file or directory
pg_restore: [archiver] input file is too short ( read 0, expected 5)

Works as expected on mac
(both systems latest Docker (17.12 something))

divio project setup failure

the project was created and never deployed

1068 static files copied to '/app/static_collected'.
 ---> e45d43a9a49f
Removing intermediate container a533cb946cb9
Successfully built e45d43a9a49f
creating new database container
 ===> Pulling database from test-divio-app-setup test server
 ---> Preparing downloadTraceback (most recent call last):
  File "/Users/divio/.local/bin/divio", line 9, in <module>
    load_entry_point('divio-cli==3.2.0', 'console_scripts', 'divio')()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/cli.py", line 291, in project_setup
    localdev.create_workspace(obj, slug, stage, path, overwrite)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/main.py", line 142, in create_workspace
    setup_website_containers(client, stage, path)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/main.py", line 97, in setup_website_containers
    ImportRemoteDatabase(client=client, stage=stage, path=path)()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/main.py", line 191, in __call__
    return self.run()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/main.py", line 197, in run
    self.setup()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/main.py", line 360, in setup
    progress = self.client.download_db_progress(url=progress_url)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/cloud.py", line 253, in download_db_progress
    return request()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/api_requests.py", line 51, in __call__
    return self.request(*args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/api_requests.py", line 78, in request
    return self.verify(response)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/api_requests.py", line 92, in verify
    return self.process(response)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/api_requests.py", line 110, in process
    return response.json()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/requests/models.py", line 866, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

divio-cli ignores docker-compose.override.yml

The default behaviour of docker-compose is to override anything in the default docker-compose.yml file with the values from docker-compose.override.yml.
https://docs.docker.com/compose/extends/#/understanding-multiple-compose-files

divio cli seems to ignore the override file.

In my case I override the db container to expose the postgres port on localhost in docker-compose.override.yml. Every time I run a command from divio-cli that uses the db container, it shuts it down and restarts it without the exposed port.

example docker-compose.yml:

version: "2"

services:
  web:
    build: .
    ports:
      - "8000:80"
    volumes:
      - ".:/app:rw"
    links:
      - "db:postgres"
    command: python manage.py runserver 0.0.0.0:80
    env_file: .env-local

  db:
    image: postgres:9.4
    volumes:
      - ".:/app:rw"
    environment:
      POSTGRES_DB: "db"

example docker-compose.override.yml:

version: "2"
services:
  db:
    ports:
      - "5432:5432"

Incorrect exit code on deploy failure

divio project deploy exits with a success exit code when the deployment fails

divio project up failure

± % divio project up                                                                                                                                                                                                          !10205 14:23:08
Traceback (most recent call last):
  File "/Users/divio/.local/bin/divio", line 9, in <module>
    load_entry_point('divio-cli==3.2.0', 'console_scripts', 'divio')()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/cli.py", line 209, in project_up
    localdev.start_project()
  File "/Users/divio/.local/venvs/divio-cli/lib/python2.7/site-packages/divio_cli/localdev/main.py", line 777, in start_project
    output = exc.output.decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 12692: ordinal not in range(128)

Unable to pull media (Windows)

Hello!

I am having issues when pulling media files to my Windows machine:

 ===> Pulling media files from ft-enterprise test server
 ---> Preparing download [141s]
 ---> Downloading [2328s]
 ---> Extracting files to C:\Users\Petr\workspace\ft-enterprise\data\mediaTraceback (most recent call last):
  File "<string>", line 16, in <module>
  File "site-packages\click\core.py", line 716, in __call__
  File "site-packages\click\core.py", line 696, in main
  File "site-packages\click\core.py", line 1060, in invoke
  File "site-packages\click\core.py", line 1060, in invoke
  File "site-packages\click\core.py", line 1060, in invoke
  File "site-packages\click\core.py", line 889, in invoke
  File "site-packages\click\core.py", line 534, in invoke
  File "site-packages\click\decorators.py", line 27, in new_func
  File "divio_cli\cli.py", line 314, in pull_media
  File "divio_cli\localdev\main.py", line 428, in pull_media
  File "tarfile.py", line 2078, in extractall
  File "tarfile.py", line 2115, in extract
  File "tarfile.py", line 2191, in _extract_member
  File "tarfile.py", line 2231, in makefile
IOError: [Errno 2] No such file or directory: 'C:\\Users\\Petr\\workspace\\ft-enterprise\\data\\media\\filer_public_thumbnails\\filer_public\\cc\\ac\\ccaca383-7556-4ebf-a5e1-d51cc512a8d4\\20160301_underground-lending-indicator_ftcr-china-underground-lending-index.png__32x32_q85_crop_subject_location-500,290_subsampling-2_upscale.png'
entrypoint returned -1

Running Divio 3.0.1 on latest version of Windows 10

Greeting fails after login

Epic

File "/usr/local/lib/python2.7/site-packages/aldryn_client/cloud.py", line 83, in login
greeting = '{} {} ({})'.format(first_name, last_name, email)
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 2: ordinal not in range(128)

don't know if this is a shell issue, though ;-) I got zsh / Oh my zsh.

Add support for uploading database dump without using a docker container

Currently, in order to upload a database to the project the process it to use:

divio project push db

This actually spins up a docker container for the project, gets the db dump, compresses it, uploads and if all goes well clear the dump files from the working directory..

It would help if the cli support uploading db dump directly without having to setup the project.

With the support for command line flags to specify website id it is possible to do this.

allows a bootstrap that breaks deployment

creating an empty bootstrap with a blank but existing base.html is considered 'valid' yet this will not deploy when you try to deploy a site built using the bootstrap.

remove 'newest_version' from cli docs

It only makes sense for the GUI version, which has little to do with the commandline client (other than sharing a codebase).

~~remove newest_version from cli-docs~~ not removing it, because it has to be there for docopt to pick up the arguments
merge the 0.9 GUI branch
release 0.9.1 on pypi

using git and not mentioning it = bad

the aldryn docs need to mention much more clearly that the sync is built on git.
i had massive sync issues due to conflicting git ignores inside a subfolder where i was using git to ensure that the upstream library in the subfolder was up to date.

SSL Error (ca.pem not found)

Today, I experienced the following issue:

$ cmscloud login
E-Mail: [email protected]
Password: 
Traceback (most recent call last):
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/bin/cmscloud", line 9, in <module>
    load_entry_point('cmscloud-client==0.4.2', 'console_scripts', 'cmscloud')()
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/cmscloud_client-0.4.2-py2.7.egg/cmscloud_client/cli.py", line 54, in main
    retval, msg = client.login()
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/cmscloud_client-0.4.2-py2.7.egg/cmscloud_client/client.py", line 145, in login
    response = self.session.post('/api/v1/login/', data={'email': email, 'password': password})
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/requests-2.0.1-py2.7.egg/requests/sessions.py", line 403, in post
    return self.request('POST', url, data=data, **kwargs)
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/cmscloud_client-0.4.2-py2.7.egg/cmscloud_client/client.py", line 77, in request
    return super(SingleHostSession, self).request(method, url, *args, **kwargs)
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/requests-2.0.1-py2.7.egg/requests/sessions.py", line 361, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/requests-2.0.1-py2.7.egg/requests/sessions.py", line 464, in send
    r = adapter.send(request, **kwargs)
  File "/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/requests-2.0.1-py2.7.egg/requests/adapters.py", line 363, in send
    raise SSLError(e)
requests.exceptions.SSLError: [Errno 185090050] _ssl.c:340: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib

After further investigation, I found out that requests/urlib2 are having problems with certificates after being packaged. See here: https://github.com/kennethreitz/requests/issues/557

The thread suggests to specify the certificate, which we are already doing in the client.py. But apparently, it cannot be found.

To fix, I did the following:

$ pip install certifi
$ python
Python 2.7.5 (default, Aug  6 2013, 11:44:53) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import certifi
>>> print certifi.where()
/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/certifi/cacert.pem
>>> 
KeyboardInterrupt

and I modified the client.py:29:

# changed
CACERT_PEM_PATH = resource_path('cacert.pem')
# to 
CACERT_PEM_PATH = '/Users/kim/workspace/aldryn/cmscloud-client/env/lib/python2.7/site-packages/certifi/cacert.pem'

python setup.py install again, and it worked.

I don't really have time at the moment for further investigation, maybe this could be fix though: https://github.com/kennethreitz/requests/issues/557#issuecomment-21309979

aldryn-client need root on Ubuntu?

Hi
I am new to aldryn and docker so maybe I am missing something. I am working on Ubuntu 15.10 and I encounter this in the first steps of the tutorial: http://docs.divio.com/en/latest/tutorial/commandline/installation.html#launch-docker-quickstart-terminal

The command aldryn doctor tries to execute docker ps (https://github.com/aldryn/aldryn-client/blob/master/aldryn_client/check_system.py#L15). This command needs to be run as root to work on Ubuntu (as far as I know).
The only way for me to pass the test is to use sudo or use a root shell.

This would not be a big deal as most of the other commands work fine but aldryn project setup also requires these checks to be successful.

So the initial project setup can only be done as root and I will have to continue to work as root.

This is not ideal. Am I missing something?
Thanks

"divio project develop <addon-name>" should add to INSTALLED_ADDONS in settings.py

divio project develop <addon-name> should also add to INSTALLED_ADDONS in settings.py. This is important for the case where the Addon was previously not installed in the project.

see https://github.com/divio/divio-cli/blob/3.2.0/divio_cli/localdev/main.py#L664-L722

respect field order on forms

Fails to udpate local project when working on a local branch

Somewhat common usecase is to work on feature branches, that may or may not be present on origin remote, but might be present on different remote.

Currently 'origin' is hardcoded https://github.com/divio/divio-cli/blob/master/divio_cli/localdev/main.py#L618

It would be great to have it pull from the set upstream remote or in case it's not on any remote yet ask for a confirmation to approve continuing instead of failing

config form should respect field order

the fields should be rendered in the same order as they are defined on the form.

share site command

add domain to site command

Error during divio project update (Windows)

I have opened up shell using latest Divio app and ran project setup successfully. However, I cannot seem to be able to run project update:

PS C:\Users\Petr\workspace\cs1-1-website> divio project update
Pulling changes from git remote
C:\Program Files (x86)\Divio\resources\app\bundle\git/libexec/git-core\git-pull: line 11: git-sh-setup: No such file or directory
C:\Program Files (x86)\Divio\resources\app\bundle\git/libexec/git-core\git-pull: line 12: git-sh-i18n: No such file or directory
C:\Program Files (x86)\Divio\resources\app\bundle\git/libexec/git-core\git-pull: line 13: set_reflog_action: command not found
C:\Program Files (x86)\Divio\resources\app\bundle\git/libexec/git-core\git-pull: line 14: require_work_tree_exists: command not found
C:\Program Files (x86)\Divio\resources\app\bundle\git/libexec/git-core\git-pull: line 15: cd_to_toplevel: command not found
Locking the website...
From git.divio.com:cs1-1-website
 * branch            develop    -> FETCH_HEAD
Unlocking the website...
sed: can't read /FETCH_HEAD: No such file or directory
There are no candidates for merging among the refs that you just fetched.
Generally this means that you provided a wildcard refspec which had no
matches on the remote end.
----------------------------------------------------------------------------------------------------------------------
There was an error trying to run a command. This is most likely
not an issue with divio-cli, but the called program itself.
Try checking the output of the command above.
The command was:
  git pull origin develop

After app upload crash I have to change app version

divio project up/open failure on windows

divio project up seems to connect to 0.0.0.0 which is seems to be not (always) available on windows. Needs further investigation.

validate cmscloud_config.py if available

aldryn/mirumee-feedback#5

Check that if a cmscloud_config.py is available, it contains an importable form.

Unable to run setup

When I run the command divio project setup example I get the following output:

Joshs-MacBook-Pro:divio joshsullivan$ divio project setup example
Traceback (most recent call last):
  File "<string>", line 16, in <module>
  File "site-packages/click/core.py", line 716, in __call__
  File "site-packages/click/core.py", line 696, in main
  File "site-packages/click/core.py", line 1057, in invoke
  File "site-packages/click/core.py", line 889, in invoke
  File "site-packages/click/core.py", line 534, in invoke
  File "site-packages/click/decorators.py", line 17, in new_func
  File "divio_cli/cli.py", line 52, in cli
  File "divio_cli/cloud.py", line 41, in __init__
  File "divio_cli/cloud.py", line 311, in __init__
  File "netrc.py", line 35, in __init__
  File "netrc.py", line 117, in _parse
netrc.NetrcParseError: bad follower token 'api.heroku.com' (/Users/joshsullivan/.netrc, line 4)
entrypoint returned -1

Any ideas on how to fix this?

divio-cli/divio_cli/localdev/main.py

Line 165 in 85aee1b

database_extensions = ['hstore', 'postgis']
the default should be hstore, postgis (so if nothing is present in .aldryn), but those values can be overriden via the .aldryn file

Divio CLI gives an error when "divio project up"

Steps to reproduce:
Setup the project
Open "Divio shell"
Go to project
Hit "divio project up"

Actual result:
At the end - it is an error shown.
But eventually the project is "upped" successfully

Expect result:
All good

divio-cli uses hard coded service names like `db` and `web`

This should be flexible and not fail if one of the services is not available.