Giter VIP home page Giter VIP logo

rancher-gitlab-deploy's Introduction

Rancher GitLab Deployment Tool

rancher-gitlab-deploy is a tool for deploying containers built with GitLab CI onto your Rancher infrastructure.

It fits neatly into the gitlab-ci.yml workflow and requires minimal configuration. It will upgrade existing services as part of your CI workflow.

Both GitLab's built in Docker registry and external Docker registries are supported.

rancher-gitlab-deploy will pick as much of its configuration up as possible from environment variables set by the GitLab CI runner.

This tool is not suitable if your services are not already created in Rancher. It will upgrade existing services, but will not create new ones. If you need to create services you should use rancher-compose in your CI workflow, but that means storing any secret environment variables in your Git repo.

Installation

I recommend you use the pre-built container:

https://hub.docker.com/r/cdrx/rancher-gitlab-deploy/

But you can install the command locally, with pip, if you prefer:

pip install rancher-gitlab-deploy

Usage

You will need to create a set of API keys in Rancher and save them as secret variables in GitLab for your project.

Three secret variables are required:

RANCHER_URL (eg https://rancher.example.com)

RANCHER_ACCESS_KEY

RANCHER_SECRET_KEY

Rancher supports two kind of API keys: environment and account. You can use either with this tool, but if your account key has access to more than one environment you'll need to specify the name of the environment with the --environment flag. This is so that the tool can upgrade find the service in the right place. For example, in your gitlab-ci.yml:

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - upgrade --environment production

rancher-gitlab-deploy will use the GitLab group and project name as the stack and service name by default. For example, the project:

http://gitlab.example.com/acme/webservice

will upgrade the service called webservice in the stack called acme.

If the names of your services don't match your repos in GitLab 1:1, you can change the service that gets upgraded with the --stack and --service flags:

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - upgrade --stack acmeinc --service website

You can change the image (or :tag) used to deploy the upgraded containers with the --new-image option:

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - upgrade --new-image registry.example.com/acme/widget:1.2

You may use this with the $CI_BUILD_TAG environment variable that GitLab sets.

rancher-gitlab-deploy's default upgrade strategy is to upgrade containers one at time, waiting 2s between each one. It will start new containers after shutting down existing ones, to avoid issues with multiple containers trying to bind to the same port on a host. It will wait for the upgrade to complete in Rancher, then mark it as finished. The upgrade strategy can be adjusted with the flags in --help (see below).

GitLab CI Example

Complete gitlab-ci.yml:

image: docker:latest
services:
  - docker:dind

stages:
  - build
  - deploy

build:
  stage: build
  script:
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.example.com
    - docker build -t registry.example.com/group/project .
    - docker push registry.example.com/group/project

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - upgrade

A more complex example:

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - upgrade --environment production --stack acme --service web --new-image alpine:3.4 --no-finish-upgrade

Help

$ rancher-gitlab-deploy --help

Usage: rancher-gitlab-deploy [OPTIONS]

  Performs an in service upgrade of the service specified on the command
  line

Options:
  --rancher-url TEXT              The URL for your Rancher server, eg:
                                  http://rancher:8000  [required]
  --rancher-key TEXT              The environment or account API key
                                  [required]
  --rancher-secret TEXT           The secret for the access API key
                                  [required]
  --rancher-label-separator TEXT  Where the default separator (',') could
                                  cause issues
  --environment TEXT              The name of the environment to add the host
                                  into (only needed if you are using an
                                  account API key instead of an environment
                                  API key)
  --stack TEXT                    The name of the stack in Rancher (defaults
                                  to the name of the group in GitLab)
                                  [required]
  --service TEXT                  The name of the service in Rancher to
                                  upgrade (defaults to the name of the service
                                  in GitLab)  [required]
  --start-before-stopping / --no-start-before-stopping
                                  Should Rancher start new containers before
                                  stopping the old ones?
  --batch-size INTEGER            Number of containers to upgrade at once
  --batch-interval INTEGER        Number of seconds to wait between upgrade
                                  batches
  --upgrade-timeout INTEGER       How long to wait, in seconds, for the
                                  upgrade to finish before exiting. To skip
                                  the wait, pass the --no-wait-for-upgrade-to-
                                  finish option.
  --wait-for-upgrade-to-finish / --no-wait-for-upgrade-to-finish
                                  Wait for Rancher to finish the upgrade
                                  before this tool exits
  --rollback-on-error / --no-rollback-on-error
                                  Rollback the upgrade if an error occured.
                                  The rollback will be performed only if the
                                  option --wait-for-upgrade-to-finish is
                                  passed
  --new-image TEXT                If specified, replace the image (and :tag)
                                  with this one during the upgrade
  --finish-upgrade / --no-finish-upgrade
                                  Mark the upgrade as finished after it
                                  completes
  --sidekicks / --no-sidekicks    Upgrade service sidekicks at the same time
  --new-sidekick-image <TEXT TEXT>...
                                  If specified, replace the sidekick image
                                  (and :tag) with this one during the upgrade
  --create / --no-create          If specified, create Rancher stack and
                                  service if they don't exist
  --labels TEXT                   If specified, add a comma separated list of
                                  key=values to add to the service
  --label <TEXT TEXT>...          If specified, add a Rancher label to the
                                  service
  --variables TEXT                If specified, add a comma separated list of
                                  key=values to add to the service
  --variable <TEXT TEXT>...       If specified, add a environment variable to
                                  the service
  --service-links TEXT            If specified, add a comma separated list of
                                  key=values to add to the service
  --service-link <TEXT TEXT>...   If specified, add a service link to the
                                  service
  --host-id TEXT                  If specified, service will be deployed on
                                  requested host
  --debug / --no-debug            Enable HTTP Debugging
  --ssl-verify / --no-ssl-verify  Disable certificate checks. Use this to
                                  allow connecting to a HTTPS Rancher server
                                  using an self-signed certificate
  --secrets TEXT                  If specified, add a comma separated list of
                                  secrets to the service
  --secret TEXT                   If specified add a secret to the service
  --help                          Show this message and exit.

History

[1.7] - 2020-12-16

Fixed a bug when updating variables on existing service, thanks to @ffouchier Added the --rancher-label-separator option, thanks to @NigelGreenway for the PR Added the --service-links option, thanks to @mrpolman for the PR Added the support for secrets with the --secret option, thanks to @earzur for the PR

[1.6] - 2018-09-09

Added the --rollback-on-error option, thanks to @TZK- for the PR Added the --label, --variables, --variable options, thankls to @tsteenkamp for the PR

[1.5] - 2017-11-25

Fixed UnicodeError bug with authentication, thank you to @evilmind for the fix

[1.4] - 2017-07-18

Fixed some bug to do with error and sidekick handling and made --no-start-before-stopping the default behaviour

[1.3] - 2017-03-16

Added the --new-sidekick-image flag to change sidekick images while upgrading, thank you @kariae for the PR

[1.2] - 2017-01-03

Added the --sidekicks flag to upgrade sidekicks at the same time, thank you @kiesiu for the PR

[1.1] - 2016-09-29

Fixed a bug that caused a crash when using --environment, thank you @mvriel for the PR

[1.0] - 2016-09-14

First release, works.

License

MIT

rancher-gitlab-deploy's People

Contributors

cdrx avatar evilmind avatar ffouchier avatar jerryverhoef avatar kariae avatar kiesiu avatar lu1as avatar mrpolman avatar mvriel avatar rtorrero avatar tevaum avatar tsteenkamp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rancher-gitlab-deploy's Issues

Working with Rancher 2.5?

Hello, I have an infrastructure where I use rancher 2.5 will this plugin work with that version? I'm having a hard time integrating it. Could you help me?

Self Signed Certificate

Hi.

We have an internal rancher deployment under https with a self-signed htttps certificate.

When we use your image we recieve the error "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed"

There is any way to avoid this whitout using a valid certificate or exposing the rancher url under http?

Blue-Green deployement strategy

Blue/Green deployement strategy

This is feature request.

With blue-Green deployement strategy I have 2 stacks (say stack_a and stack_b) plus a reverse-proxy container that swaps beetween stack_a and stack_b.

The green stack is the stable version (initially stack_a).
While the blue stack is the development version (initially stack_a).

The stack where to deploy is determined at runtime by reading the reverse-proxy current status with this command: docker exec -t proxy_container /status.sh.

After the deploy we can manually swap the blue/green in our reverse-proxy.

After the swap the green stack is stack_b, and the blue stack is stack_a.

I'm awaiting your considerations, opinions, questions or suggestions.
Thanks for everyone.

[Rancher v1.2.1] Error: Unable to request an upgrade on Rancher

I'm having some trouble with rancher:v1.2.1

$ upgrade --rancher-url $RANCHER_URL --rancher-key $RANCHER_ACCESS_KEY --rancher-secret $RANCHER_SECRET_KEY --environment Default --stack DataWarehouse --service code --new-image $CONTAINER_RELEASE_IMAGE_STABLE
Upgrading datawarehouse/code in environment Default...
Error: Unable to request an upgrade on Rancher
ERROR: Build failed: exit code 1

I've followed API to the url /v1/projects/1a5/services/1s1049 but in actions I see only remove action.
The ACCESS_KEY and SECRET_KEY has been created by administrator as Account API Keys, I am missing something?

No healthy hosts meet the resource constraints: ports

Dear,

I have on Rancher a stack and service with the same names as the gitlab group and project.
So my gitlab-ci looks like:
deploy: image: cdrx/rancher-gitlab-deploy:latest stage: deploy only: - develop script: - upgrade

But when this stage of the pipeline runs i get the following error on rancher:
middleware (Failed to allocate instance [container:1i451]: Bad instance [container:1i451] in state [error]: Allocation failed: No healthy hosts meet the resource constraints: [ports: [8082:8082/tcp], ResourceType: instanceReservation, Amount: 1])

Is this because of the way i created the service (with rancher UI 'Add service')?
Can someone help? This is quite important in our setup.

kind regards,
Thomas

Better key=value splitting

In case if the value is, for example, an url with query params then split('=') tries to split by every "=" and it is not what I want.

Restart service instead of upgrade

Is it possible to only restart a service? Perhaps as a new feature. Typically, my use case would be restarting a nginx proxy in front of a newly deployed backend.

Add --new-sidekick-image

In Rancher you can specify multiple sidekicks. You add the "--sidekicks" option, which only makes pull a image from old path.
Which is needed that we can specify new image url for each sidekicks.

Feature request: zero-downtime upgrade

Hi, thanks for the tool - it comes really handy and works great out of the box with minimal setup required, exactly how it should.
But unfortunately, it doesn't provide zero-downtime upgrade feature (--start-before-stopped is a bit different). For me it seems it can be easily done with this tool, following this strategy for example:

  • stack must have service alias "SERVICENAME-alias" for the service to be upgraded
  • when upgrade is requested, add new service like "SERVICENAME-upgraded"
  • add the "SERVICENAME-upgraded" to "SERVICENAME-alias" links
  • perform some kind of check against "SERVICENAME-upgraded" to ensure it have started (this seems tricky since different services require different checks)
  • remove original SERVICENAME from SERVICENAME-alias
  • rename SERVICENAME-upgraded to just SERVICENAME

Thank you.

My Rancher environment is completely migrated to Rancher 2 at this point, and I'm now using an image based on bitnami:kubectl in my GitLab CI jobs to deploy updated applications. I may never have gotten to this point without the work you did to simplify deployments to Rancher 1.x environments, though, so I just wanted to send a heartfelt thank you for that, and best wishes for whatever projects you're working on now.

Upgrade multiple containers in parallel

Would be super nice if we could update multiple containers in parallel. I know there are tools that let you do that, but none of these tools are installed on the image, afaik

Update By Tag

Not sure if this is possible or not, but it would be nice if this was able to update by tag. Instead of saying

upgrade --stack my-stack
upgrade --stack my-other-stack
upgrade --stack yet-another-stack

I could instead just say

upgrade --tag stack-group

no-ssl-verify not work complated

my rancher user private ssl ca,and add --no-ssl-verify,happened error also,key url token is right;
$ upgrade --service $MODULE --new-image $HARBOR_URL/$HARBOR_STORAGE_NAME/$MODULE:$DOCKER_IMAGE_TAG --no-ssl-verify
Error: Unable to connect to Rancher at demo.rancher.com - is the URL and API key right?
/usr/local/lib/python3.8/site-packages/urllib3-1.26.2-py3.8.egg/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'demo.rancher.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
warnings.warn(

Force an Upgrade

I could use a --force-upgrade option which would allow me to upgrade even a unhealthy service. In fact, some images pushed by my CI might not be working all the time. I seen that inactive services can also be upgraded. Maybe the option could stop the service and then upgrade it ?
It would be great! Currently I have to get the service healthy by hand in the rancher UI, which is annoying..

Error in error handling

$ upgrade --environment Default --stack services --service bonus --no-finish-upgrade
Traceback (most recent call last):
  File "/usr/local/bin/upgrade", line 11, in <module>
    load_entry_point('rancher-gitlab-deploy==1.2', 'console_scripts', 'rancher-gitlab-deploy')()
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/rancher_gitlab_deploy-1.2-py2.7.egg/rancher_gitlab_deploy/cli.py", line 93, in main
    bail("Unable to find a stack called '%s'. Does it exist in the '%s' environment?" % environment_name)
TypeError: not enough arguments for format string
ERROR: Job failed: exit code 1

[error] Crashes, and i can't figure out why

Hi,
I'm currently having a lot of fun with docker, gitlab-ci and rancher :)
and was really happy to see your tool rancher-gitlab-deploy to deploy my build from gitlab to rancher
but unfortunately, i can't make it work.

I've try from gitlab-ci in a deploy job and on my local machine inside your docker.
I end up with the same pb. I'm not fluent in python so i don't understand much ^^

here is my stacktrace:

upgrade --rancher-url http://MY_URL --rancher-key 4242424242 --rancher-secret blablablabala --environment production --stack MY_STACK --service apache --new-image MY_IMAGE --no-finish-upgrade

Traceback (most recent call last):
  File "/usr/local/bin/upgrade", line 11, in <module>
    load_entry_point('rancher-gitlab-deploy==1.2', 'console_scripts', 'rancher-gitlab-deploy')()
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/rancher_gitlab_deploy-1.2-py2.7.egg/rancher_gitlab_deploy/cli.py", line 57, in main
    environments = r.json()['data']
  File "/usr/local/lib/python2.7/site-packages/requests-2.13.0-py2.7.egg/requests/models.py", line 866, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
ERROR: Job failed: exit code 1

Wait for migrator service

Hi,

I have pipeline with three steps:

  1. run maintenance mode
  2. migrate db - Started once service
  3. run production mode back

I am facing to problem with waiting for migrator in step 2.

Following command:

upgrade --environment Staging --stack $STACK_NAME --service migrator --new-image myImage:$APP_VERSION --finish-upgrade

run migrations which usually running 1-2 minutes but pipeline dont wait for migrations done and start step 3 (run production mode).

It is possible wait for migrator and then continue to next step?

No 'annotations' cause failed deployment

In some random cases deployment fails to this error

File "/usr/local/lib/python2.7/site-packages/rancher_gitlab_deploy-1.5-py2.7.egg/rancher_gitlab_deploy/cli.py", line 156, in main
    upgrade['annotations']['gitlab.com/updateTime'] = datetime.datetime.today().strftime("%Y%m%d%H%M%S");
KeyError: 'annotations'

fix for this is really simple to make, just add before line 156

if not upgrade.get('annotations', None):
        upgrade['annotations'] = {}

Question : After upgrade, container created on another host

Hello,

Thanks for your amazing script, got a little question.
After auto deploy, new container is created on another host (multi-nodes rancher config) so my backend URL is changing at each deploy.

Is there a way to upgrade on the same node than the previous container ?

Thanks!

Can you add "upgrade --replace-Command" ?

I would like to be able to dynamically change the container's commands.

Could be a "restart --container-new-command yarn build && Yarn tests"
but for upgrade would be helpful.

upgrade --replace-Command 'yarn install && yarn build && yarn tests'
then
upgrade --replace-Command 'yarn start'

I still have not found anything that could do that. I'm doing a complex roundtrip by creating a container with Rancher CLI inside the Rancher and updating it via Gitlab CI/CD and then doing Rancher-compose up myfile.yml.

In case my yml files are triggered according to the context of the command I want to keep in the container. What I'm doing It works, but it's hard to maintain and teach how to use them.

Cheers

Can not update containers. Line 64: Upgrade: command not found

Hi, your job is awesome, unfortunately I can not make it work. When the deploy stage comes it throws this error "Line 64: Upgrade: Command not found. Exit status 1". I provided all secrets to GitLab secret variables: the environment API keys, URL (http://IP_address:8080). My gitlab-ci.yml is below

image: docker:latest
services:
  - docker:dind
  
stages:
  - build
  - deploy

variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:webapp_v1.$CI_BUILD_ID

build:
  stage: build
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - upgrade --new-image $CI_REGISTRY_IMAGE:webapp_v1.$CI_BUILD_ID --stack RoR-web-application --service rails

Thanks,
Cheers

Version-tagged images on Docker Hub

First off, thanks so much for this project! Rancher and GitLab work great together, but it's nice not to have to maintain the docker-compose/rancher-compose files inside the project in order to upgrade, especially since the Rancher Compose format doesn't always track with the UI features as they get introduced.

I'm wondering if you'd be open to adjusting your build process to tagging versioned images, in addition to latest? Just so we can pin to a specific version for predictable runs?

Thanks!

Upgrade sidekick

Hello.
Thnx for your utility.
But I have a question - how to upgrade only a sidekick? I'm trying to set sidekick name to --new-image, but got the error like this: "Error: Unable to find a service called 'code', does it exist in Rancher?".

Example of stack:

php-fpm:
  labels:
    io.rancher.sidekicks: code
    io.rancher.container.pull_image: always
  image: gitlab.local:4567/integrations/docker:php-fpm
  volumes_from:
  - code
code:
  labels:
    io.rancher.container.pull_image: always
    io.rancher.container.start_once: 'true'
  image: gitlab.local:4567/integrations/code:dev

Execute compose instead of run

Is it possible to have rancher use compose to be able to deploy the dependencies (databases, reverse proxies, etc) with this tool?

Command to execute commands into containers

This is a feature request,

I don't know if this is out of the scope of the project, but most of the projects deployed needs to run some commands after the deployment, for example, Symfony needs to update database schema ...

The actual Rancher API handle this using containerExec ressource.

I can work on that !

Error when specifying nonexistent environment

I've run into small issue when I accidentally specified wrong environment name which triggers this error:

upgrade --environment env --service betag-wb-frontend
Traceback (most recent call last):
  File "/usr/local/bin/upgrade", line 11, in <module>
    load_entry_point('rancher-gitlab-deploy==1.2', 'console_scripts', 'rancher-gitlab-deploy')()
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/rancher_gitlab_deploy-1.2-py2.7.egg/rancher_gitlab_deploy/cli.py", line 68, in main
    if not environment_id:
UnboundLocalError: local variable 'environment_id' referenced before assignment

Would be great to get same feedback as for nonexistent service:

Error: Unable to find a service called 'betag-wb-frontend', does it exist in Rancher?

Unable deploy to rancher

Hi

I have this error on my gitlab deploy pipeline

this is my deploy script

image: docker:latest
 services:
  - docker:dind
.......


deploy_devel:
  stage: deploy
  tags:
      - docker
  environment:
    name: develop
  image: cdrx/rancher-gitlab-deploy
  script:
    - upgrade --rancher-url $URL --rancher-key $MASTER_ACCESS_KEY --rancher-secret $MASTER_SECRET_KEY --stack testing --service web

Any idea why this happens?

Thanks

What about rollback if upgrade fail ?

Hello,

First, this project is very good. First time i try, all work well even in production. i appreciate it :)

I use image rancher-gitlab-deploy image with gitlab-ci to upgrade my project in rancher.

I'im wondering if we can add some optionnal parameter to rollback if upgrade fail, somethings like --rollback.

In rancher api, it's available. See

If you need some others informations, i'm here.

Using Rancher Host Labels ??

I tried the tool and it worked perfect, great job.

I just need to make this work with Rancher host labels, I couldn't find a way to achieve this.
Every time I do an upgrade the tool/Rancher chooses a random host do pull the image, and we would like to manage the deployment with rancher labels, so we can deploy on the desired hosts only by using:
io.rancher.scheduler.affinity:host_label: app=true

Is there any way I can achieve this with this image ???

Add teardown ability

@cdrx What are you thoughts to creating an end point to remove containers from Rancher?

My use case is I use this currently to add and push changes to review branches but would be up for writing further code to allow for the following:

# With current upgrade options but reword the command
deploy {...options}

#  Will destroy the container with current options
teardown {...options} 

I was thinking of doing this all in GoLang as a way to learn but happy to use this project if possible?

Would require the package to be bumped to v2.0.0 though, but I could put some changes in to warn of deprecation's if that possible?

Add --env-variable to options

Is there a way to pass environment variable to upgraded service? Upgraded services have same variables, but I have to i.e. change RELEASE_VERSION after service upgrade.

Error: Got unexpected extra argument

I have no idea why this error occurred, I tried to run it on my device, it doesn't have any error, but when I run it at Gitlab, it doesn't work.

CI error log

$ upgrade --environment $RANCHER_ENVIRONMENT --stack $RANCHER_STACK --service $RANCHER_SERVICE --new-image $IMAGE_TAG --finish-upgrade
Usage: upgrade [OPTIONS]

Error: Got unexpected extra argument (registry.gitlab.com/<GROUP>/<REPO>:<TAG>)
ERROR: Job failed: exit code 2

gitlab-ci.yml

variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

deploy_staging_api:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  only:
    - test/staging
  script:
    - upgrade
      --environment $RANCHER_ENVIRONMENT
      --stack $RANCHER_STACK
      --service $RANCHER_SERVICE
      --new-image $IMAGE_TAG
      --finish-upgrade

Labels causing failed deployments

Due to label.split(','), my deployment is failing due to the following example:

"io.rancher.container.pull_image=always,traefik.docker.network=my-network,traefik.frontend.rule="Host:example.co.uk,www.example.co.uk",traefik.enable=true,traefik.backend=my-backend-name,traefik.port=8080"

Due to traefik.frontend.rule="Host:example.co.uk,www.example.co.uk" containing a ,, it's causing .split to split that also.

Support for Rancher 2

The new Rancher 2 is using a new api for services. Could that be used with this repo?

You beauty!

Hi CDRX,
I've just tried this and it worked first time. Thank you so much - great for dev environments, and I'm sure (I just need to get more familiar with gitlab to work out how to segregate the flows for dev and prod)

Thanks again,
Andy

Rancher 2.0

With Rancher 2.0 not far away, its time to start thinking about supporting the new API version.

I'm considering rewriting this tool in Go, to make it faster / smaller and take advantage of the Rancher API bindings.

Does anyone have any thoughts on that? Will it break your workflow?

¿How change default options?

Hi! I deploy the containers using the upgrade command but I would be interested in using the option --start-before-stopping, that I see is set false by default. I run the upgrade like this:

upgrade --stack MyStack --service phpfpm --finish-upgrade --sidekicks --new-sidekick-image phpfpm-app $CONTAINER_RELEASE_IMAGE

If I add the option in there I got this and fails:

$ upgrade --stack MyStack --service phpfpm --finish-upgrade --sidekicks --new-sidekick-image --start-before-stopping phpfpm-app $CONTAINER_RELEASE_IMAGE
Usage: upgrade [OPTIONS]

How can I change that option?

random fail on service upgrade

Hello,

Thank you dude nice job with this tool !
Just to tell you sometime y have this error and i don t know why..

`$ upgrade --stack jankenpon --service jankenpon --no-start-before-stopping --environment lab

Upgrading JanKenPon/jankenpon in environment Lab...
Upgrade started, waiting for upgrade to complete...
Traceback (most recent call last):
File "/usr/local/bin/upgrade", line 11, in
load_entry_point('rancher-gitlab-deploy==1.2', 'console_scripts', 'rancher-gitlab-deploy')()
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/rancher_gitlab_deploy-1.2-py2.7.egg/rancher_gitlab_deploy/cli.py", line 200, in main
api, environment_id, service['id']
File "/usr/local/lib/python2.7/site-packages/requests-2.13.0-py2.7.egg/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.13.0-py2.7.egg/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.13.0-py2.7.egg/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.13.0-py2.7.egg/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.13.0-py2.7.egg/requests/adapters.py", line 473, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))`

if i retry it s works

thank again.

`$ upgrade --stack jankenpon --service jankenpon --no-start-before-stopping --environment lab
Upgrading JanKenPon/jankenpon in environment Lab...
Upgrade started, waiting for upgrade to complete...
Finishing upgrade...
Upgrade finished

Build succeeded`

rancher ui is on and the url is http://someurl.domaine.ovh:8080

Not Pulling New Images

This may or may not be a duplicate of #48, but I've been messing with this for a while trying to figure out why I'm not getting new images pulled down to my docker hosts.

The GitLab task completes successfully:

Checking out 0452d2aa as development...
Skipping Git submodules setup
$ upgrade --stack project-dev --service app --new-image registry.inf.project.com/project/prj_backend:development
Upgrading project-dev/app in environment Default...
Upgrade started, waiting for upgrade to complete...
Finishing upgrade...
Upgrade finished
$ upgrade --stack project-dev --service web
Upgrading project-dev/web in environment Default...
Upgrade started, waiting for upgrade to complete...
Finishing upgrade...
Upgrade finished
Job succeeded

However, the application isn't updating and running docker images on the host shows old images:

$ docker images
REPOSITORY                                       TAG                 IMAGE ID            CREATED             SIZE
registry.inf.project.com/project/prj_frontend   master              90e5db513f5b        2 days ago          16.1MB
registry.inf.project.com/project/prj_backend    development         1e97dd2ee07f        2 days ago          120MB

Do you have any ideas why this is so?

Side kicks don't start

Rancher 1.5.6
Latest deploy docker image

I have some container with data only sidekicks that show as 'stopped' after an upgrade via this tool and never actually run once. This both prevents the code from running on those containers as well as marking the stack as degraded. I've tried a few different flags but here is what I am using for options

--finish-upgrade --sidekicks --wait-for-upgrade-to-finish

Scripts Sometimes fails. Second try works

Sometimes it fails mostly second time it then works:

job_deploy:
stage: deploy
dependencies:
- job_build_image
image: cdrx/rancher-gitlab-deploy
script:
- upgrade --environment dod --stack bv-bva --service bv-bva-app

Skipping Git submodules setup
$ upgrade --environment dod --stack bv-bva --service bv-bva-app
Upgrading bv-bva/bv-bva-app in environment dod...
Upgrade started, waiting for upgrade to complete...
Traceback (most recent call last):
File "/usr/local/bin/upgrade", line 11, in
load_entry_point('rancher-gitlab-deploy==1.4', 'console_scripts', 'rancher-gitlab-deploy')()
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/rancher_gitlab_deploy-1.4-py2.7.egg/rancher_gitlab_deploy/cli.py", line 212, in main
api, environment_id, service['id']
File "/usr/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/adapters.py", line 490, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))
ERROR: Job failed: exit code 1

Trim env. variable names before passing them to Rancher

Actual behaviour:
When passing multiple variables alongside with the --variables multiline configuration might be used:

deploy:
  stage: deploy
  image: cdrx/rancher-gitlab-deploy
  script:
    - >
       upgrade
       --variables "
       DB_HOST=db.example.com,
       DB_USER=foo,
       DB_PASSWORD=bar
       "

Which result in the following command after YAML is processed:

upgrade --variables " DB_HOST=db.example.com, DB_USER=foo, DB_PASSWORD=bar"

Due to the current string splitting logic, the spaces before the env variables become a part of the variable passed to Rancher (1.6.30) which result in an invalid env variable which appears in the Rancher UI (e.g. <space>DB_HOST, incl. the space) but is not passed to the container.

Expected outcome:
Variable names and values are trimmed when passing as string with --variables option.

Pre/post deploy hooks

Hi there!

I'm not sure if that's even possible but i was thinking that it would be awesome to run a set of commands as a pre/post deploy hooks of rancher-gitlab-deploy.

For example - rails apps needs to run a $ rake db:migrate to check if there are any pending database migration and if there are, run them. Right now, i need to do this step manually which kinda sucks. Implementin such feature ins rancher-gitlab-deploy would make this way easier.

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.