Giter VIP home page Giter VIP logo

workflow-e2e's Introduction

Deis Workflow will soon no longer be maintained.
Please read the announcement for more detail.
09/07/2017 Deis Workflow v2.18 final release before entering maintenance mode
03/01/2018 End of Workflow maintenance: critical patches no longer merged

Deis Workflow End to End Tests v2

Build Status Go Report Card Docker Repository on Quay

Deis (pronounced DAY-iss) Workflow is an open source Platform as a Service (PaaS) that adds a developer-friendly layer to any Kubernetes cluster, making it easy to deploy and manage applications on your own servers.

For more information about the Deis Workflow, please visit the main project page at https://github.com/deisthree/workflow.

We welcome your input! If you have feedback, please submit an issue. If you'd like to participate in development, please read the "Development" section below and submit a pull request.

About

The code in this repository is a set of Ginkgo and Gomega based integration tests that execute commands against a running Deis cluster using the Deis CLI.

Development

The Deis project welcomes contributions from all developers. The high level process for development matches many other open source projects. See below for an outline.

  • Fork this repository
  • Make your changes
  • Submit a pull request (PR) to this repository with your changes, and unit tests whenever possible.
    • If your PR fixes any issues, make sure you write Fixes #1234 in your PR description (where #1234 is the number of the issue you're closing)
  • The Deis core contributors will review your code. After each of them sign off on your code, they'll label your PR with LGTM1 and LGTM2 (respectively). Once that happens, the contributors will merge it

Prerequisities

Before you run the tests, you'll need a full Deis cluster up and running in Kubernetes. Follow the instructions here to get one running.

Run the Tests

There are three options for how to execute the tests. These include two options for executing the tests against Deis Workflow installed on a remote Kubernetes cluster, and one option for installing the same tests directly into a Kubernetes cluster and executing them there.

Remote Execution

Either of two options for remote execution of the test suite require the DEIS_CONTROLLER_URL environment variable to be exported. Its value should be the the controller endpoint you would normally use with the deis register or deis login commands:

$ export DEIS_CONTROLLER_URL=http://deis.your.cluster

Tests execute in parallel by default. If you wish to control the number of executors, export a value for the GINKGO_NODES environment variable:

$ export GINKGO_NODES=5

If this is not set, Ginkgo will automatically choose a number of test nodes (executors) based on the number of CPU cores on the machine executing the tests. It is important to note, however, that test execution is constrained more significantly by the resources of the cluster under test than by the resources of the machine executing the tests. The number of test nodes, therefore, should be explicitly set and scaled in proportion to the resources available in the cluster.

For reference, Workflow's own CI pipeline uses the following:

Test Nodes Kubernetes Worker Nodes Worker Node CPU Worker Node Memory
5 3 4 vCPUs 15 GB

Setting the GINKGO_NODES environment variable to a value of 1 will allow serialized execution of all tests in the suite.

Native Execution

If you have Go 1.5 or greater already installed and working properly and also have the Glide dependency management tool for Go installed, you may clone this repository into your $GOPATH:

git clone [email protected]:teamhephy/workflow-e2e.git $GOPATH/src/github.com/deis/workflow-e2e

One-time execution of the following will resolve the test suite's own dependencies:

$ make bootstrap

To execute the entire test suite:

$ make test-integration

To run a single test or set of tests, you'll need the Ginkgo tool installed on your machine:

$ go get github.com/onsi/ginkgo/ginkgo

You can then use the --focus option to run subsets of the test suite:

$ ginkgo --focus="deis apps" tests

Containerized Execution

If you do not have Go 1.5 or greater installed locally, but do have a Docker daemon running locally (or are using docker-machine), you can quite easily execute tests against a remote cluster from within a container.

In this case, you may clone this repository into a path of your own choosing (does not need to be on your $GOPATH):

git clone [email protected]:teamhephy/workflow-e2e.git /path/of/your/choice

Then build the test image and execute the test suite:

$ make docker-build docker-test-integration

Within the Cluster

A third option is to run the test suite from within the very cluster that is under test.

To install the helm chart and start the tests, assuming helm and its corresponding server component tiller are installed:

git clone git clone [email protected]:teamhephy/workflow-e2e.git; cd workflow-e2e
helm dependency update charts/workflow-e2e
helm upgrade --install workflow-e2e charts/workflow-e2e --namespace deis

To monitor tests as they execute:

$ kubectl --namespace=deis logs -f workflow-e2e tests

Special Note on Resetting Cluster State

All tests clean up after themselves, however, in the case of test failures or interruptions, automatic cleanup may not always proceed as intended. This may leave projects, users or other state behind, which may impact future executions of the test suite against the same cluster. (Often all tests will fail.) If you see this behavior, run these commands to clean up. (Replace deis-workflow-qoxhz with the name of the deis/workflow pod in your cluster.)

$ kubectl exec -it deis-workflow-qoxhz python manage.py shell
Python 2.7.10 (default, Aug 13 2015, 12:27:27)
[GCC 4.9.2] on linux2
>>> from django.contrib.auth import get_user_model
>>> m = get_user_model()
>>> m.objects.exclude(username='AnonymousUser').delete()
>>> m.objects.all()

Note that this is an ongoing issue for which we're planning a more comprehensive fix.

workflow-e2e's People

Contributors

mboersma avatar carmstrong avatar aledbf avatar arschles avatar helgi avatar krancour avatar johanneswuerbach avatar joshua-anderson avatar smothiki avatar bengrunfeld avatar vdice avatar tombh avatar technosophos avatar xe avatar nathansamson avatar kmala avatar progrium avatar iancoffey avatar croemmich avatar ngpestelos avatar glogiotatidis avatar kalbasit avatar babarinde avatar wenzowski avatar romansergey avatar tscheepers avatar sedouard avatar scottstamp avatar adamkdean avatar jgmize avatar

Watchers

James Cloos avatar Kingdon Barrett avatar AMO ❤️⛺✨ avatar

workflow-e2e's Issues

e2e runner thinks it is succesfully interacting with router

From @jchauncey on August 5, 2016 19:32

00:01:13.872 All pods are running!
00:01:13.872 Checking to see if the workflow has come up properly.
00:01:14.265 The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:14.270 curl: (3) <url> malformed
00:01:15.674 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:15.679 curl: (3) <url> malformed
00:01:17.084 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:17.089 curl: (3) <url> malformed
00:01:18.489 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:18.494 curl: (3) <url> malformed
00:01:19.891 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:19.897 curl: (3) <url> malformed
00:01:21.296 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:21.302 curl: (3) <url> malformed
00:01:22.711 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:22.716 curl: (3) <url> malformed
00:01:24.120 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:24.126 curl: (3) <url> malformed
00:01:25.519 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:25.524 curl: (3) <url> malformed
00:01:26.929 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:26.935 curl: (3) <url> malformed
00:01:28.340 .The connection to the server 104.198.208.203 was refused - did you specify the right host or port?
00:01:28.345 curl: (3) <url> malformed
00:01:30.262 .curl: (3) <url> malformed
00:01:31.972 .curl: (3) <url> malformed
00:01:33.683 .curl: (3) <url> malformed
00:01:35.403 .curl: (3) <url> malformed
00:01:37.120 .curl: (3) <url> malformed
00:01:38.838 .curl: (3) <url> malformed
00:01:40.542 .curl: (3) <url> malformed
00:01:42.259 .curl: (3) <url> malformed
00:01:43.959 .curl: (3) <url> malformed
00:01:45.668 .curl: (3) <url> malformed
00:01:47.407 .curl: (3) <url> malformed
00:01:49.091 .curl: (3) <url> malformed
00:01:50.785 .curl: (3) <url> malformed
00:01:52.483 .curl: (3) <url> malformed
00:01:54.194 .curl: (3) <url> malformed
00:01:55.898 .curl: (3) <url> malformed
00:02:20.933 .Successfully interacted with Deis platform 10 time(s).

Copied from original issue: deis/workflow-e2e#292

add autoscale tests

From @vdice on May 25, 2017 18:53

 $ deis autoscale --help

Valid commands for autoscale:

autoscale:list   list autoscale options of an application
autoscale:set    turn on autoscale for an app
autoscale:unset  turn off autoscale for an app

Use 'deis help [command]' to learn more.

Copied from original issue: deis/workflow-e2e#359

This repo needs some TLC

Now that hopefully most of the deis/* go packages have been updated to teamhephy repos, this can be updated to point at them and used again.

I'll take a look at it.

test app needs to be configurable

From @mboersma on February 19, 2016 23:42

Currently it's hardcoded to Deis' example-go app, but the actual git URL used should be customizable via environment variable or another mechanism, so that any 12-factor app repo (not just those in the deis org) can be run through its paces.

Copied from original issue: deis/workflow-e2e#44

apps:run test with lengthy output -- don't log actual output on failure

From @vdice on May 26, 2017 22:17

When/if this apps:run test fails, the Expect statement proceeds to log the entire command's output, corrupting the user's terminal and crowding out logs with byte code.

We should re-jigger the test, perhaps checking first if the byte array does not match expected (outside of Expect) and then, if not, logging some standard error/failure message accordingly.

Copied from original issue: deis/workflow-e2e#360

add support for running with private registry

From @vdice on May 3, 2017 20:35

Currently e2e tests don't support running on a cluster using a private registry.

To deploy an application on Workflow while using a private registry, the PORT env var needs to be set in the application config before a deploy can occur. E2E doesn't yet attempt to do this, so all such tests will fail with:

...
$ deis apps:create test-661286376 --no-remote
Creating Application... done, created test-661286376
If you want to add a git remote for this app later, use `deis git:remote -a test-661286376`
$ deis builds:create --app=test-661286376 deis/example-dockerfile-http
Creating build... Error: Unknown Error (400): {"detail":"PORT needs to be set in the application config when using a private registry"}
$ deis apps:destroy --app=test-661286376 --confirm=test-661286376
Destroying test-661286376...
done in 2s
...

Filing this issue so it is more readily known and perhaps to actually add support in future!

Copied from original issue: deis/workflow-e2e#356

Make test code print an informational error message if it doesn't find 'DEIS_CONTROLLER_URL' in the environment

From @arschles on April 12, 2016 20:13

After deis/workflow-e2e#157, the tests look for DEIS_CONTROLLER_URL in the environment to determine what server to talk to in order to run tests. This variable can be set to a variety of different values, and the test should explain the options:

  • If you're inside a Kubernetes pod inside a namespace with a full Deis install, set the value to the internal router address: DEIS_CONTROLLER_URL=${DEIS_ROUTER_SERVICE_HOST}:${DEIS_ROUTER_SERVICE_PORT}
  • If you're not inside a Kubernets pod, set the value to the external router address. This value depends on what kind of ingress server you have set up. It might be a cloud load balancer or otherwise. Assuming you have a public IP for the router: DEIS_CONTROLLER_URL=deis.${EXTERNAL_ROUTER_IP}.xip.io (or you can use .nip.io, or your own domain if you've configured one...)

Copied from original issue: deis/workflow-e2e#163

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.