For Maven

• sutbrunner.ids property
• apicompatibility profile
• smoke profile
• e2e profile

For Gradle

• apply from pipeline.gradle
• provide the necessary ext properties

Does the pipeline only supports PCF instance for now? Any guide for how to discover computing resource on K8s, Openstack or other platforms?

Due to this GIT repo is not properly parsed

First version will be just revert the A/B testing

As a Spring user, I'd like to understand the current gaps and challenges leveraging the concourse pipeline for PCF deployments.

Acceptance:

• It would be ideal for me to review the list of pros/cons
• I'd also be interested in any known gaps and workarounds to make a spring-cloud project work seamlessly
• Any other general concerns and/or feature requirements that we need to bring it up with the Concourse team

I have run into some scenarios (due to my own mistakes) where the Jenkins pipeline will try to download artifacts from Artifactory that do not exist. Artifactory responds to these requests with some JSON response that has a 404 message in it. However the scripts will then try to deploy these "jars" to CF but then CF fails to start them. The end result is a very misleading error message from CF. It would be nice if we could inspect what is saved when pulling Jars from Artifactory to make sure it is not a 404 JSON response.

Some repositories require that artifacts be signed in order to do a release. To do this one needs their GPG keys they want to use to sign the artifacts imported into the GPG agent running in the Docker image.

By adding a few details to the CF deployment phase, it's possible that apps deployed from a Concourse pipeline would be visible as proper "server groups" inside a Spinnaker installation monitoring the same space.

Env Variables

This screenshot shows full complement of variables, that when set, makes a server group visible to Spinnaker.

• SPINNAKER_LOAD_BALANCERS is the name of the route from which the CF app would be visible, i.e. it's public URL (minus the domain). This is required.

However, just about everything else is optional, and provides extra information, but isn't needed for the app to be visible to Spinnaker (though if avaiable, let's populate!)

App name

The other thing needed to make the CF app visible to Spinnaker as a "server group" is the actual name of the application. There is a precise format that must be followed or it doesn't work.

While it might be old habit to name some demo pushed to CF spring-a-gram-demo-for-jug, Spinnaker has a library to enforce a format that looks like this => springagram-dev-demo-for-jug-v023.

The app name is split up by -'s, and tokenized as follows:

• springagram - app name. Think of github repo where the code comes from. Or if you have a set of microservices that work together, this could be common element between all of them.
• dev - stack. It's a logical label, often using things like "dev", "staging", "production", "gh123". This can also be how you distinguish a suite of microservices running in staging vs. the same stuff in production.
• demo-for-jug - details. A free form where you can put anything, even dashes. When you have a set of microservices, it's great for describing each one, e.g. "circuit-breaker", "service-discovery", "email-service", "frontend", etc.
• v023 - version. The last piece which denotes the exact version. Has to start with "v", and has to contain exactly three digits. Instead of "blue" and "green", they simply bump up the number and NEVER update an existing CF app, but instead push out a newer copy. For an out-of-the-box pipeline, we merely need to start with "v000".

NOTE: None of this dictates what org/space an app is deployed into.

Issues with this

1. This can introduce complexity. That's why I'm leaning toward making it an option. The extra metadata and formatting of the app can add confusion we don't want if the user doesn't intend to use Spinnaker.
2. If someone starts without Spinnaker-compatible, they can't expect to convert already deployed stuff into Spinnaker-compatible.
3. It also introduces a wrinkle if you intend to support blue/green deployments. Blue/green deployments that are Spinnaker-compatible would require a poor man's version of what Spinnaker does. When pushing, instead of finding the inactive side and updating it, instead find the latest, bump up the version by 1, and push things out, then move the route. Issues arise when you see potential roll over scenarios, e.g. v999 and v000 currently deployed.

We should be setting the JAVA_OPTS env var in the Dockerfile for Jenkins instead of in the Docker Compose file.

Most likely the path to the .m2 is wrong

Currently, we're hardcoding the memory to 256m. It would be good to be able to override that value.

The new concourse 3.3.0 introduces a feature (https://github.com/concourse/concourse/releases/tag/v3.3.0) where certain directories can be marked as cached, for subsequent runs the cached directory will be reused. With this new feature in place, I feel that the base custom image used with the concourse pipelines with cached resources can be changed to a generic image with java/maven/gradle in it

That way Artifactory won't be really necessary in the cloud

I was just trying to run through the documentation and try out the jenkins version of the demo and ran into this problem when trying to run

./start.sh gitUser gitPass gitOrg

Obviously with my real credentials and what not in there and got this error during starting up the containers via docker-compose

/bin/sh: 1: cannot create /usr/share/jenkins/gituser: Permission denied
ERROR: Service 'jenkins' failed to build: The command '/bin/sh -c printf "%s" "${gituser}" > /usr/share/jenkins/gituser' returned a non-zero code: 2

Looking into the Dockerfile I added my credentials there as well (that's a documentation miss I think)

Tried again, same error, which isn't shocking because my change has nothing to do with permissions.

Using ROOT that error disappears, and the credentials appear in jenkins (which I think is all that's doing there) so it seems like a viable solution.

I'm using Mac OS X El Capitan, Docker for Mac 1.12.1, and Virtual Box 5.1.8

Describe rationale behind each step of the opinionated pipeline. Talk about configuration options.

Work is started here - https://github.com/spring-cloud-samples/github-webhook/tree/kubernetes
Templating scripts - https://gist.github.com/marcingrzejszczak/6941e89e6f5a9fe2e69091345aada936

• template the deployment script
• template the service script
• build docker images for eurekas
• build docker images for stub runner boot
• setup deployment configurations in sc-pipelines-repo for stub runner boot
• build docker images for mysql
• setup deployment configurations in sc-pipelines-repo for mysql
• build docker images for rabbit
• setup deployment configurations in sc-pipelines-repo for rabbit
• ensure that the application & stub runner boot are services (the rest can be a deployment)
• update Jenkins tests to use PersistentVolume & VolumeChain together with a Pod for test steps

K8 deployment scripts for Maven

• ./mvnw clean deploy docker:build -DpushImageTag

@marcingrzejszczak - @malston and I saw this error when executing start.sh script to spin up the jenkins/artifactory:

Successfully tagged jenkins_jenkins:latest
Pulling artifactory (jfrog-docker-registry.bintray.io/artifactory/artifactory-oss:latest)...
ERROR: unauthorized: Repository '/jfrog/registry' does not support Docker API version v2

We got past it by using the docker.bintray.io/jfrog/ docker registry, but had to create the maven repo(lib-releases) ourselves.

Using docker-compose version 1.14.0, Docker version 17.06.0-ce

Let's assume that version v1 for com.example:foo got deployed to production. In the stubs jar we have packed the groovy contracts for the foo service.

Now we run the new pipeline - v2. We can check backward compatibility of the API by running the contract tests again but by picking the contracts from the v1 jar. We can easily do that with the contracts jar dependency and by providing a path to the contracts.

with a default to present.

If the checkbox is switched off then there will be no stage step.

for github-analytics and test environment, it should be stubrunner-github-analytics-test

The ideas:

• define a yaml where you define which app in which version and configuration should be deployed. Next the spring cloud deployer would pick it up and deploy everything to CF. Cool thing about this is that it would work both for CF and on a developer's machine

That way we can run stubs against latest PROD and DEV version

Create "Deploy to staging" and "Deploy to production" pipelines.

Deploy to staging should:

• Depend on a Jenkins build job for a trigger
• Leverage a Jenkins smoke test job
• Deploy using highlander strategy, presuming staging resources should be kept to a minimum.

Deploy to production should:

• Depend on deploy to staging pipeline as a trigger
• Use Find Image for artifact discovery, finding the newest server group
• Deploy using a red/black strategy with 2 server groups.

These jobs should presume user configures Spinnaker for a single CF space.

cc @ryanjbaxter

It confuses the Concourse users (they use pipeline.yml often)

for manual step deployment

http://micrometer.io/docs/prometheus#_spring_support

Under this issue please comment on what you had to manually customize in the project so that it suit the needs for your company.

• I don't want to use CF (#61)
• Remove all the stage / prod Eureka / MySQL / Rabbit installations - move them to a script that should be executed before running the pipeline
• Add customization of settings.xml / gradle.properties
• Create a CLI to remove unnecessary parts of code

That way we'll be able to speed the build cause currently poms are checked against every repo

Maybe we can ping a few jetbrains folks to help out.

Hi, I am likely missing some step - for some reason I am getting the following exception when running the update-m2 step in the concourse pipeline. Does anything jump out on what I may be missing here.

Root folder is [/tmp/build/0a57e892]
Repo resource folder is [repo]
Tools resource folder is [tools]
Version resource folder is [version]
tools/concourse/tasks/cache-m2.sh: 14: tools/concourse/tasks/cache-m2.sh: source: not found
Changing the maven local to [/tmp/build/0a57e892/]
tools/concourse/tasks/cache-m2.sh: 20: [: unexpected operator
tools/concourse/tasks/cache-m2.sh: 25: tools/concourse/tasks/cache-m2.sh: ./mvnw: not found

Currently it's hardcoded

After cloning a project we could read some particular environment variables for that project. That way the pipeline can remain the same but it can try to read env vars from a file by convention e.g. sc-pipelines.properties

If one wants to redeploy the JAR to production then if the tag is there, then it's not possible

Currently the repositoryRoot param is passed to the app even when we want stub runner to use classpath