cloud-provisioning's People
Forkers
cassiothadeu nuaays andrewrothstein jasonnic ngocngv mavencode01 necroscope soanni arkranon314 sdelrio patrickleet ankitarya10 merbla beta-uy zarbis albers 40a sasund sardjv joostvdg captainjack33 sanjaypandrala yifanyou asinishin jake-bladt miqui svunnam pseonghoon raf7raf namach briantyr cmendesce mpun flybyray prageethw pranabeshn isalihbegovic sonorati alitheondotcom dxbwalker chriscancompute mario21ic christopherauthier rogerscuall 9ide0n ozlevka-work do-agileplus khayrat tuffy-cloud hitomidennis nooproblem ytra-cloud vligade raghavendrads22 pramine amoskale benkorichard dolkensp-bookmarks picassio zelfick papioyue arun9thejacloud-provisioning's Issues
Prometheus targets problem
Hi, I am reading your book that devops-toolkit 2.1.
At "Collecting Metrics and Monitoring The Cluster"
I am trying your example but Prometheus Target - cadvisor and node Enpoints are wrong. I think beacuse of dns.
I am using latest version of cloud-provisioning repo.
Is there anything missed?
swarm creation: VIRTUALBOX_SHARE_FOLDER issue on Windows
@vfarcic As you already mention in your book The Devops 2.1 Toolkit, chapter "Setting Up Continuous Delivery Clusters", there is an issue with the volume mounts in current versions of docker-machine and VirtualBox on Windows.
Here's what I found out on this subject using Docker Toolbox 1.13.1 (i.e. Docker Machine 0.9.0 & VirtualBox 5.1.14) on Windows 7:
Your code uses the current directory mounted as a VirtualBox shared folder in several VMs in order to share registry data between the testing and production swarms.
Sadly, mounting custom directories as shared folders on Windows is broken in docker-machine 0.9.0.
Only the default mount of the user's home directory works.
This leads to two problems:
- The VMs do not contain the path
$PWD
.docker service create
fails because of an invalid mount option. - No data is shared between the swarms.
To verify the shared directory issue, create two machines (xx is a substitute for your user name):
docker-machine create --driver virtualbox default-shared-folder
docker-machine create --driver virtualbox --virtualbox-share-folder "/c/Users/xx:/XXX" custom-shared-folder
In the Oracle VM VirtualBox Manager, check the VM details. default-shared-folder
has one shared folder, custom-shared-folder
has none.
The reason for the failed creation of shared folders is that VIRTUALBOX_SHARE_FOLDER
and --virtualbox-share-folder
both have a source:destination
syntax, where source
on Windows machines must be given as c:/...
. The extra colon causes parsing of the argument to fail, see docker/machine#3975. This is fixed in docker/machine#3975. Using a custom build of docker-machine
, I was able to create a custom mount of the current directory with
docker-machine create --driver virtualbox --virtualbox-share-folder "$(cygpath -m $PWD):$PWD" custom-shared-folder
Because $pwd
returns the path in /c/...
format, some kind of conversion is needed. This example will not work on Macs. We need OS specific treatment.
I see the following solutions:
- create separate scripts for Windows and Mac, the Windows scripts using conversion with
cygpath
(available in Docker Toolbox) or variable substitutions. - enhance the existing scripts with OS specific conversion.
- create an issue in Docker-Machine that on Windows, paths like
/c/...
should be converted toc:/...
. - Remove
VIRTUALBOX_SHARE_FOLDER="$PWD:$PWD"
from the scripts. All users will have to work in a subdir of their homedir (like invfarcic/go-demo
). This is the only solution that works with the current Docker Machine.
Hope this information is helpful for you.
AWS related Packer Scripts failing to provision AMIs
I have run the Packer Build command against the various Packer json files in the "terraform/aws" and the "terrform/aws-full" directories and I encountered the same error with each one... resulting in build termination and no AMI image existing for any of them:
Error:
"amazon-ebs: E: can not open /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_xenial_InRelease - fopen (2: No such file or directory)"
Recommend:
Upon adding the "sudo apt-get clean" command to the "provisioners : inline" section for each of the Packer json files I was able to get them to build correctly.
Example:
"provisioners": [{ "type": "shell", "inline": [ "sudo apt-get clean", "sudo apt-get update", "sudo apt-get install -y apt-transport-https ca-certificates nfs-common", "sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D", "echo 'deb https://apt.dockerproject.org/repo ubuntu-xenial main' | sudo tee /etc/apt/sources.list.d/docker.list", "sudo apt-get update", "sudo apt-get install -y docker-engine", "sudo usermod -aG docker ubuntu", "sudo curl -L \"https://github.com/docker/compose/releases/download/1.8.1/docker-compose-$(uname -s)-$(uname -m)\" -o /usr/local/bin/docker-compose", "sudo chmod +x /usr/local/bin/docker-compose" ] }]
DevOps2.1: Automating Continuous Deployment Flow WithJenkins
I'm following along with the leanpub DevOps2.1 Toolkit. So far so good, but now the jenkins-agent service creating is proving difficult. With the swarm running:
export USER=admin
export PASSWORD=admin
docker service create --name jenkins-agent \
-e COMMAND_OPTIONS="-master \
http://$(docker-machine ip swarm-1):8082/jenkins \
-username $USER -password $PASSWORD \
-labels 'docker' -executors 5" \
--mode global \
--constraint 'node.labels.env == jenkins-agent' \
--mount "type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock" \
--mount "type=bind,source=$HOME/.docker/machine/machines,target=/machines" \
--mount "type=bind,source=/workspace,target=/workspace" \
vfarcic/jenkins-swarm-agent
If the line --mount "type=bind,source=$HOME/.docker/machine/machines,target=/machines" \
exist, it 'll show ERROR "invalid mount config for type…"
[weblogic@robertleepc cloud-provisioning]$ docker service ps jenkins-agent
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
s1za0gvjjrri jenkins-agent.meb853xkpo07kzz618yct2vua vfarcic/jenkins-swarm-agent:latest swarm-test-1 Ready Rejected 3 seconds ago "invalid mount config for type…"
6ik84sdnmml6 \_ jenkins-agent.meb853xkpo07kzz618yct2vua vfarcic/jenkins-swarm-agent:latest swarm-test-1 Shutdown Rejected 8 seconds ago "invalid mount config for type…"
sw49sebngqv9 \_ jenkins-agent.meb853xkpo07kzz618yct2vua vfarcic/jenkins-swarm-agent:latest swarm-test-1 Shutdown Rejected 8 seconds ago "invalid mount config for type…"
I need the line --mount "type=bind,source=$HOME/.docker/machine/machines,target=/machines" \
exist, it 'll show ERROR ``` in order to let Jenkins execute the Job "Pipeline go-demo"(scripts/go-demo-pull.groovy)!
withEnv([
"DOCKER_TLS_VERIFY=1",
"DOCKER_HOST=tcp://${env.PROD_LIKE_IP}:2376",
"DOCKER_CERT_PATH=/machines/${env.PROD_LIKE_NAME}"
])
I'm using Centos 7.3 ( 3.10.0-514.10.2.el7.x86_64)
Version info:
[weblogic@robertleepc cloud-provisioning]$ docker version
Client:
Version: 17.03.1-ce
API version: 1.27
Go version: go1.7.5
Git commit: c6d412e
Built: Fri Mar 24 00:36:45 2017
OS/Arch: linux/amd64
Server:
Version: 17.03.1-ce
API version: 1.27 (minimum version 1.12)
Go version: go1.7.5
Git commit: c6d412e
Built: Mon Mar 27 16:58:30 2017
OS/Arch: linux/amd64
Experimental: false
dm-test-swarm.sh doesn't work if `--virtualbox-memory 512` is set
I've tried this on my mac and on my linux box.
In both cases it just hangs at "waiting for an IP" and the docker-machine is never provisioned.
terraform error on digitalocean_droplet example
The following fails for the current Digital Ocean terraform source:
$ terraform plan
...
Error running plan: 2 error(s) occurred:
- Resource 'digitalocean_droplet.swarm-manager' not found for variable 'digitalocean_droplet.swarm-manager.2.id'
- Resource 'digitalocean_droplet.swarm-manager' not found for variable 'digitalocean_droplet.swarm-manager.1.id'
DevOpts2.1: Using Packer To Create Amazon Machine Images (AMIs)
Hi Viktor,
As always, Thank you sir for your hard work and great books. I've run into an issue though...
The first instruction in the "Using Packer To Create Amazon Machine Images (AMIs)" section of the book is to change to a specific directory [from root: vfarcic/cloud-provisioning]
cd terraform/aws
However, the 'aws' directory doesn't exist. Is this a typo, or, is a directory missing?
Add Elastic IP configuration to terraform
https://www.terraform.io/docs/providers/aws/r/eip.html
It'd be great if we could have elastic IPs created as part of the terraform scripts, that way it wouldn't be a manual step after the cluster's creation.
update swarm deploy to support Terraform v0.9.1
When deploying swarm using terraform v0.9.1 output variables are not showing but with v0.8.5 everything works as it show do.
After deploying the first manager node with
terraform apply \ -target aws_instance.swarm-manager \ -var swarm_init=true \ -var swarm_managers=1
No output variables are displayed.
DevOps2.1: Jenkins CI/CD
I'm following along with the leanpub DevOps2.1 Toolkit. So far so good, but now the Jenkins CI/CD setup is proving difficult. With the swarm running:
Let’s create a Jenkins service.
1 mkdir -p docker/jenkins
2
3 docker service create --name jenkins \
4 -p 8082:8080 \
5 -p 50000:50000 \
6 -e JENKINS_OPTS="--prefix=/jenkins" \
7 --mount "type=bind,source=$PWD/docker/jenkins,target=/var/jenkins_home" \
8 --reserve-memory 300m \
9 jenkins:2.7.4-alpine
10
11 docker service ps jenkins
...
docker service ps jenkins
reports an error for the service ("invalid bind mount source, source path not found..."). However, the directory $PWD/docker/jenkins
exists on local host.
I find a reference to this issue here: https://github.com/docker/docker/issues/25912. Still I can't resolve it.
Feels like I'm missing something obvious... Any help greatly appreciated. Thanks for sharing your work.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.