Giter VIP home page Giter VIP logo

omg-cli's Introduction

- omg -

it's kind of like an (o)ps (m)anager in (g)olang

an iaas independent plugable executable to install bosh, cloud configs and product deployments

wercker status

release info

What is OMG (http://enaml.pezapp.io)

omg is a cli tool. It natively allows users to:

  • spin up a bosh on a target iaas,
  • load it up with a cloud config
  • deploy 'products' via their new bosh (vault, cloudfoundry, concourse, etc)

What are plugins

plugins are your way of extending omg, providing a deployment definition or cloud config definition. instead of dealing with yaml or tiles, we build testable plugins using enaml and simply register them with omg.

download a bundled plugin from a omg release or build your own available plugin types are cloudconfig or product for more info about how to build a plugin take a look at one of the bundled plugins (ie. https://github.com/enaml-ops/omg-cli/tree/master/cloudconfigs/aws)

Version Compatibility

currently uses pluginlib v1 w/ protocol version 2. will not work with plugins prior to version 1.

how we do bosh / cloud config / deployments

composes bosh-init, enaml and plugins to create a simple cli installer

downloads:

install a BOSH using OMG-cli (aws example)

check the bosh docs to setup your vpc (https://bosh.io/docs/init-aws.html)

# download omg cli
$> wget -O omg https://github.com/enaml-ops/omg-cli/releases/download/v0.0.25/omg-osx && chmod +x omg
# the below dependencies only apply if you are looking to install a BOSH Director with omg-cli
$> sudo apt-get update
$> sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3

or

$> xcode-select --install
$> brew install openssl

# deploy your bosh using the omg cli
$> ./omg aws \
--mode uaa \
--aws-subnet subnet-xxxxxxxxxxx \
--bosh-public-ip x.x.x.x \
--aws-pem-path ~/bosh.pem \
--aws-access-key  xxxxxxxxxxxxxxxxxxxxxx \
--aws-secret xxxxxxxxxxxxxxxxxxx \
--aws-instance-size t2.micro \
--aws-region us-east-1 \
--aws-availability-zone us-east-1c

instructions on how to install BOSH on other supported iaas can be found by:

$> ./omg azure --help
$> ./omg aws --help
$> ./omg vsphere --help
$> ./omg photon --help
$> ./omg gcp --help

Setup Cloud Config on your BOSH (aws example)

# download cloudconfig plugin for aws
$> wget https://github.com/enaml-ops/omg-cli/releases/download/v0.0.25/aws-cloudconfigplugin-osx
# register the cloud config plugin for your iaas
$> ./omg register-plugin \
-type cloudconfig \
-pluginpath aws-cloudconfigplugin-osx
# to see your newly added plugin
$> ./omg list-cloudconfigs
Cloud Configs:
aws  -  .plugins/cloudconfig/aws-cloudconfigplugin-osx  -  map[]
# upload cloud config
$> ./omg deploy-cloudconfig \
--bosh-url https://bosh.url.com --bosh-port 25555 \
--bosh-user admin --bosh-pass admin --ssl-ignore \
aws-cloudconfigplugin-osx \
--az z1 \
--az z2 \
--aws-availablity-zone us-east-1a \
--aws-availablity-zone us-east-1b \
--network-name-1 deployment \
--network-az-1 z1 \
--network-cidr-1 10.0.16.0/20 \
--network-gateway-1 10.0.16.1 \
--network-dns-1 10.0.0.2 \
--network-reserved-1 10.0.16.2-10.0.16.10 \
--network-static-1 10.0.16.11 \
--aws-subnet-name-1 subnet-1 \
--aws-security-group-1 sg-1234
--network-name-2 deployment \
--network-az-2 z2 \
--network-cidr-2 10.0.32.0/20 \
--network-gateway-2 10.0.32.1 \
--network-dns-2 10.0.0.2 \
--network-reserved-2 10.0.32.2-10.0.32.10 \
--network-static-2 10.0.32.11 \
--aws-subnet-name-2 subnet-2 \
--aws-security-group-2 sg-1234

for information on other options and flags:

$> ./omg deploy-cloudconfig aws-cloudconfigplugin-osx --help

How to use omg + plugins to install a product (ex,. concourse on aws)

tips & tricks

  • set LOG_LEVEL=debug for verbose output
  • adding the --print-manifest flag with the bosh creds will simply print the manifest you are about to deploy

bosh deployed concourse

deploy a concourse

# download concourse product plugin
$> wget https://github.com/enaml-ops/omg-product-bundle/releases/download/v0.0.14/concourse-plugin-osx
# register concourse product plugin
$> ./omg register-plugin -type product -pluginpath concourse-plugin-osx
# please only upload your releases and stemcells manually if your deployment does not use remote urls
# otherwise this will be automatically uploaded via omg-cli
# deploy your concourse
$> ./omg deploy-product \
--bosh-url https://bosh.url.com --bosh-port 25555 --bosh-user admin \
--bosh-pass admin --ssl-ignore \
concourse-plugin-osx \
--web-vm-type small \
--worker-vm-type small \
--database-vm-type small \
--network-name private \
--url my.concourse.com \
--username concourse \
--password concourse \
--web-instances 1 \
--web-azs us-east-1c \
--worker-azs us-east-1c \
--database-azs us-east-1c \
--bosh-stemcell-alias trusty \
--postgresql-db-pwd secret \
--database-storage-type medium

omg-cli's People

Contributors

anwarchk avatar bijukunjummen avatar cholick avatar dlinsley avatar jmcdice avatar patrickhuber avatar s-matyukevich avatar sneal avatar xchapter7x avatar z4ce avatar zmb3 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar

Forkers

dambor daverc1-pivotal ubergeekcd crdant patrickhuber matthewcosgrove s-matyukevich dlinsley z4ce jtgammon

omg-cli's Issues

`omg` photon does not add properties.director.cpi_job

omg does not add properties.director.cpi_job and then deployments fail with an empty cpi path element:

Director task 8
  Started update stemcell
  Started update stemcell > Extracting stemcell archive. Done (00:00:04)
  Started update stemcell > Verifying stemcell manifest. Done (00:00:00)
  Started update stemcell > Checking if this stemcell already exists. Done (00:00:00)
  Started update stemcell > Uploading stemcell bosh-vsphere-esxi-ubuntu-trusty-go_agent/3232.17 to the cloud. Failed: Failed to run cpi: '/var/vcap/jobs//bin/cpi' is not executable (00:00:00)

Error 100: Failed to run cpi: '/var/vcap/jobs//bin/cpi' is not executable

Hardcoded Key Name "bosh"

As an operator, I should be able to either pass in a key name or default key name should match pem filename used in flag --aws-pem-path

Usability improvements

Some usability improvements that would be welcome:

  • If a required flag is not present, or the data is empty in vault, don't just panic. Print what's missing and exit 1.
  • Plugins should provide more useful output. At the very least, exit with the task id being executed in BOSH, so a deployer could eventually follow it.
  • register-plugin should overwrite the existing plugins by default.
  • In register-plugin, the type should be part of the metadata, and eliminate the -type flag altogether. This makes all plugins equal, regardless of the type. This would also make deploy-cloudconfig and deploy-product redundant. It would just be deploy.
  • Eliminate the -pluginpath flag. Just make the path to the plugin mandatory.
  • Add a --get flag where the plugin is downloaded by name and installed directly. No local files needed (more like an apt-get).
  • Add an --available-plugins flag, that will list all plugins available and not installed. It can optionally mark the ones that are outdated.
  • Add an --update flag, to update plugins directly from the Internet (or a repo).
  • A man page.

aws bosh plugin fails to deploy for private ip addresses 

omg aws \
--mode uaa \
--aws-subnet subnet-X \
--aws-pem-pathX \
--aws-access-key  X \
--aws-secret X \
--aws-instance-size t2.micro \
--aws-region us-west-1 \
--aws-availability-zone us-west-1b \
--cidr 10.0.16.0/20 \
--gateway 10.0.16.1 \
--dns 10.0.16.2 \
--bosh-private-ip 10.0.16.6 \
--aws-security-group enamltest

gives this error:

Command 'deploy' failed:
  Deploying:
    Creating instance 'bosh/0':
      Creating VM:
        Creating vm with stemcell cid 'ami-74b7f414 light':
          CPI 'create_vm' method responded with error: CmdError{"type":"Bosh::Clouds::CloudError","message":"No IP provided for vip network `public'","ok_to_retry":false}

Add a flag to add a VIP IP for public facing jobs

We don't always deploy products (concourse, cloudfoundry) behind a non-bosh managed load balancer. It'd be nice to be able to specify a public VIP for those internet facing jobs (web job in concourse, which I'm told listens on :8080, and HAProxy in cloudfoundry). You'd probably also want to add some logic so someone doesn't attempt to have 3 HAProxies with only 1 VIP, etc.

--cpi-name should be removed / defaulted / explained

Trying to do an Azure deployment using OMG of bosh-init deployed microbosh.

The end user doesn't know what this needs to be set to (default: cpi).
omg azure --help shows nothing.

Deploying prints this:
Sorry you need to provide [cpi-name] flags to continue

String slice arguments are not parsed corectly

I try to execute the following command

omg-cli photon \
  --dns 10.148.20.5,10.148.20.6

on the deployed vm I see in /etc/resolv.conf

nameserver 10.148.20.5,10.148.20.6

Passing coma separated list of values also don't work for '--network-dns-1' flag when deploying cloud-config, though help message says that this parameter expects comma separated list.

vsphere cloud config should allow creation of a single network object w multiple subnets

Currently when multiple networks are passed to omg-cli / vpshere-cc to gen a cloud config, 1 subnet is created per network:

AZ1->Net1->Subnet1
AZ2->Net2->Subnet1
AZ3->Net3->Subnet1

We should be able to support a single network object with multiple subnets to match what customers can deploy w Ops Man today:

Net1->Subnet1->AZ1
Net1->Subnet2->AZ2
Net1->Subnet3->AZ3

omg help is not displayed unless the user is in the same dir as the executable 

sandbox $ pwd
/home/ops

$ which omg-linux
/home/ops/bin/omg-linux

sandbox $ omg-linux -h
<nothing>

sandbox $ cd /home/ops/bin
sandbox $ omg-linux -h
NAME:
   omg-linux - A new cli application

USAGE:
   omg-linux [global options] command [command options] [arguments...]

VERSION:
   v1.0.3-572df1

COMMANDS:
     azure               azure [--flags] - deploy a bosh to azure
     aws                 aws [--flags] - deploy a bosh to aws
     gcp                 gcp [--flags] - deploy a bosh to GCP
     photon              photon [--flags] - deploy a bosh to photon
     vsphere             vsphere [--flags] - deploy a bosh to vsphere
     list-cloudconfigs
     list-products
     product-meta        product-meta <prod-name> - show product metadata
     register-plugin     register-plugin -type [cloudconfig, product] -pluginpath <plugin-binary>
     deploy-cloudconfig  deploy-cloudconfig <cloudconfig-name> [--flags] - deploy a cloudconfig to bosh
     deploy-product      deploy-product <prod-name> [--flags] - deploy a product via bosh
     help, h             Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help (default: false)
   --version, -v  print the version (default: false)

Concourse plugin requires --web-ip

+1 to @haydonryan #37. Saw that exact error. Then after uploading the stemcell, I got the exact same error again. This time, the root cause was:

==> beacon_ctl.log <==
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'
/var/vcap/jobs/groundcrew/bin/beacon_ctl: line 51: syntax error near unexpected token `}'

Turns out that happens when you don't provide the --web-ip parameter, because then the number of web instances defaults to 0 and that breaks a shell script. This time, even the bosh task logs wouldn't have been very helpful in identifying the root cause:

2016/11/09 12:56:50 D1109 12:56:50.526933 10963 api.go:177] rest resp:  {"id":23,"state":"error","description":"create deployment","timestamp":1478717809,"started_at":1478717482,"result":"'worker/0 (4837b190-e7b7-41d4-8ff2-144743ba6b0d)' is not running after update. Review logs for failed jobs: beacon,...","user":"director","deployment":"concourse"}

Seems the the concourse plugin should require the --web-ip parameter since omitting it leads to an obscure failure.

Add Flag to support trusted certs in bosh.

Gemfire tile requires the director to have a trusted cert inserted by bosh.

director:
      address: 127.0.0.1
      name: bosh
      db: *db
      cpi_job: cpi
      enable_snapshots: false
      trusted_certs: |
        -----BEGIN CERTIFICATE-----

Generated manifest for bosh director doesn't work unless a public IP address is specified

Hi,
I am trying to spin up a bosh director in a private subnet, hence no public ip assigned to it. When trying to run the manifest generated by omg, i am getting this error

Command 'deploy' failed:
  Deploying:
    Creating instance 'bosh/0':
      Creating VM:
        Creating vm with stemcell cid 'ami-51c1e546 light':
          CPI 'create_vm' method responded with error: CmdError{"type":"Bosh::Clouds::CloudError","message":"No IP provided for vip network `public'","ok_to_retry":false}

Can you please advice?

Manifest error when deploying bosh director on GCP

I am seeing the following error when I try to deploy the bosh director to GCP on omg version v1.0.7-628bb0.

omg gcp --mode uaa --gcp-network-name pcf --gcp-subnetwork-name bosh --gcp-project fe-jgammon --gcp-machine-type g1-small --gcp-default-zone us-central1-a --gateway 10.0.0.1 --dns 8.8.8.8 --cidr 10.0.0.0/24 --bosh-private-ip 10.0.0.10 

Deployment manifest: '/home/jgammon/omg/omg-bosh.349171'
Deployment state: '/home/jgammon/omg/omg-bosh-state.json'

Started validating
  Downloading release 'bosh'... Skipped [Found in local cache] (00:00:00)
  Validating release 'bosh'... Finished (00:00:04)
  Downloading release 'uaa'... Skipped [Found in local cache] (00:00:00)
  Validating release 'uaa'... Finished (00:00:08)
  Downloading release 'bosh-google-cpi'... Skipped [Found in local cache] (00:00:00)
  Validating release 'bosh-google-cpi'... Finished (00:00:07)
  Validating cpi release... Finished (00:00:00)
  Validating deployment manifest... Failed (00:00:00)
Failed validating (00:00:20)

Command 'deploy' failed:
  Validating deployment manifest:
    only one network can be the default for 'dns'
    only one network can be the default for 'gateway'

I looked at the manifest, and I'm pretty sure the issue is here where it lists the private network twice.

  resource_pool: vms
  instances: 1
  networks:
  - name: private
    static_ips:
    - 10.0.0.10
    default:
    - dns
    - gateway
  - name: private
    static_ips:
    - 10.0.0.10
    default:
    - dns
    - gateway
cloud_provider:

`omg gcp` Creates Bad Manifest

Using the most recent release (1.0.1), the omg gcp command creates an un-deployable manifest. The cloud_provider section's template is missing a name:

cloud_provider:
  template: {name: "", release: bosh-google-cpi}

Deploy fails with:

Command 'deploy' failed:
  Parsing installation manifest ...:
    Validating installation manifest:
      cloud_provider.template.name must be provided

Invoked with:

omg gcp \
    --bosh-private-ip some-value \
    --gcp-network-name some-value \
    --gcp-subnetwork-name some-value \
    --gcp-default-zone some-value \
    --gcp-project some-value

omg deploy-product should take product name rather than plugin name

It seems like omg should use the friendly product name for the deploy-product option:

$ omg-linux deploy-product concourse -h
No help topic for 'concourse'

rather than the plugin name:

$ omg-linux deploy-product concourse-plugin-linux -h
NAME:
   omg-linux deploy-product concourse-plugin-linux - deploy the concourse-plugin-linux product

USAGE:
   omg-linux deploy-product concourse-plugin-linux [command options] [arguments...]

OPTIONS:
   --deployment-name value        deployment name (default: "concourse") [$OMG_DEPLOYMENT_NAME]
   --concourse-username value     concourse user id [$OMG_CONCOURSE_USERNAME]
   --concourse-password value     concourse password [$OMG_CONCOURSE_PASSWORD]
   --external-url value           URL to access concourse [$OMG_EXTERNAL_URL]
...

template is wrong for default nats properties bosh 256.2 

./omg-linux azure \
  --azure-vnet boshvnet-crp \
  --azure-subnet Bosh \
  --azure-subscription-id <Redacted> \
  --azure-tenant-id <Redacted>  \
  --azure-client-id <Redacted> \
  --azure-client-secret amazon4ever \
  --azure-resource-group hrcentral \
  --azure-storage-account <Redacted> \
  --azure-security-group nsg-bosh \
  --azure-ssh-pub-key-path bosh-privatekey.pub \
  --azure-ssh-user vcap \
  --azure-environment AzureCloud \
  --azure-private-key-path bosh-privatekey \
  --cpi-name cpi \
  --bosh-private-ip 10.0.0.5

Generates job properties:

nats:
  user:
  password:
  listen_address: 127.0.0.1

should be

nats:
  user:
  password:
  address: 127.0.0.1

How do I get Root CA of the bosh director?

  1. Seems I can not inject a custom root ca
  2. If the root ca is generated, how can I achieve it. Print manifest seems like a dry run. When next time deploy, the root ca is changed

As a user protect me from my stupidity

Error message thrown if I failed to register a plugin should tell me I failed to follow even the most basic instructions and am a dumbass instead of the current generic Nil message. Negative reinforcement is always more effective.

Need support for Azure CPI V14

Hi,

There were a breaking change in Azure CPI V14. The CPI job name was changed form cpi to azure_cpi.
You get the following error when deploying Bosh with CPI V14:

Command 'deploy' failed:
Invalid CPI release 'bosh-azure-cpi':
CPI release must contain specified job 'cpi'

Thanks,

Azure: blobstore missing in bosh yaml

More template issues from command:


./omg-linux azure \
  --azure-vnet boshvnet-crp \
  --azure-subnet Bosh \
  --azure-subscription-id <Redacted> \
  --azure-tenant-id <Redacted>  \
  --azure-client-id <Redacted> \
  --azure-client-secret amazon4ever \
  --azure-resource-group hrcentral \
  --azure-storage-account <Redacted> \
  --azure-security-group nsg-bosh \
  --azure-ssh-pub-key-path bosh-privatekey.pub \
  --azure-ssh-user vcap \
  --azure-environment AzureCloud \
  --azure-private-key-path bosh-privatekey \
  --cpi-name cpi \
  --bosh-private-ip 10.0.0.5 \
  --print-manifest


Uploading stemcell 'bosh-azure-hyperv-ubuntu-trusty-go_agent/3232.8'... Skipped [Stemcell already uploaded] (00:00:00)

Started deploying
  Waiting for the agent on VM '55h755votbjp2cfdefaultsa-e4416f6a-9a10-4ae0-6b03-749e145aa955'... Finished (00:00:00)
  Stopping jobs on instance 'unknown/0'... Finished (00:00:00)
  Unmounting disk 'bosh-data-55h755votbjp2cfdefaultsa-41580660-ef72-45ea-96bc-c3a2bf5acd81-None'... Finished (00:00:01)
  Deleting VM '55h755votbjp2cfdefaultsa-e4416f6a-9a10-4ae0-6b03-749e145aa955'... Finished (00:02:40)
  Creating VM for instance 'bosh/0' from stemcell 'bosh-stemcell-0a915ed0-638d-4171-949a-e4b6c719666a'... Finished (00:01:55)
  Waiting for the agent on VM '55h755votbjp2cfdefaultsa-df5639a3-edf1-45af-6d87-bcb5f5a08505' to be ready... Finished (00:00:21)
  Attaching disk 'bosh-data-55h755votbjp2cfdefaultsa-41580660-ef72-45ea-96bc-c3a2bf5acd81-None' to VM '55h755votbjp2cfdefaultsa-df5639a3-edf1-45af-6d87-bcb5f5a08505'... Finished (00:00:45)
  Rendering job templates... Failed (00:00:01)
Failed deploying (00:05:48)

Stopping registry... Finished (00:00:00)
Cleaning up rendered CPI jobs... Finished (00:00:00)

Command 'deploy' failed:
  Deploying:
    Building state for instance 'bosh/0':
      Rendering job templates for instance 'bosh/0':
        Rendering templates for job 'director/8b7977b57a31aea379d10ab04851d0ea57009b07':
          Rendering template src: director.yml.erb.erb, dst: config/director.yml.erb:
            Rendering template src: /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/bosh-init-release672283891/extracted_jobs/director/templates/director.yml.erb.erb, dst: /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/rendered-jobs839132171/config/director.yml.erb:
              Running ruby to render templates:
                Running command: 'ruby /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/erb-renderer602830570/erb-render.rb /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/erb-renderer602830570/erb-context.json /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/bosh-init-release672283891/extracted_jobs/director/templates/director.yml.erb.erb /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/rendered-jobs839132171/config/director.yml.erb', stdout: '', stderr: '/home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/erb-renderer602830570/erb-render.rb:180:in `rescue in render': Error filling in template '/home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/bosh-init-release672283891/extracted_jobs/director/templates/director.yml.erb.erb' for director/0 (line 140: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'blobstore.address'>) (RuntimeError)
    from /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/erb-renderer602830570/erb-render.rb:166:in `render'
    from /home/haydon/.bosh_init/installations/e641e384-5322-4095-4fba-6c5c53a35dbd/tmp/erb-renderer602830570/erb-render.rb:191:in `<main>'
':
                  exit status 1

photon username/password if not given creates an invalid manifest

if we do not add the username/password flags (which are not required) then it creates a manifest with empty values. This seems to have the cpi try to authenticate using empty string, rather than simply not authenticate.

we likely need to remove the values from the generated manifest completely if not given as flags.

Rolling an existing Microbosh into OMG.

After testing, the only creds that can't be 'autorolled' every time the microbosh is deployed is the agent-password.

Can we expose the agent-password out as a parameter please?

(Would be interested in pairing on this).

using --bosh-private-ip and no --bosh-public-ip means static ip is put into both fields. 

./omg-linux azure \
  --azure-vnet boshvnet-crp \
  --azure-subnet Bosh \
  --azure-subscription-id <Redacted> \
  --azure-tenant-id <Redacted>  \
  --azure-client-id <Redacted> \
  --azure-client-secret amazon4ever \
  --azure-resource-group hrcentral \
  --azure-storage-account <Redacted> \
  --azure-security-group nsg-bosh \
  --azure-ssh-pub-key-path bosh-privatekey.pub \
  --azure-ssh-user vcap \
  --azure-environment AzureCloud \
  --azure-private-key-path bosh-privatekey \
  --cpi-name cpi \
  --bosh-private-ip 10.0.0.5

output of the command results in a manifest with:

  - name: private
    static_ips:
    - 10.0.0.5
    default:
    - dns
    - gateway
  - name: public
    static_ips:
    - 10.0.0.5

This is incorrect. OMG should allow for a bosh deployed to a subnet that is not public (ie when using a jumpbox / bastion box to deploy bosh).

AWS cloud config plugin fails to create > 2 subnets

This example is before @calebwashburn's recent refactor. I have yet to figure out the flag changes to try and reproduce on 1.x.

Try to create 3 subnets:

omg-cli deploy-cloudconfig \
--bosh-url https://x.x.x.x \
--bosh-port 25555 \
--bosh-user director \
--bosh-pass `cat directorpwd` \
--ssl-ignore \
--print-manifest \
cloudconfig-plugin-linux \
--aws-region us-east-1 \
--aws-security-group PCF-Sec-Group \
--aws-az-name-1 us-east-1a \
--aws-az-name-2 us-east-1b \
--dns-1 x.x.x.x \
--cidr-1 x.x.x.x/x \
--gateway-1 x.x.x.x \
--bosh-az-name-1 az1a \
--aws-subnet-name-1 subnet-1 \
--bosh-reserve-range-1 "x.x.x.x-x.x.x.x" \
--bosh-static-range-1 "x.x.x.x-x.x.x.x" \
--dns-2 x.x.x.x \
--cidr-2 x.x.x.x/x \
--gateway-2 x.x.x.x \
--bosh-az-name-2 az1b \
--aws-subnet-name-2 subnet-2 \
--bosh-reserve-range-2 "x.x.x.x-x.x.x.x" \
--bosh-static-range-2 "x.x.x.x-x.x.x.x" \
--dns-3 x.x.x.x \
--cidr-3 x.x.x.x/x \
--gateway-3 x.x.x.x \
--bosh-az-name-3 az2a \
--aws-subnet-name-3 subnet-3 \
--bosh-reserve-range-3 "x.x.x.x-x.x.x.x" \
--bosh-static-range-3 "x.x.x.x-x.x.x.x"

Produces a cloud config with only 2 subnets:

azs:
- name: az1a
  cloud_properties:
    availability_zone: us-east-1a
    security_groups:
    - PCF-Sec-Group
- name: az1b
  cloud_properties:
    availability_zone: us-east-1b
    security_groups:
    - PCF-Sec-Group
networks:
- name: private
  type: manual
  subnets:
  - range: x.x.x.x/x
    gateway: x.x.x.x
    dns:
    - x.x.x.x
    reserved:
    - x.x.x.x-x.x.x.x
    static:
    - x.x.x.x-x.x.x.x
    az: az1a
    cloud_properties:
      subnet: subnet-1
  - range: x.x.x.x/x
    gateway: x.x.x.x
    dns:
    - x.x.x.x
    reserved:
    - x.x.x.x-x.x.x.x
    static:
    - x.x.x.x-x.x.x.x
    az: az1b
    cloud_properties:
      subnet: subnet-2
- name: vip
  type: vip

validation of parameters

It'd be really nice if the CLI performed validation on two use cases (fast feedback to user):

  1. if you specify a flag, but don't specify a parameter.
  2. if you repeat parameters (especially likely in creating a complex cloud config)

Exit with clear message when Vault is sealed

When using Vault, if the vault status is sealed, or unauthenticated, the current omg version just panics leaving no clear message behind. Requesting a chore to exit with a clear message and a non-zero exit status code to incorporate in scripts and pipelines. Thanks!

Latest Release v0.0.29 gives panic

Panic if there are no params.

./omg-linux azure
**********************************
Director PWD: qd3w6gyrfpd3d3asppat
**********************************
2016/08/18 14:57:06 C0818 14:57:06.916818 35990 utils.go:105] Sorry you need to provide [azure-vnet azure-subnet azure-subscription-id azure-tenant-id azure-client-id azure-client-secret azure-resource-group azure-storage-account azure-security-group azure-ssh-pub-key-path azure-ssh-user azure-private-key-path] flags to continue
panic: Sorry you need to provide [azure-vnet azure-subnet azure-subscription-id azure-tenant-id azure-client-id azure-client-secret azure-resource-group azure-storage-account azure-security-group azure-ssh-pub-key-path azure-ssh-user azure-private-key-path] flags to continue [recovered]
    panic: Sorry you need to provide [azure-vnet azure-subnet azure-subscription-id azure-tenant-id azure-client-id azure-client-secret azure-resource-group azure-storage-account azure-security-group azure-ssh-pub-key-path azure-ssh-user azure-private-key-path] flags to continue

goroutine 1 [running]:
panic(0x9369e0, 0xc42012d940)
    /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli.HandleAction.func1(0xc420165790)
    /go/src/github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli/app.go:476 +0x29e
panic(0x9369e0, 0xc42012d940)
    /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/enaml-ops/omg-cli/vendor/github.com/op/go-logging.(*Logger).Panicf(0xc4200129f0, 0xa20f7c, 0x2e, 0xc42012d710, 0x1, 0x1)
    /go/src/github.com/enaml-ops/omg-cli/vendor/github.com/op/go-logging/logger.go:194 +0x127
github.com/enaml-ops/omg-cli/utils.CheckRequired(0xc42001da40, 0xc420165108, 0xc, 0xc)
    /go/src/github.com/enaml-ops/omg-cli/utils/utils.go:105 +0x274
github.com/enaml-ops/omg-cli/azure-cli.GetAction.func1(0xc42001da40, 0x0, 0x0)
    /go/src/github.com/enaml-ops/omg-cli/azure-cli/azure-cli.go:53 +0xd1
reflect.Value.call(0x947aa0, 0xc42012c780, 0x13, 0xa0333f, 0x4, 0xc420165730, 0x1, 0x1, 0x48caf8, 0x9f6ee0, ...)
    /usr/local/go/src/reflect/value.go:434 +0x5c8
reflect.Value.Call(0x947aa0, 0xc42012c780, 0x13, 0xc420165730, 0x1, 0x1, 0x0, 0x138, 0x138)
    /usr/local/go/src/reflect/value.go:302 +0xa4
github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli.HandleAction(0x947aa0, 0xc42012c780, 0xc42001da40, 0x0, 0x0)
    /go/src/github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli/app.go:485 +0x1e0
github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli.Command.Run(0xa04023, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0xa1da8e, 0x28, 0x0, ...)
    /go/src/github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli/command.go:186 +0xc2b
github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli.(*App).Run(0xc42014a000, 0xc42000c240, 0x2, 0x2, 0x0, 0x0)
    /go/src/github.com/enaml-ops/omg-cli/vendor/github.com/codegangsta/cli/app.go:239 +0x611
main.main()
    /go/src/github.com/enaml-ops/omg-cli/main.go:154 +0x84e

Is it posible to write universal plugin that would work with any PCF tile?

Now when you work with omg you need to create separate plugin for each product. This includes a lot of extra work, but the biggest issue with this approach is that it is hard to maintain all plugins up to date: if new properties were added to manifest in a new version of the product you need to modify plugin itself.

My idea is to write a plugin that would extract any PCF tile, and then analyze it manifest. Actually all information that is needed to build a bOSH manifest already exists in PCF tile manifest, it should be just a matter of yml manipulation. We also can extract parameters from the PCF tile manifest and their default values and convert them to command line parameters.

Errors when trying to deploy concourse.

Hi,
I'm getting unhelpful errors when I try and deploy concourse as per below.

enamltest $ omg deploy-product \
> --bosh-url https://10.0.16.6 --bosh-port 25555 --bosh-user admin \
> --bosh-pass `cat director.pwd` --ssl-ignore \
> concourse-plugin-linux \
> --external-url https://concourse.REDACT \
> --web-vm-type small \
> --worker-vm-type small \
> --database-vm-type small \
> --network-name private \
> --concourse-db-pwd secret \
> --database-storage-type medium
Checking product deployment for remote assets...
Checking for remote stemcells...
remote stemcells complete
Checking for remote releases
polling task...
task state done [ | ] Finished with Task create release
polling task...
task state done [ / ] Finished with Task create release
remote releases complete
remote asset check complete.
Uploading product deployment...
upload complete
polling task...
2016/11/10 22:02:21 E1110 22:02:21.482649 5608 bosh.go:268] task error: error - create deployment
Finished with Task create deployment
2016/11/10 22:02:21 E1110 22:02:21.482695 5608 bosh.go:109] there was an error:  error - create deployment
error - create deployment

Current default bosh azure cpi + stemcell violates the release notes requirements and does not deploy 

./omg-linux azure \
  --azure-vnet boshvnet-crp \
  --azure-subnet Bosh \
  --azure-subscription-id <Redacted> \
  --azure-tenant-id <Redacted>  \
  --azure-client-id <Redacted> \
  --azure-client-secret amazon4ever \
  --azure-resource-group hrcentral \
  --azure-storage-account <Redacted> \
  --azure-security-group nsg-bosh \
  --azure-ssh-pub-key-path bosh-privatekey.pub \
  --azure-ssh-user vcap \
  --azure-environment AzureCloud \
  --azure-private-key-path bosh-privatekey \
  --cpi-name cpi \
  --bosh-private-ip 10.0.0.5 \
  --print-manifest

then remove the public IP address attempts to deploy cpi v11 and azure stemcell 3169 which is < 3181 as per below:

Fixes: - Use LUN and host device id as the disk identifier - Compatible Stemcell Versions: v3181 or later - Stemcell v3232.5 or later is recommended with CPI v11. You may hit the issue #135 if you still use an older stemcell than v3232.5 - Use a data disk as the persistent ephemeral disk - The VM size should not be Standard_A0 if you want to use persistent data disks - The CPU cores of VM should be more than 1 if you plan to resize the persistent data disks in future - The premium storage is recommended for better performance - You can see more information about the ‘Max data disks’ and ‘Max disk IOPS & bandwidth’ of Azure Linux VMs here - Fix an issue in calculating the sleep interval when copying blobs
http://bosh.io/releases/github.com/cloudfoundry-incubator/bosh-azure-cpi-release?version=11

bosh-init plugin should not require a public IP

The bosh-init plugin is hard coded to always create a public network. If you don't specify --bosh-public-ip you will get a manifest that can't be deployed.

If the --bosh-public-ip flag is not provided then we should not create the public network.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.