cf-platform-eng / tile-generator Goto Github PK

In the Prerequisite , add bosh-cli

Should contain requirements.txt

➜ sample git:(master) ✗ tile build
name: test-tile
icon: resources/icon.png
label: Generated Test Tile
description: A sample tile generated by tile-generator
version: 0.0.1

bosh init release
bosh generate package app1
bosh generate job deploy-app-app1
bosh generate job delete-app-app1
bosh generate package app2
bosh generate job deploy-app-app2
bosh generate job delete-app-app2
bosh generate package broker1
bosh generate job deploy-app-broker1
bosh generate job delete-app-broker1
bosh generate job register-broker-broker1
bosh generate job destroy-broker-broker1
bosh generate package broker2
bosh generate job deploy-app-broker2
bosh generate job delete-app-broker2
bosh generate job register-broker-broker2
bosh generate job destroy-broker-broker2
bosh generate package broker3
bosh generate job register-broker-broker3
bosh generate job destroy-broker-broker3
bosh generate package buildpack1
bosh generate job deploy-buildpack-buildpack1
bosh generate job delete-buildpack-buildpack1
bosh generate package buildpack2
bosh generate job deploy-buildpack-buildpack2
bosh generate job delete-buildpack-buildpack2
bosh generate job docker-bosh-docker-bosh1
bosh generate job deploy-app-docker-app
bosh generate job delete-app-docker-app
bosh generate job deploy-app-docker-app-broker
bosh generate job delete-app-docker-app-broker
bosh generate job register-broker-docker-app-broker
bosh generate job destroy-broker-docker-app-broker
bosh generate package blob1
bosh generate package cf_cli
bosh upload blobs
bosh create release --force --final --with-tarball --version 0.0.1

tile import release test-tile
tile import release docker
tile build docker release
Traceback (most recent call last):
File "/home/df/workspace/tile-generator/tile", line 76, in
cli()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in call
return self.main(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, *_ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(_args, *_kwargs)
File "/home/df/workspace/tile-generator/tile", line 46, in build_cmd
build.build(config, verbose)
File "/home/df/workspace/tile-generator/lib/build.py", line 24, in build
create_tile(context)
File "/home/df/workspace/tile-generator/lib/build.py", line 217, in create_tile
docker_release = build_docker_release()
File "/home/df/workspace/tile-generator/lib/build.py", line 241, in build_docker_release
bash('./update')
File "/home/df/workspace/tile-generator/lib/build.py", line 275, in bash
return subprocess.check_output(argv, stderr=subprocess.STDOUT)
File "/usr/lib/python2.7/subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, *popenargs, *_kwargs)
File "/usr/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Properties are not declared before being used on the form

The top level README.md does not indicate how to build the included sample tile. Perhaps a section should be added there? I am guessing the steps would be:

1. cd sample
2. ./src/build.sh
3. tile build

Maybe optionally

This eventually results in exceeding the memory limit.

cf apps
Getting apps in org test-tile-org / space test-tile-space as admin...
OK

name                     requested state   instances   memory   disk   urls   
app1-0.3.57              started           1/1         256M     1G     app1.cfapps-03.haas-26.pez.pivotal.io   
app1-0.3.58              stopped           0/1         256M     1G     app1.cfapps-03.haas-26.pez.pivotal.io   
app2-0.3.57              started           1/1         256M     1G     app2-hostname.cfapps-03.haas-26.pez.pivotal.io   
app3-0.3.54              started           1/1         256M     1G     app3.cfapps-03.haas-26.pez.pivotal.io   
app3-0.3.57              stopped           0/1         256M     1G     app3.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.47   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.50   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.51   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.52   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.53   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.54   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker1-0.3.57   started           1/1         256M     1G     service-broker1.cfapps-03.haas-26.pez.pivotal.io   
service-broker2-0.3.47   started           1/1         256M     1G     service-broker2.cfapps-03.haas-26.pez.pivotal.io   
service-broker2-0.3.50   started           1/1         256M     1G     service-broker2.cfapps-03.haas-26.pez.pivotal.io   
service-broker2-0.3.52   started           1/1         256M     1G     service-broker2.cfapps-03.haas-26.pez.pivotal.io   
service-broker2-0.3.53   started           1/1         256M     1G     service-broker2.cfapps-03.haas-26.pez.pivotal.io   
service-broker2-0.3.54   started           1/1         256M     1G     service-broker2.cfapps-03.haas-26.pez.pivotal.io   
service-broker2-0.3.57   started           1/1         256M     1G     service-broker2.cfapps-03.haas-26.pez.pivotal.io   

When trying to add/upgrade a new version of a tile, clicking the add/upgrade button on the package more then once some times result in a 500 server error.

this is due to trying to launch the add/upgrade job twice before the button is disabled.

docker-bosh and blob

If the tile name has spaces in the tile.yml, it breaks. Maybe you want to replace the name with underscore or dash in the bosh build

Scenario :

1. use tile generator to generate a tile.yml file
2. edit your tile.yml file
3. build a tile
4. import the tile to ops manager
5. deploy the tile (in my case, deployment of service broker failed on "cf create-service-broker" errand)

• at this point you should have a running instance of your tile service in CF.
• change the package name in your tile.yml file
• rebuild your tile
• import the new tile to ops manager
• upgrade your tile to the new version
• deploy changes
  (at this point the deployment still failed in the "cf create-service-broker" phase)
• at this point you should have 2 running apps (service brokers) in CF - the old artifacts are not being cleaned.

The default forms for each app seem great for debugging and development, but many of those options probably should not be given to actual end-users of the production tile. We need a way to turn these on for development, and migrate only the required ones to custom forms for production.

The tile.yml from both the sample and the tile init command have a global defaults section which, for my PCF (v1.6.9.0) at least, are not optional as indicated.

# Global defaults (all optional)
#
# org: test-org                         # Name of org to create for your apps
# org_quota: 4096                       # Memory quota for that org in MB
# space: test-space                     # Name of space to create for your apps
apply_open_security_group: true         # Apply open security group, default: false

Recipe:

• leave these commented out as above
• tile build
• import the .pivotal to Ops Manager
• Add>> the product
• Click the new (orange) tile
• Fill in required settings as indicated by orange circles and save
• Observe the tile has turned green

At this point, the "Apply Changes" button remains disabled with tool tip: "You must configure one or more PCF products before installing" and cannot be pressed. There is no indication by orange circles etc what else needs to be configured.

Expected:

The YML should either be rejected at some point in the process with a useful message, or it should apply.

Workaround:

Uncommenting the global defaults and repeating the above will allow an apply.

I can attach an offending .pivotal if that would be helpful.

Expected: List of missing properties

Actually got: An attributeerror instead - see partial error trace below

Partial Error trace:
[2/23/16, 12:26:14 PM] Hans Hrasna: empty properties error pcf configure tilename emply_properties_file.yml
[2/23/16, 12:26:15 PM] Hans Hrasna: AttributeError: 'NoneType' object has no attribute 'iteritems'

Need one more pass through the loop after the status changes to success/failed

happened in Nurego-tile ver 9-11 (uploaded to slot 3 pcf)
trying to untar the cf-cli/cf-linux-amd64.tgz resulted in "is not a valid tar file" error.
deleting all but the tile.yml and rebuild seemed to solve the problem, while just doing a rebuild seemed to result in more corrupt tiles

Installation API now requires list of errands to be passed in

✗ tile build 0.1.0  
✗ more tile-history.yml 

---
history:
- 0.0.1
version: !!python/unicode '0.1.0'

causes the next build to fail

It seems to fail for most people other than me.

For instance:

pez-03 pcf import sample/product/test-tile-0.3.72.pivotal
Traceback (most recent call last):
  File "/Users/gwestenberg/git/tile-generator/bin/pcf", line 200, in <module>
    cli()
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/gwestenberg/git/tile-generator/bin/pcf", line 77, in import_cmd
    opsmgr.upload('/api/products', zipfile)
  File "/Users/gwestenberg/git/tile-generator/bin/../lib/opsmgr.py", line 92, in upload
    files = { 'product[file]': open(filename, 'rb') }
IOError: [Errno 2] No such file or directory: u'sample/product/test-tile-0.3.72.pivotal'

I created a bosh_docker tile from a working docker image. During deploy, the job fails to start.

Capturing the syslog from the job vm provides little information about reasons for the failure. See syslog snippet: https://gist.github.com/mdcarlson/d8b18888752965dd18b7#file-syslog

The bosh task debug doesn't offer much to go on either

Error during update deployment for 'pact-broker-test-tile-d446fcff8ec8206e53ab' against Director 'd8e0378c-065f-4dce-ba9c-a895f5acb319': #<Bosh::Director::AgentJobNotRunning: docker-bosh-pact_broker_docker_bosh_app-partition-3703fe24e2d17f8c8e32/0' is not running after update>","created_at":1454627669}`

Can you recommend a location of additional log files or a means of turning up verbosity that would enable further troubleshooting into reasons why the containerized app is not starting when deployed as a bosh release? I suspect issues with start command or inability to find/see environment variables, but don't see anything pointing to that in the logs.

The docker daemon appears to be running in the job's VM, but I get "Cannot connect to docker daemon" when attempting to run docker commands from var/vcap/packages/docker/bin whether logged in as vcap or root. Is there a trick to accessing the Docker container of a bosh_docker job?

Thanks in advance!

CF push uses manifest.yml but with no value set for memory (uses default).

( applicable to tile with service brokers )
Scenario:

1. Create a tile.yml file using tile generator
2. name your package any name with hyphen(-) or under score (_)
3. build and deploy your tile using ops manager

• deployment fails to register service broker with error "invalid URL" since the url contain under score (_)

this is due to the fact that bosh deployment doesn't accept hyphens that are automatically changed to under scores, and since CF- cloud controller doesn't accept URLs with underscores.

Error Observed: The 'route xxxxxxx is already in use' error occurs even if tile.yaml has random-route:true defined under properties: manifest

Possible cause: --HOSTNAME is being passed as a parameter and the random-route property is being overridden each time

Current hack to fix this:
Add host: under properties:manifest in tile.yml

Partial Error Trace:
Using route edb_postgresql.cfapps-02.haas-26.pez.pivotal.io
Binding edb_postgresql.cfapps-02.haas-26.pez.pivotal.io to edb_postgresql-0.0.6...
FAILED
The route edb_postgresql.cfapps-02.haas-26.pez.pivotal.io is already in use.
TIP: Change the hostname with -n HOSTNAME or use --random-route to generate a new route and then push again.

[stderr]
None

{"type": "step_finished", "id": "errands.running.hans-edb-broker-7a61ed19cfd379ef1542.deploy-all"}
Exited with 1.

Large package will cause compilation to fail. We should base the ephemeral disk size of the compilation vm on the largest package size (max size * 4 seems reasonable)

Seen in a CI pipeline:

cf bind-service apigee-cf-service-broker-0.1.27 apigee_cf_service_broker-p-redis
  cf start apigee-cf-service-broker-0.1.27
  cf delete -f Getting
  cf create-service-broker apigee-cf-service-broker 11b645173a3bb81d c6****** https://apigee-cf-service-broker.cfapps-03.haas-26.pez.pivotal.io
  cf enable-service-access apigee-edge

Note the delete -f Getting. Since this is in a pipeline, there should not be any older versions. Is there something wrong with the grep for old versions?

If a service registers to CF marketplace and the service name contain space (" ") in it.
when that service is clicked on in the apps manager marketplace to view the plans a 500 error is returned.

I created a simple test tile from a working docker image. When I attempted to delete the tile from OpsMan, I received the following error.

/var/vcap/jobs/delete-app-pact_broker_docker_app/bin/run: line 38: syntax error near unexpected tokenif'`

I did not modify the generated delete-app script.

Happy to share additional details, if you'd tell me where to send them and what you need.