cloudfoundry / service-fabrik-boshrelease Goto Github PK

Service-fabrik deployment fails with the provided template, with the error that the broker and scheduler jobs are unable to come up.

Reason:
The virtual-host provider config is currently being sent as empty from the template. But, the service-fabrik-broker expects a provider object and a name attribute within that object.
https://github.com/cloudfoundry-incubator/service-fabrik-broker/blob/master/lib/iaas/index.js#L34

Hence, this throws as an error as provider comes as null here.

Fix:
We need to pass virtual-host provider params in the deployment file of our template.
Same as: https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/blob/master/templates/deployment.yml#L147-L157

After the service-fabrik-broker's manifest has migrated to v2, the blueprint manifest template also needs to be migrated to v2 format. Else, the service-instance creation will fail with the current changes and consuming the existing blueprint manifest.

Description
Unable to deploy service fabrik with the latest changes made on the bosh director. Following are the issues:

1. The generated manifest talks about two bosh url one pointing out to 192.168.50.4 and other to 192.168.50.3. However there are no instructions on to deploy 2 Bosh.
   https://github.com/SAP/service-fabrik-boshrelease/blob/master/templates/properties.yml#L222
2. The https://github.com/SAP/service-fabrik-boshrelease/blob/master/jobs/service-fabrik-broker/templates/config/circuit-breaker-config.yml.erb#L16 has reference to bootstrap_bosh_director which does not exist in manifest
3. Similarly like #2 in this file https://github.com/SAP/service-fabrik-boshrelease/blob/master/jobs/service-fabrik-scheduler/templates/config/circuit-breaker-config.yml.erb

Steps to reproduce the issue:

1. Follow the steps in README and you will hit the erro on bosh deploy

Describe the results you received:
Unable to deploy service fabrik broker

Describe the results you expected:
Be able to deploy service fabrik broker

Output of bosh -v:

BOSH 1.3262.26.0

Output of cat /etc/*-release:

Ubuntu 14.04.5 LTS

Additional environment details (AWS, VirtualBox, OpenStack, etc.):
Bosh-Lite on VirtualBox

Mongodb:

If we set both mongodb.plan_id and mongodb.url to undefined, the broker does not need the mongodb and just prints out a warning, which is fine. However in the bosh release the broker consumes the scheduler.
The scheduler seems to require the mongodb and can’t be disabled, because it provides the bosh link, which is required by the broker.

The public URL (NETWORK_PUBLIC_URL) detection is buggy in the https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/blob/7ff95cc2c7880105aa44edd2249a819a05945981/jobs/keepalived/templates/bin/utils/openstack/generate_token#L47..L48

In case, when the endpoint URL is https://network.my.cloud (no trailing slash), the port_operation script fails: https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/blob/7ff95cc2c7880105aa44edd2249a819a05945981/jobs/keepalived/templates/bin/utils/openstack/port_operations#L36

because it generates the https://network.my.cloudv2.0/ports url, which is not valid.

https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/blob/master/jobs/service-fabrik-broker/templates/config/settings.yml.erb#L118

https://github.com/cloudfoundry-incubator/service-fabrik-boshrelease/blob/master/jobs/service-fabrik-scheduler/templates/config/settings.yml.erb#L119

In both the above configs , the value should be fetched from jobs.service_instance_update.should_start_after_hr .

For the dev scenario provided in the template, we generate services.yml from the '.rb' files of various services (redis/blueprint).
For a deployment made on AWS, the instance provisioned using this manifest shows a wrong dashboard url.
Fix:
The dashboard url always contains the bosh-lite's cf domain in the url. This should be modified to the AWS's cf domain (this could be provisioned through vars file) and inserted into the '.rb' files of the services before generating the services.yml file. This operation could be taken care of in the render script.

Currently schedule update operation is not configured in event log config

Virtual host manager should use a seperate IaaS container to store relationship
between virtual_host instances and deployment name.
This issue compliments Issue: cloudfoundry/service-fabrik-broker#122

NodeJS - December 2019 Security Releases stated npm CLI is vulnerable to Binary Overwrite of Global node modules.

As per above Security Release News, it is recommended that the project should be updated with nodejs v12.14.0 patch.

service-fabrik-boshrelease/packages/node/spec shows nodeJS v12.13.1 may have the security loop hole/s.

Could you please verify it? Thanks

regards,
Jayesh Thamke, Steampunk1, DE

Hi,

I was interested in your solution, including the usage of the ServiceManager and the rest of the architecture that you have, but nothing works.

It seems that your documentation is not updated, as well as your templates, because many options presented there didn't even exist in the bosh.io/releases usage of the service-fabrik as well as in the spec of the jobs.

It was necessary to read your code many times to try to deploy into bosh but I think I have reach a dead end, with many options and external services that seems to be needed and not even provided or specified.

Is there any good documentation with examples that you can provide for the usage of this solution?.

I tried to install it in both, Cloudfoundry and also in Kubernetes and for both cases it failed. For kubernetes (minikube as you specify in your documentation for service-fabrik-broker), some instructions of helm didn't work, because the repos didn't even exist. Also some default configurations didn't work or seems that they where completely outdated.

If this solution is yet available and published in bosh.io/releases, I would like to have some support from your side to make a POC of this, if not please let me know.

Thanks

David

During final release call we were getting error.

Building packages
-----------------
Building golang...
  Using final version '95c358c1eeabe9495e089de333e0c91eee2fef0c'
  Uploading final version '95c358c1eeabe9495e089de333e0c91eee2fef0c'...
  Uploaded, blobstore id '5924ebab-6dec-411b-86c5-8491e94064fc'

Building bosh-helpers...
  Using final version 'e1bbff85fa1e4b1d51887ba6ecff6949ca92e8f6'
  Uploading final version 'e1bbff85fa1e4b1d51887ba6ecff6949ca92e8f6'...
  Uploaded, blobstore id 'ad205f16-f98a-4045-a609-74e9b341c3a9'

Building docker...
  Using final version '45062aefa4be8e9f0a2af53e5591e04be5a8db0b'
  Uploading final version '45062aefa4be8e9f0a2af53e5591e04be5a8db0b'...
  Uploaded, blobstore id 'f932657e-3a77-414d-b90d-fe2c2a42f138'

Building service-fabrik-broker...
  No artifact found for service-fabrik-broker
  Generating...
  Pre-packaging...
  Generated version 'aec6a7837e3d7b2995506145cbeada2c4134ca8f'
  Uploading final version 'aec6a7837e3d7b2995506145cbeada2c4134ca8f'...
Blobstore error: Failed to create object, S3 response error code NoSuchKey: The specified key does not exist.

We were using bosh cli v1 (BOSH 1.3262.26.0) to create final release.

The issue is solved after separating bucket name and folder parameter in config/final.yml

Hello,

I would like to use extend the service fabrik functionality, but currently it is quite hard to understand how it works. Could you provide some comments in the code about the basic functionality of the functions? JS-Doc seems like the standard in node.(I am new to node). Especially the calculation of inside of the NetworkSegmentIndex.js are quite hard to understand.

When trying to create the release it fails with an error about accessing the blobstore. It does not seem as if the S3 (service-fabrik-boshrelease) blobstore has been granted public get access.

ubuntu@ip-10-100-4-35:/mnt/workspace/boshreleases/service-fabrik-boshrelease-master$ bosh create release --force
Syncing blobs...
golang/go1.6.2.linux-amd64.tar.gz downloading 80.9M (0%)
/var/lib/gems/2.3.0/gems/blobstore_client-1.3262.24.0/lib/blobstore_client/simple_blobstore_client.rb:44:in `get_file': Could not fetch object, 403/ (Bosh::Blobstore::BlobstoreError)

Thanks,
Clinton

Since you're using custom scripts to migrate the VIP, they don't use keepalived features to advertise new MAC addresses.

I investigated the keepalived configuration and action.sh script. and noticed that it is not necessary to use attach_vip_to_interface and detach_vip_from_interface shell funcs, but it is necessary to configure the virtual_ipaddress inside the keepalived configuration and keepalived will take care about the rest.

• service-fabrik-broker job is dependent on swarm-manager.
• In service instance dashboard, Pre defined key-value pairs are present inside dashboard. Currently no support available, If a service owner wants to add some fields specific to the service.