cloudfoundry / smb-volume-release Goto Github PK

View Code? Open in Web Editor NEW

8.0 19.0 10.0 10.99 MB

BOSH release with SMB driver and broker

License: Apache License 2.0

Shell 44.32% Go 27.69% HTML 4.04% Dockerfile 2.14% Ruby 21.62% Makefile 0.20%

cff-wg-service-management

smb-volume-release's Introduction

SMB volume release

This is a bosh release that packages:

an smbdriver
an smbbroker for preexisting SMB shares
a test SMB server that provides a preexisting share to test against

The broker and driver pair allows you:

to provision preexisting shares and bind the shares to your applications for share file access.

The test server provides an easy test target with which you can try out volume mounts of preexisting shares.

Deploying to Cloud Foundry

Pre-requisites

Install Cloud Foundry, or start from an existing CF deployment. If you are starting from scratch, the article Overview of Deploying Cloud Foundry provides detailed instructions.

Redeploy Cloud Foundry with SMB enabled

You should have it already after deploying Cloud Foundry, but if not clone the cf-deployment repository from git:

$ cd ~/workspace
$ git clone https://github.com/cloudfoundry/cf-deployment.git
$ cd ~/workspace/cf-deployment

Now redeploy your cf-deployment while including the smb ops file:

 $ bosh -e my-env -d cf deploy cf.yml -v deployment-vars.yml -o operations/experimental/enable-smb-volume-service.yml

Note: the above command is an example, but your deployment command should match the one you used to deploy Cloud Foundry initially, with the addition of a -o operations/experimental/enable-smb-volume-service.yml option.

Your CF deployment will now have a running service broker and volume drivers, ready to mount or create SMB volumes.
Unless you have explicitly defined a variable for your broker password, BOSH will generate one for you.

Testing or Using this Release

Deploying the Test SMB Server (Optional)

If you do not have an existing SMB Server then you can optionally deploy the test SMB server bundled in this release.

The easiest way to deploy the test server is to include the enable-smb-test-server.yml operations file when you deploy Cloud Foundry, also specifying smb-username and smb-password variables:

$ bosh -e my-env -d cf deploy cf.yml -v deployment-vars.yml \
  -v smb-username=smbuser \
  -v smb-password=something-secret \
  -o operations/experimental/enable-smb-volume-service.yml \
  -o operations/test/enable-smb-test-server.yml

NOTE: This test SMB server only works with Ubuntu stemcells.

Register smbbroker

Deploy and register the broker and grant access to its service with the following command:
```
$ bosh -e my-env -d cf run-errand smb-broker-registrar
$ cf enable-service-access smb
```

Testing and General Usage with smbbroker

You can refer to the Cloud Foundry docs for testing and general usage information.

Follow the cf docs to deploy and test a sample app

Test instructions are here The smbbroker uses credhub as a backing store, and as a result, does not require separate scripts for backup and restore, since credhub itself will get backed up by BBR.

Troubleshooting

If you have trouble getting this release to operate properly, try consulting the Volume Services Troubleshooting Page

smb-volume-release's People

Contributors

Stargazers

Watchers

Forkers

hdya lancefrench alamages sneal gmrodgers isabella232 efejjota dsboulder nouseforaname davewalter

smb-volume-release's Issues

Submit smb-volume-release to bosh.io

So it is easy to download release tarballs.

`smbbrokerpush` uses a `CF_HOME` dir that collides with other errands

Refer to similar issue on NFS here.

Errands may fail with the following error upon subsequent deploys when CF_HOME is set to the same directory as another errand:

           cf version 6.41.0+dd4c76cdd.2018-11-28  
           API endpoint: https://api.sys.maraudon.cf-app.com  
             
             
           API endpoint:   https://api.sys.maraudon.cf-app.com (API version: 2.128.0)  
           Not logged in. Use 'cf login' to log in.  
           FAILED  
           Service account currently logged in. Use 'cf logout' to log out service account and try again.

Steps to reproduce:

Run smbbrokerpush errand followed by nfsbrokerpush (this should fail with the above error)

We're feeling pretty confident that this is the root case:

The NFS errand sets CF_HOME to /var/vcap/bosh/home/cf/.cf here as does the SMB errand (we'll open a separate issue on those repos)
If you login as a CF UAA client, then login as a CF UAA user, the CF CLI will throw the above error
We configure the NFS errand with a user, but the SMB errand with a client which causes the NFS errand will fail

Workaround is to either disable one of the errands or SSH onto the co-located errand VM and run rm -rf /var/vcap/bosh/home/cf/.cf.

The way other components avoid this is to create a CF_HOME dir that is unique to their release, e.g. export CF_HOME=/var/vcap/data/push-apps-manager/. Could y'all do something similar with SMB errand?

Get `permission denied error` while writing to the share drive

Hi:

We've deployed smb-volume-release with the latest CF release and the deploy works, and we've also deployed SMB test server so we can run some test.

After pushing pora and try to write a file into the shared mount, we get an permission denied error.

After some troubleshooting, we noticed we were using bind options like {"uid":"1000","gid":"1000","username":"smbuser","password":"<redacted>"},

it used a different uid/gid from what is used by smbdriver. the default uid/gid used by smbdriver is 2000.

after changing the bind options to below fixed the issue.
{"uid":"2000","gid":"2000","username":"smbuser","password":"<redacted>"},

so our questions are: Is that behavior expected? why do we want uid/gid to be configured by the user? would it be easier just removing this option?

Regard
Kai

Test Tracker integration removal

Generate the smb tile to use xenial stemcell

SMB3 Encryption support

Does this tile support Encryption in SMB3?

Add support for cf CLI v8

cf CLI v7 support was added 2 years ago.

Now v8 is available and the relevant errands should be updated to accept that version of the cf CLI.

Support Bionic stemcell

Describe the bug

Hi all,

we tried to compile the smb-volume release for Ubuntu Bionic and ran into the following error:

+ make
help2man: can't get `--help' info from automake-1.15
Try `--no-discard-stderr' if option outputs to stderr
make: *** [doc/automake-1.15.1] Error 255

A possible explanation could be that automake-1.15 from the cifs-utils package no longer works with newer versions of Perl which are consumed from Bionic (see issue: crosstool-ng/crosstool-ng#987 (comment)).

To Reproduce
Steps to reproduce the behavior:

bosh -e bosh export-release -d compilation-workspace smb-volume/3.1.0 ubuntu-bionic/0.18

With deployed manifest:

name: compilation-workspace

releases:
- name: smb-volume
  version: "3.1.0"

stemcells:
- alias: default
  os: ubuntu-bionic
  version: latest

instance_groups: []

update:
  canaries: 1
  max_in_flight: 1
  canary_watch_time: 1000-90000
  update_watch_time: 1000-90000

Expected behavior
Successful compilation and export of tgz packages.

Version of smb-volume-release
3.1.0

Full error log:

Task XXXX | 14:03:06 | Compiling packages: cifs-utils/2b5f90c43280309ef547f546893b29386d66c4b4ac3387a6e65aad6c696bade5 (00:01:42)
                      L Error: Action Failed get_task: Task 30846a25-06d1-4396-538f-1da9cee3c248 result: Compiling package cifs-utils: Running packaging script: Running packaging script: Command exited with 2; Truncated stdout:  /usr/bin/install -c -m 644 autoconf.m4 general.m4 status.m4 oldnames.m4 specific.m4 autoheader.m4 autoupdate.m4 autotest.m4 autoscan.m4 lang.m4 c.m4 erlang.m4 fortran.m4 functions.m4 go.m4 headers.m4 types.m4 libs.m4 programs.m4 '/var/vcap/data/compile/cifs-utils/temp/share/autoconf/autoconf'
[...]
Makefile:3687: recipe for target 'doc/automake-1.15.1' failed
, Stderr: + source /var/vcap/packages/python-2.7/bosh/compile.env
++ export PATH=/var/vcap/packages/python-2.7/bin:/var/vcap/bosh/bin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin
++ PATH=/var/vcap/packages/python-2.7/bin:/var/vcap/bosh/bin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin
++ export PYTHONPATH=/var/vcap/packages/python-2.7/lib/python2.7/site-packages:
++ PYTHONPATH=/var/vcap/packages/python-2.7/lib/python2.7/site-packages:
+ temp_path=/var/vcap/data/compile/cifs-utils/temp
+ mkdir /var/vcap/data/compile/cifs-utils/temp
+ tar xf autoconf-2.69.tar.gz
+ pushd autoconf-2.69
+ ./configure --prefix=/var/vcap/data/compile/cifs-utils/temp
+ make
+ make install
+ export PATH=/var/vcap/packages/python-2.7/bin:/var/vcap/bosh/bin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/var/vcap/data/compile/cifs-utils/temp/bin
+ PATH=/var/vcap/packages/python-2.7/bin:/var/vcap/bosh/bin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/var/vcap/data/compile/cifs-utils/temp/bin
+ popd
+ tar xf automake-1.15.tar.gz
+ pushd automake-1.15
+ ./configure --prefix=/var/vcap/data/compile/cifs-utils/temp
configure: WARNING: Fortran compiler cannot create executables
configure: WARNING: Fortran 77 compiler cannot create executables
+ make
help2man: can't get `--help' info from automake-1.15
Try `--no-discard-stderr' if option outputs to stderr
make: *** [doc/automake-1.15.1] Error 255

Thanks!
cc @StefanWutz

Release 0.1.6 fails to export to linux

If I try to export the smb-volume-release 0.1.6 to a linux stemcell I get the error below. I'm not sure how other releases that have windows packages mixed in solve this.

Perhaps the windows package's packaging script can skip if exporting to a linux stemcell since the linxu stemcell won't be using the windows job anyway?

Task 107619 | 21:20:20 | Compiling packages: golang-smbvolume-windows/a979fc2219d6964af5f0ae38f1e9cd1df75e5325 (00:01:07)
                      L Error: Action Failed get_task: Task 22c10af0-a5e0-44bd-4473-633707b871d8 result: Compiling package golang-smbvolume-windows: Running packaging script: Running packaging script: Command exited with 2; Stdout: , Stderr: packaging: line 1: =: command not found
packaging: line 2: syntax error near unexpected token `('
packaging: line 2: `trap { $host.SetShouldExit(1) }'

Please configure GITBOT

Pivotal uses GITBOT to synchronize Github issues and pull requests with Pivotal Tracker.
Please add your new repo to the GITBOT config-production.yml in the Gitbot configuration repo.
If you don't have access you can send an ask ticket to the CF admins. We prefer teams to submit their changes via a pull request.

Steps:

Fork this repo: cfgitbot-config
Add your project to config-production.yml file
Submit a PR

If there are any questions, please reach out to [email protected].

Cosmetic issues w/marketplace and space name

The broker release has some minor cosmetic issues needing cleanup, specifically:

the system space created has a typo "smb-voloume-service-space"
no marketplace icon image within Apps Manager.

Would also suggest leaving the "Default Location" field under the Service Broker Application pane blank (and mandatory). Currently the tile defaults to "useast" which most users overlook and often wish to override (in some circumstances this also causes issue with no Gen4 hardware support in USEast).