uclalibrary / packer-samvera Goto Github PK

It seems vagrant is using version string rather than timestamp to pull updated versions of the box. Our current versioning string (e.g., v0.3-centos-SNAPSHOT-1537418555) could cause problems if there is ever an Ubuntu version (I know we're not going to run Ubuntu at UCLA, but just generally speaking). A low priority fix is to update the version string format to: v0.3-SNAPSHOT-1537418555-centos. This allows the timestamp to sort before the OS.

This will be a personal time fix one evening. It's not a priority for UCLA.

Acceptance of this ticket will be when the automated builds are working again and are producing a nightly snapshot.

Hardy gets the following error. I'll need to install libnotify and configure it to test on my machine.

 default: Running: inline script
    default: /tmp/vagrant-shell: line 1: syntax error near unexpected token `)'
    default: /tmp/vagrant-shell: line 1: `notify-send --urgency=critical -t 20000 californica is up! Get back to work! :-)''

The VM uses the Ansible roles to provision so the fedora.yml what's in the californica repo doesn't quite match up. We need to parameterize the fedora.yml to take a customizable Tomcat context, while still using /rest as a default.

What's there:

vagrant@localhost:~/californica$ cat config/fedora.yml
development:
  user: fedoraAdmin
  password: fedoraAdmin
  url: http://127.0.0.1:<%= ENV['FCREPO_DEVELOPMENT_PORT'] || 8080 %>/rest
  base_path: /dev
test:
  user: fedoraAdmin
  password: fedoraAdmin
  url: http://127.0.0.1:<%= ENV['FCREPO_TEST_PORT'] || 8986 %>/rest
  base_path: /test
production:
  user: fedoraAdmin
  password: fedoraAdmin
  url: http://127.0.0.1:8080/fedora/rest
  base_path: /prod
vagrant@localhost:~/californica$ lynx http://127.0.0.1:8080/fedora/rest

We need something like: url: http://127.0.0.1:<%= ENV['FCREPO_DEVELOPMENT_PORT'] || 8080 %><%= ENV['FCREPO_DEVELOPMENT_CONTEXT'] || /rest (there is a separate ticket in californica for this: uclalibrary/californica#59).

Once this is done we just need to add the FCREPO_DEVELOPMENT_CONTEXT parameter to this Packer build.

Related #28

If developer doesn't have a .gitignore (or .gitconfig) the Vagrantfile provisioning will throw and error:

There are errors in the configuration of this machine. Please fix
the following errors and try again:

File provisioner:
* File upload source file /home/hpottinger/.gitignore must exist

Here is issue: https://github.com/UCLALibrary/packer-samvera/blob/master/vagrant/hyrax/Vagrantfile#L28

Can this be more forgiving or do I have to drop back to using the shell provisioner?

At least one dev uses this and finds it useful. Add it?

https://github.com/geelen/git-smart

It has some recommended aliases that would need to be configured too (I think).

The current build works with DCE's Ubuntu roles. We want one that works with RHEL/CentOS too. I think it's still wise to maintain the build based on DCE roles since that's really the upstream, but this ticket is for adding RHEL/CentOS builds too.

I think it's wise to wait until Stephen says "These are done" though before starting to attempt this. Even better, if Hardy is doing a Vagrant test environment for the RHEL roles, wait until he's done too (since he will have worked out some of the potential kinks involved with running the roles on a single VM). If Hardy isn't doing that though, proceed with this as soon as Stephen hits a "finished" point.

This is the page to update:

https://github.com/UCLALibrary/amalgamated-samvera/wiki/Setting-up-a-Californica-development-environment,-VM-style

Virtualbox vms are slow and inefficient. I'm already running docker. I believe docker containers are supported as a packer build target.

Probably not worth too much work – but if it's not to hard, a docker image that's roughly equivalent to the vm would be very helpful.

Since work is beginning on ursus, it would be nice to add it to the VM. It should:

• share californica's solr index
• add a sshfs mount
• ... ?

Useful layer over tmux that devs might like to use --> https://github.com/tmuxinator/tmuxinator

Vagrant shared folders usually copy what's on the host to the guest. We want a shared folder that will contain the contents of the samvera-mgmt/hyrax github repo (that's checked out as a part of the build) -- so contents that are shared from the guest to the host.

Should these values warned about by Solr be bumped up by the ansible-samvera roles?

vagrant@localhost:~/samvera-mgmt$ bundle exec rake hydra:server
rake aborted!
Failed to execute solr start: *** [WARN] *** Your open file limit is currently 1024.  
 It should be set to 65000 to avoid operational disruption. 
 If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
*** [WARN] ***  Your Max Processes Limit is currently 15656. 
 It should be set to 65000 to avoid operational disruption. 
 If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh

[Edited to add context: Originally had thought a new commit had caused surfaced an issue with the Packer build. I referenced the commit and issue in the Packer build]

UCLALibrary/californica@bbf1afb

The first among the errors:
mkdir: cannot create directory ‘/opt/californica’: Permission denied

Changing the Vagrant exposed ports broke deploy tests, which were not updated at the same time. Update tests.

The error message:

Rails Error: Unable to access log file. Please ensure that
/home/vagrant/californica/log/test.log exists and is writable (ie, make it writable for user
and group: chmod 0664 /home/vagrant/californica/log/test.log). The log level has been
raised to WARN and the output directed to STDERR until the problem is fixed.

You see this when you run a test. Fix so that the log is writeable.

We now have a new default admin user whose password is pulled from the ENV variable ADMIN_PASSWORD. Configure this for the packer-samvera build.

Cap deploy deploys from a github repo. So, a developer who wants to test a Gemfile change, for instance, needs to commit code up to a new branch in the Hyrax GitHub repo and then run cap deploy with a branch name:

BRANCH=my_branch bundle exec cap localhost deploy

There used to be a deploy_via that allowed deploying via copying but that's been removed in Capistrano 3.

Possible solution: One could configure the environment to build from the local machine's git repo so that deploy will work even if you're at a place without network access. Cf. https://stackoverflow.com/questions/5532323/how-to-configure-capistrano-to-deploy-from-local-git-repository

This ticket is to add that functionality, but in the meantime we should document what happens when cap deploy is run so it's not confusing for new devs like myself when they don't see their dev box's Gemfile edits being deployed on their dev machine.

The output of the build is very long. Rather than have someone scan it to determine if the cron job succeeded or failed, we should update the job so that email is only sent on failure.

Right now the vagrant box is "versioned" with a packer-samvera version + a timestamp. It really should use a version from the GitHub checkout of the Hyrax app being built.

Pull the most useful labels from the other waffle.io projects for use here too.

We document what ports are exposed from the VM, but we should also document what ports the services are running on in the VM and note that upstream documentation may refer to the ports that are used when one is using a "native" dev setup (which are different from the ports the VM uses).

An upgrade to samvera-mgmt code yields the following from:

name: deploy to production directories with capistrano
shell: cap -p --trace=stdout {{ cap_deployment }} deploy

"** Execute deploy:restart_apache",
    "04:53 deploy:restart_apache",
    "      01 sudo systemctl restart apache2",
    "      01 sudo",
    "      01 : ",
    "      01 no tty present and no askpass program specified",
    "      01 ",
    "cap aborted!",

Using the Serverspec test for Vagrant-Embark, write a test for the devbox.

This repo should have a different published box from the ksclarke/packer-samvera repo so its README needs to be updated with that.

A couple devs use this so let's put it in the build

Set up a nightly build that uploads the artifact into Vagrant Cloud so there is always a fresh copy of the VM available for use.

To be able to import images we need two variables set in the dev box's env setup. This will go in the Ansible roles for capdeploy:

https://github.com/UCLALibrary/uclalib_role_samvera_capdeploy/blob/master/tasks/dotenv_setup.yml

(in the template file)

Acceptance:

• The environment files have been updated in the related role
• The example.csv spec fixture can be imported with images that display in the UI

The time it takes to boot up a new VM should be closer to the Ubuntu range. 10 minutes is really long to wait. Is there something that can be done with the SSHFS caching to improve startup time? Or what is the problem here?

Time it takes CentOS VM to load
===============================
real    11m50.080s
user    0m5.338s
sys    0m3.895s

Time it takes Ubuntu VM to load
===============================
real    2m7.896s
user    0m3.531s
sys    0m2.292s

0;32m    box: TASK [ansible-samvera/roles/fits : install zip utility] ************************ [0m
 [0;32m    box:  [0;32mok: [127.0.0.1] [0m [0m
 [0;32m    box: [0m
 [0;32m    box: TASK [ansible-samvera/roles/fits : download fits zip version 0.8.4] ************ [0m
 [0;32m    box:  [0;33mchanged: [127.0.0.1] [0m [0m
 [0;32m    box: [0m
 [0;32m    box: TASK [ansible-samvera/roles/fits : unpack fits] ******************************** [0m
 [0;32m    box:  [0;31mfatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Failed to find handler for "/home/vagrant/install/fits-0.8.4.zip". Make sure the required command to extract the file is installed. Command "/bin/tar" could not handle archive. Command "/usr/bin/unzip" could not handle archive."} [0m [0m

Having a separate 'prod' and 'dev' fedora environment on the dev box causes problems. On that machine, we want there to be just 'test' and another Fedora node/context/environment. In fact, on our servers it's also fine that there is only one Fedora node/context/environment (the production one).

We need to make some changes to the Ansible roles to synchronize.

Add to templates/dotenv_config.j2:

FEDORA_BASE_PATH={{ fedora_base_path }}

Add to defaults/main.yml:

fedora_base_path: "{{ project_name }}"

This will make the Fedora node/context/environment be 'californica' instead of 'prod'.

We also need to add to README.md documentation about setting the FEDORA_BASE_PATH variable. It exists in the californica code base already but is new to our Ansible roles.

The Vagrant box uses ssh-agent installed on the host to be able to do deploys to remote servers and commits to GitHub from within the VM. Like the SSHFS stuff, we should add a build message to the Vagrantfile telling someone they need to have ssh-agent installed on their host machine to be able to use those features of the box.

This is documented in the Packer project, but it's likely the Vagrant box will be downloaded from Vagrant Cloud without people even looking at the Packer project.

Right now, the output when you run without reading the requirements in the README is a little cryptic. It'd be nice if a clear error message told you that you need to install SSHFS and vagrant-sshfs in order to take advantage of the VM's shared/synced folder.

@sgurnick has worked out putting ursus and californica on a single VM. Use his work (Ansible roles) to configure the dev box too.

• Developer's box runs Ursus and Californica
• Expose both Ursus and Californica code directories to host machine via SSHFS

depends on uclalibrary/ursus#32

It's used in:

https://github.com/UCLALibrary/californica/blob/c48fdf617c5936ddcd1a29458488d5e286698769/config/honeybadger.yml

Not something my work needs so will do this on my own time, but now that we have an automated build of our 'californica' Hyrax application it would be nice to set up a nightly build of the new 'nurax' app too.

I've made some changes for 'californica' that have probably broken the Nurax build (mainly in the .env configuration). Fixing those will also make the project more robust for different Hyrax apps (at least in that some of the areas of difference can be documented or perhaps some best practices established for naming db variables, etc.)

When a Mac goes to sleep the shared dir gets unmounted and this causes all kinds of problems because the Mac still thinks it's there and won't let you easily remount.

It's been tested on Linux. Test it on Windows.

It's been tested on Linux. Confirm it works for Mac users too.

Probably not a high priority, but would be nice to have a script to clean up nightly snapshots of packer-samvera that are persisted to Vagrant Cloud. Doing snapshots nightly produces a lot of boxes. There isn't an explicit storage limit with Vagrant Cloud but I'd bet there is an unofficial one, at which point they ping people and ask about their disk usage.

It doesn't look like the SSHFS plugin detection is working as intended. Below is the output message, which has the error message in a single string (\n's are not interpreted).

There was an error loading a Vagrantfile. The file being loaded and the error message are shown below. This is usually caused by a syntax error.

Path: /Users/kevin/.vagrant.d/boxes/uclalibrary-VAGRANTSLASH-californica/0.3-centos-SNAPSHOT-1538508888/virtualbox/Vagrantfile

Line number: 0

Message: RuntimeError: required plugin vagrant-sshfs is not installed! \nRecommended action: \ninstall the plugin with the following command:\n\n  vagrant plugin install vagrant-sshfs\n\nGentle reminder: also be sure to install the SSHFS package on your host OS.

The error message was seen on a Mac.

Acceptance:

• The error message should come out formatted or formatting should be removed from the string

Hardy is using 'be' (for 'bundle exec'). Alternatively could be 'do' or 'run' (or we could set more than one).

be rake spec
do rake spec
run rake spec

be rake hydra:test_server
do rake hydra:test_server
run rake hydra:test_server

Should be able to:

bundle exec rake ci

and do:

https://github.com/samvera/hyrax/wiki/Making-Admin-Users-in-Hyrax

but something in the way the VM is configured is wrong

Things like expected ENV variables (DATABASE_NAME instead of DB_NAME, for instance)
Things in the database.yml and fedora.yml (like parameterized values)
etc., etc...

The build was broken by the PR that changed the databases. rake ci now says the database does not exist. We run db:setup as a part of the provisioning but may have now hit the issue Hardy had where the test dbs were not created by the db:setup: https://github.com/UCLALibrary/californica/issues/36

Predecessor: @andrewbenedictwallace opened uclalibrary/californica#19 which I think is related to this ticket (plus there have been lots of good comments over on that ticket that might be relevant for doing the work for the VM).

I'm creating the ticket here though for the VM since we'll probably want to make these config changes optional in Californica (so that the VM can override Californica defaults to implement as desired).

For the VM, it probably makes sense to use /tmp and /var/log as standard system locations (though tmp might be different since the testing framework uses this to download Fedora/Solr and perhaps these things can be cached locally so they don't have to be downloaded multiple times (since /tmp would be wiped on system restart)).

andy@YRL-ML-DLP704:californica$ vagrant halt
==> default: Attempting graceful shutdown of VM...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

shutdown -h now

Stdout from the command:



Stderr from the command:

bash: line 5: shutdown: command not found

Forgot to take notes, but shutdown now -h inside the VM shell fails with a similar "command not found" message. sudo shutdown now -h works, but I assume there's other vagrant stuff (like w/ the sshfs mount) that might not get done.

So, acceptance criteria (now that the problem is understood):

• Create a uclalib_role_developers_box Ansible role
• Create tests for uclalib_role_developers_box that run with Travis config
• Update packer-samvera to pull in and use the uclalib_role_developers_box role
• Confirm that vagrant halt works for the dev box published to Vagrant Cloud

Right now, there is a lot of copy and paste between the two scripts. This could be reduced so there is less to maintain over time (i.e. make one call the other).