curationexperts / ansible-samvera Goto Github PK
View Code? Open in Web Editor NEWPrototype Configuration Management Scripts for Samvera based servers
License: Apache License 2.0
Prototype Configuration Management Scripts for Samvera based servers
License: Apache License 2.0
I'm seeing this in Tufts and then trying to run from-cm as it's currently checked in. I've tried against both ubuntu 16.04 and 16.10 and get the same error.
Testing from the command line I get the following:
fatal: [54.197.209.70]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/gem install --no-user-install --no-document bundler", "failed": true, "msg": "ERROR: Error installing bundler:\n\t\"bundle\" from bundler conflicts with /usr/local/bin/bundle", "rc": 1, "stderr": "ERROR: Error installing bundler:\n\t\"bundle\" from bundler conflicts with /usr/local/bin/bundle\n", "stderr_lines": ["ERROR: Error installing bundler:", "\t\"bundle\" from bundler conflicts with /usr/local/bin/bundle"], "stdout": "", "stdout_lines": []}
to retry, use: --limit @/Users/mark/Documents/workspace/_no_backup_/deploying_hydra/frbm-cm/build_cypripedium_server.retry
PLAY RECAP ***************************************************************************************************
54.197.209.70 : ok=27 changed=20 unreachable=0 failed=1
MARKs-MacBook-Pro-2:frbm-cm mark$ ssh [email protected]
Welcome to Ubuntu 16.10 (GNU/Linux 4.8.0-56-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
Your Ubuntu release is not supported anymore.
For upgrade information, please visit:
http://www.ubuntu.com/releaseendoflife
New release '17.04' available.
Run 'do-release-upgrade' to upgrade to it.
*** System restart required ***
Last login: Fri Nov 3 00:32:40 2017 from 67.220.22.48
ubuntu@demo:~$ sudo gem install bundler
bundler's executable "bundle" conflicts with /usr/local/bin/bundle
Overwrite the executable? [yN] n
ERROR: Error installing bundler:
"bundle" from bundler conflicts with /usr/local/bin/bundle
ubuntu@demo:~$ bundle --version
Bundler version 1.16.0
ubuntu@demo:~$ gem install bundler
Fetching: bundler-1.16.0.gem (100%)
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the /usr/local/lib/ruby/gems/2.4.0 directory.
ubuntu@demo:~$
Run this cron job daily to remove any files owned by the deploy user that are older than 7 days. Otherwise /tmp fills up with imagemagick temp files.
sudo find /tmp -type f -mtime +7 -user deploy -execdir rm -- '{}' \;
set up appropriate configurations for all content typically stored in $RAILS_ROOT/tmp in development environments
https://github.com/curationexperts/epigaea/blob/80b78ded44ee21ee3ead3c7b19497f99f37c6fb6/config/deploy.rb#L37-L49
append :linked_dirs, "public/assets"
append :linked_dirs, "tmp/pids"
append :linked_dirs, "tmp/cache"
append :linked_dirs, "tmp/sockets"
append :linked_dirs, "log"
append :linked_dirs, "tmp/drafts"
append :linked_dirs, "tmp/exports"
append :linked_dirs, "tmp/templates"
Currently, Val uses Ed25519 ssh keys (and they're more secure, so maybe all of us should be doing this). As-is, our sshd config does not support these keys. We should fix this. See https://linux-audit.com/using-ed25519-openssh-keys-instead-of-dsa-rsa-ecdsa/ for some pointers.
Compare this repo's usage https://github.com/curationexperts/ansible-samvera/search?utf8=โ&q=deploy&type=
to ansible-hydra usage https://github.com/curationexperts/ansible-hydra/search?utf8=โ&q=capistrano_user&type=
If for any reason we use a different user name to run the rails application - like when building a Vagrant VM for Samvera Camp, you can't use any of the scripts that assume there's a user named deploy
. For my specific case, I was going to try to leverage the sidekiq role to get sidekiq running as a system service, but the role is hardcoded to use the deploy
user which doesn't exist in the VM environment. Ideally I could use the role, but with the vagrant
user.
I'm getting
TASK [first_deploy : ensure default admin set with capistrano] ********************************************************************************
fatal: [52.206.81.96]: FAILED! => {"changed": true, "cmd": "cap localhost hyrax:ensure_default_admin_set", "delta": "0:00:00.287908", "end": "2017-11-03 20:28:26.710093", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2017-11-03 20:28:26.422185", "stderr": "(Backtrace restricted to imported tasks)\ncap aborted!\nDon't know how to build task 'hyrax:ensure_default_admin_set' (see --tasks)\n\n(See full trace by running task with --trace)", "stderr_lines": ["(Backtrace restricted to imported tasks)", "cap aborted!", "Don't know how to build task 'hyrax:ensure_default_admin_set' (see --tasks)", "", "(See full trace by running task with --trace)"], "stdout": "", "stdout_lines": []}
to retry, use: --limit @/Users/mark/Documents/workspace/_no_backup_/deploying_hydra/tufts-cm/tufts-ubuntu.retry
- name: ensure default admin set with capistrano
shell: cap localhost hyrax:ensure_default_admin_set
args:
chdir: /home/{{ ansible_ssh_user }}/{{ project_name }}
But the hyrax:ensure_default_admin_set Capistrano task isn't defined in all of our projects. Could we just call the rake task directly?
execute :rake, 'hyrax:default_admin_set:create'
It looks like after some packages are installed via apt get upgrade, apt get update, the server needs a restart. Rather than adding this to the packages role, however, let's make it a separate role that can be placed at the end of a playbook. That ensures it captures any other changes made during the configuration and proves that the web server, solr, and fedora restart correctly after a reboot.
They should be configured with the name of the solr index we created:
deploy@mira-fc4:/opt/epigaea/current/config$ cat solr.yml
production:
url: http://127.0.0.1:8983/solr/epigaea
deploy@mira-fc4:/opt/epigaea/current/config$ cat blacklight.yml
production:
adapter: solr
url: http://127.0.0.1:8983/solr/epigaea
One of the most time-consuming maintenance tasks on ansible-samvera built systems is looking up the postgres password. Instead, let's record it in a secure way and make the ubuntu user not have to type it.
Today the fits download site went offline. I hope it comes back soon, and in the meantime I've been able to copy the .zip file from previously installed machines, but we should cache important installers somewhere for these kinds of outages.
Hyrax's default admin set contains a slash ("admin_set/default"), which leads to a bug where it can't be displayed unless passenger is configured to allow encoded slashes. For more context, see:
The fix is to add this line to conf-enabled/passenger.conf
:
PassengerAllowEncodedSlashes on
See ffmpeg vs. fits. Fits will inherit project-wide {{ install_path }}. ffmpeg assumes a ~/install
exists. Pick one pattern or the other and make sure all roles ensure that their expected install directory exists at the beginning of the role.
ISSUE
We'd like to keep the home directory for the install as clean as possible.
Right now we have something like
vagrant@camper:~$ ls
imagemagick_sources install nodesource_setup.sh
ISSUE
For security purposes our servers should drop ssh sessions after a specified amount of inactivity. We want to be resilient on poor network connections though.
PROPOSED SETTINGS
ClientAliveInterval 120
ClientAliveCountMax 5
In the proposed settings, the client would have to fail at five consecutive 2-minute checks (i.e. 10 minutes), but would remain alive if the system provides a successful response in any one of the tries. In the original, the session dies immediately on a single check at 10 minutes with no retries.
This needs to go in /etc/ssh/sshd_config
See sample at https://github.com/curationexperts/ansible-hydra/blob/master/roles/system_setup/templates/ssh_config
I've created a Packer build using these roles and it creates a VM output. In the process, I found there were slight differences between the VM source image I used and the source Ubuntu image/AMI you all must be using. I'm debating submitting PRs for these but for the moment have just handled them via a shell script in Packer.
I thought I'd document them here in case others use these roles and run into them.
Missing: nokogiri and nodejs (or any JavaScript runtime, but NodeJS was the one I chose to install)
The environment also needed to be set to UTF-8 for the Postgres databases to be created using that rather than LATIN1: update-locale LANG="en_US.UTF-8" LANGUAGE="en_US"
(Having the databases created as LATIN1 caused the Fedora environment fail to start -- though this wasn't noticed in the Fedora role but in the last step of the first_deploy role where the admin set is created).
Fwiw, I also had to set the perms on /var/www to deploy:deploy (though for some reason you all aren't needing to do this, I guess). But this https://stackoverflow.com/questions/24470520/capistrano-mkdir-permission-denied solved a perm problem for me.
If you think it's worth splitting some of these into smaller tickets and getting rid of this all-in-one ticket, that's fine with me.
We've hardcoded the ssh timeout into our config and may have cases (like building VMs or training EC2 instances for Camps) where we either want longer or no timeouts.
Possible options would be to make the timeouts settings configurable variables, or to pull out the timeout setting and make it a separate role:
roles:
...
- { role: sshd_config, ClientAliveInterval=0 }
...
OR
roles:
...
- { role: sshd_config }
- { role: sshd_timeout }
...
ACCEPTANCE
ISSUE
We'd like to keep the install user's home directory as clean as possible, right now we have
vagrant@camper:~$ ls
imagemagick_sources install nodesource_setup.sh
Talk to Jamie about whether we could use the nailgun pattern for imagemagick and ffmpeg as well as fits
Is there any reason not to include RAILS_ENV=production in the environment instead of having to prefix every command with it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.