consuldemocracy / installer Goto Github PK

CONSUL DEMOCRACY installer for production environments

License: GNU Affero General Public License v3.0

Dockerfile 35.17% Shell 15.44% DIGITAL Command Language 10.24% Jinja 39.16%

installer's Introduction

CONSUL DEMOCRACY

Citizen Participation and Open Government Application

This is the opensource code repository of the eParticipation website CONSUL DEMOCRACY, originally developed for the Madrid City government eParticipation website, and currently maintained by the open source software community in collaboration with the CONSUL DEMOCRACY Foundation.

Documentation

Check the ongoing documentation to learn more about how to start your own CONSUL DEMOCRACY fork, install it, customize it and learn to use it as an administrator/maintainer.

CONSUL DEMOCRACY Foundation and project website

You can access the main website of the project at http://consuldemocracy.org where you can find information about the use of the platform, the CONSUL DEMOCRACY Foundation, the global community of users and local partners, news, and ways to get more support or get in touch.

Configuration for development and test environments

NOTE: For more detailed instructions, check the local installation docs.

Prerequisites: install git, Ruby 3.2.5, CMake, pkg-config, Node.js 18.20.3, ImageMagick and PostgreSQL (>=9.5).

Note: The bin/setup command below might fail if you've configured a username and password for PostgreSQL. If that's the case, edit the lines containing username: and password: (adding your credentials) in the config/database.yml file and run bin/setup again.

git clone https://github.com/consuldemocracy/consuldemocracy.git
cd consuldemocracy
bin/setup
bin/rake db:dev_seed

Run the app locally:

bin/rails s

You can run the tests with:

bin/rspec

Note: running the whole test suite on your machine might take more than an hour, so it's strongly recommended that you setup a Continuous Integration system in order to run them using parallel jobs every time you open or modify a pull request (if you use GitHub Actions or GitLab CI, this is already configured in .github/workflows/tests.yml and .gitlab-ci.yml) and only run tests related to your current task while developing on your machine. When you configure the application for the first time, it's recommended that you run at least one test in spec/models/ and one test in spec/system/ to check your machine is properly configured to run the tests.

You can use the default admin user from the seeds file:

user: [email protected] pass: 12345678

But for some actions like voting, you will need a verified user, the seeds file also includes one:

user: [email protected] pass: 12345678

Configuration for production environments

See installer

Current state

Development started on 2015 July 15th. Code was deployed to production on 2015 september 7th to decide.madrid.es. Since then new features are added often. You can take a look at the current features at the project's website and future features at the Roadmap and open issues list.

License

Code published under AFFERO GPL v3 (see LICENSE-AGPLv3.txt)

Contributions

See CONTRIBUTING.md

installer's People

Contributors

Stargazers

Watchers

installer's Issues

Restarting the Server leaves the application dead

We followed your install steps and magically the app runned. We restarted the server and the app was not running. This steps is supposed to be for production environments. Restarting the server shouldn't leave the application killed. Is there a way to keep a service running the app alive?

The install scripts does so many hidden jobs, the person that deploys the app is hard to understand. It might be easy for ruby people to understand it, but for me that i am a 10 years none ruby developer seems complicated. It used to be more clear what you should do to install for example a wordpress site. I don't know if ruby makes the things more complicate, but i would expect to grab some files, configure the db, configure ngnix/ruby and Boom the website appears.

We didn't also find any mention in the documentations for the update procedure. Does it exist?

thank you in advance for clarifying our questions.

Staging environment

What

Add an option to run the playbook for a staging environment

Why

To be able to deploy changes to a staging environment using cap staging/preproduction deploy before deploying the changes to a production server

How

Add a variable environment and substitute the hardcoded production words and files in the installer

Notes

The current CONSUL codebase uses preproduction instead of staging environment
It's probably simpler to go ahead and use preproduction for now 👌

Installer - Spec Fail

I was having a deploy running v0.19. Then I rebased it on top of v1.0.0 and did a Capistrano deployment. Then I noticed some thing like (or at least that is what I have now).

01:36 deploy:restart
      01 /etc/init.d/unicorn_consul restart
      01 bash: /etc/init.d/unicorn_consul: No such file or directory
      01
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as [email protected]: /etc/init.d/unicorn_consul restart exit status: 127
/etc/init.d/unicorn_consul restart stdout: bash: /etc/init.d/unicorn_consul: No such file or directory

Then I thought of running the installer as some thing seemed to be missing. After some fixes the installation got stuck in specs. Then I thought of running it on a bare Ubuntu 16.04 server with a not modified (except the base config) and ended up in the same error.

I didn't see any thing related to the error in the issue list so thought of opening an issue.

BTW is there upgrade docs that give advice on upgrading. I remember getting an email from v0.18 to v0.19. But couldn't find a page that might suggest some thing like.


TASK [specs : Redirect to https] ****************************************************************************************************************
skipping: [192.241.137.219] => changed=false 
  skip_reason: Conditional result was False

TASK [specs : uri] ******************************************************************************************************************************
fatal: [192.241.137.219]: FAILED! => changed=false 
  connection: close
  content: |-
    <html>
    <head><title>502 Bad Gateway</title></head>
    <body bgcolor="white">
    <center><h1>502 Bad Gateway</h1></center>
    <hr><center>nginx/1.10.3 (Ubuntu)</center>
    </body>
    </html>
  content_length: '182'
  content_type: text/html
  date: Fri, 19 Jul 2019 13:27:32 GMT
  elapsed: 0
  msg: 'Status code was 502 and not [200]: HTTP Error 502: Bad Gateway'
  redirected: false
  server: nginx/1.10.3 (Ubuntu)
  status: 502
  url: http://127.0.0.1

PLAY RECAP **************************************************************************************************************************************
192.241.137.219            : ok=77   changed=54   unreachable=0    failed=1    skipped=29   rescued=0    ignored=0

Add email configuration

What

Add SMTP configuration

Add brew documentation

What

Add documentation to README on how to install brew

TASK [Gathering Facts] fatal: UNREACHABLE!

Hi,

Trying to run directly into the vanilla Ubuntu 14.06 instance (LXC), but encountered the following error:

$ sudo -S ansible-playbook consul.yml -i hosts --extra-vars "target=servers"

PLAY [Create deploy user] *******************************************************************************************

TASK [Gathering Facts] **********************************************************************************************
fatal: [192.168.10.115]: UNREACHABLE! => {"changed": false, "msg": "Authentication failure.", "unreachable": true}
        to retry, use: --limit @/home/zenny/consul_installer/consul.retry

PLAY RECAP **********************************************************************************************************
192.168.10.115             : ok=0    changed=0    unreachable=1    failed=0

HOWTO: Successful consul ansible installation in Ubuntu 16.04 locally

Hi,

In order to install consul with this ansible script locally in Ubuntu 16.04, one needs to do the following step-by-step:

Login as root user (sudoers did not work)
Create a rsa key
# ssh-keygen -t rsa
Install necessary dependency for 4 below:
# apt-get install software-properties-common -y
Install latest ansible and dependencies from upstream (requires ansible 2.4+ which is not readily available in ubuntu repository)
# add-apt-repository ppa:ansible/ansible
# apt-get update
# apt-get -y install ansible curl python-simplejson git
Pull this repo (as root itself) and change to the installer directory:
# git clone https://github.com/consul/installer
# cd installer
Disable remote_user as it is a local ansible instance:
# sed -i -e 's/remote_user: vagrant/#remote_user: vagrant/g' consul.yml
Make hosts to reflect localhost (127.0.0.1)
# echo '[servers]' > hosts
# echo '127.0.0.1' >> hosts
Make changes to /root/installer/group_vars/all to reflect your own needs.
Run ansible-playbook with -c local flag to run locally.
# ansible-playbook consul.yml -i hosts -c local --extra-vars "target=servers"
Once finished, browse to the server_ip_address in a browser and login with default credentials:

username: [email protected]
password: 12345678

Once logged in, change admin password. That's all!

Hope this helps for anyone who shall be in my shoes.

Kudos goes to the entire consul team for developing such a thought-provoking software and a comprehensive installer.

Cheers,
/z

Add server name to secrets

Context

Emails send by the application sometimes have links, these links use the server_name to generate the corresponding urls

What

Add server_name to the secrets.yml

Why

Links in emails do not have the correct urls

How

Use the IP by the default and add instructions to configure a variable with the real domain name in group_vars/all

Deploy production and preproduction in the same server

Hi Consul commnunity!

I want to display production and preproduction on the same server. I can install the production environment on the server, but when I try to do the same with the preproduction environment I get an error.

Capistrano's task is aborted, and he does not continue to deploy.

deploy-secrets.yml

preproduction:
  deploy_to: "/home/deploy/preproduction-consul"
  ssh_port: "22"
  server1: "my-server-ip"
  db_server: "localhost"
  user: "deploy"
  server_name: "my-web1.com"
  full_app_name: "preproduction-consul"

production:
  deploy_to: "/home/deploy/consul"
  ssh_port: "22"
  server1: "my-server-ip"
  db_server: "localhost"
  user: "deploy"
  server_name: "my-web.com"
  full_app_name: "consul"

Production and preproduction have to be on the same server. But the domains of the application are different.

If you need anything else, let me know.

Thanks!

Update REAMDE with login information

What

Explain that to perform operations in the server we should login as the deploy user instead of the root user

Add documentation on how to setup email delivery

Context

Setting up email is giving a hard time to many developers

Objetives

Add comprehensive documentation on how to setup email deliveries

Rerun the playbook after activating the Capistrano role

Refereces

Comment: #57 (comment)

Context

The installer runs fine multiple times with the default setup, but if the capistrano role is activated, the second time the installer is run it raises an error. This is one the error messages that have been reported:

TASK [rails : Deploy user permissions] *****************************************
fatal: [178.128.162.212]: FAILED! => {"changed": false, "gid": 0, "group": "root", "mode": "0644", "msg": "chown failed: [Errno 1] Operation not permitted: '/home/deploy/consul/app/views/pages/.conditions.html.erb.swp'", "owner": "root", "path": "/home/deploy/consul/app/views/pages/.conditions.html.erb.swp", "size": 1024, "state": "file", "uid": 0}

Objectives

Allow the installer to run multiple times with the Capistrano role activated.

Remove duplicate file structure

Context

The installer installs CONSUL in 2 folders:
/home/deploy/consul
/home/deploy/consul/current

This is due to an previous version of the installer that installed it in /home/deploy/consul
And an updated version of the installer that uses capistrano and installs it at /home/deploy/consul/current

What

Make the installer install a single copy of CONSUL at /home/deploy/consul/current

Support other GNU/Linux distributions

It would be great if we could make the installer fully compatible with other distributions, like Debian, instead of supporting just Ubuntu.

Note: in order to support other distributions we'd have to find a way to run the tests on machines with those distributions.

Fix user permissions when copying deploy.rb file locally

Context

Some users are having permission problems when running the installer.

TASK [capistrano : Copy deploy.rb (locally)] ***********************************
fatal: [*********]: FAILED! => {"changed": false, "checksum": "******", 
"dest": "/home/consul/installer/consul/config/deploy.rb", "gid": 0, 
"group": "root", "mode": "0644", "msg": "chown failed: failed to look up user deploy",
 "owner": "root", "path": "/home/consul/installer/consul/config/deploy.rb", 
"size": 1922, "state": "file", "uid": 0}
    to retry, use: --limit @/home/consul/installer/consul.retry

PLAY RECAP *********************************************************************
192.168.1.10               : ok=47   changed=3    unreachable=0    failed=1

How to fix

An option would be to allow running the installer directly from the host machine to avoid these problems. Another option would be to create the necessary user groups in the local machine.

'Start Unicorn' task fails due to syntax error in production.rb

Running installer via Ansible for a production environment on a fresh build of Ubuntu 16.04 LTS

Output:

TASK [unicorn : Start Unicorn] *******************************************************************************************************
fatal: [178.79.181.38]: FAILED! => {"changed": true, "cmd": "/home/deploy/.rvm/gems/ruby-2.3.2/wrappers/unicorn -c config/unicorn.rb -E production -D", "delta": "0:00:04.677967", "end": "2019-05-03 16:57:27.613531", "msg": "non-zero return code", "rc": 1, "start": "2019-05-03 16:57:22.935564", "stderr": "master failed to start, check stderr log for details", "stderr_lines": ["master failed to start, check stderr log for details"], "stdout": "", "stdout_lines": []}
        to retry, use: --limit @/root/installer/consul.retry

Error log (unicorn.stderr.log):

I, [2019-05-03T16:57:23.671088 #10526]  INFO -- : Refreshing Gem list
	/home/deploy/.rvm/gems/ruby-2.3.2/gems/activesupport-5.0.7.2/lib/active_support/dependencies.rb:293:in `require': /home/deploy/consul/config/environments/production.rb:104: syntax error, unexpected ':', expecting keyword_end (SyntaxError)
	    port:                 "25",
	         ^
	/home/deploy/consul/config/environments/production.rb:104: syntax error, unexpected ',', expecting keyword_end
	/home/deploy/consul/config/environments/production.rb:109: syntax error, unexpected end-of-input, expecting keyword_end

production.rb:

  if ENV["RAILS_LOG_TO_STDOUT"].present?
    logger           = ActiveSupport::Logger.new(STDOUT)
    logger.formatter = config.log_formatter
    port:                 "25", <--- THIS IS LINE 104
  end

Dockerized deployment

Hi all,
Thanks for the good work.
I'm used to deploying docker images in production (AWS or CleverCloud) and to have a docker-compose file similar to my production env (1 postgresql and at least 2 app instances).
I'm a bit lost in this setup.

Do you know if someone is working on a production docker image of consul ?
I guess, we would need a way to enable cron and emails sender on only one instance.
To create and migrate automatically the database schema/data.

Regards,
Thomas.

Add SSL configuration

What

Add basic ssl configuration with Let’s Encrypt

Configure consul repo

Right now this installer clones consul/consul repo but its recommended to use a fork in order to be able to make customizations and store them in version control.

So I guess we could add steps to:

Create a consul fork (if it doesn't exist already)
https://consul_docs.gitbooks.io/docs/content/en/forks/create.html
Configure the fork's url at https://github.com/consul/installer/blob/configuration/roles/rails/tasks/main.yml#L10 and

For the second step, it would be nice to avoid changing the installer source code. I guess passing the git repo url as a parameter of the sudo ansible-playbook -v consul.yml -i hosts --extra-vars "target=vagrant" could work? But the best solution would be to store a particular installation details on a file.. maybe consul.yml? I don't know 🙈

Playbook is stuck

I used the method in issue #14 to install consul on my Ubuntu server. The first time i could run the playbook without any problem. Only when I try to run the playbook again it gets stuck on a certain task.
It gets stuck on:

TASK [rails : Load configuration seeds] **************************************************************************
task path: /root/installer/roles/rails/tasks/main.yml:65
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189 `" && echo ansible-tmp-1542197596.54-5055084817189="` echo /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189 `" ) && sleep 0'
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-32731ATOFOF/tmpLysohr TO /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/ /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1542197596.54-5055084817189/AnsiballZ_command.py && sleep 0'

ERROR: installer/roles/postgresql/tasks/main.yml': line 2, column 3

Hi,

The installer stucks at ansible-playbook execution as follows inside a brand new Ubuntu 16.04VM:

$ sudo -S ansible-playbook consul.yml -i hosts --extra-vars "target=servers"         
ERROR! no action detected in task

The error appears to have been in '/home/zenny/consul_installer/roles/postgresql/tasks/main.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- include_role:
  ^ here

Establish Log files permissions

When trying to run bin/rails c development an error message appeared (I lost it, my bad) that was saying something like "no permissions to write log files". So I did sudo chmod +664 *.log at /log folder.

This should be done by the installer script

Run in production environment by default

What

Right now the configuration is setup to run in Rails's development environment.
Instead, run all necessary commands and setups in the production environment by default

Add documentation on how to setup without root access

Context

The installer assumes we can login as the root user using a public ssh key and without password. Which is only used to create a deploy user, who is the one that will actually install all libraries.

However some developers are not granted root user access and cannot run the installer with the default configuration.

Objectives

Allow the installer to be run with any user with sudo privileges.

How

After doing a couple of tests, it seems we only need to add this line: remote_user: name_of_user here.

Update README to run playbook in verbose mode

What

When running the playbook locally in virtual box, the verbose option is active
However when running in a remote server, we need to add the -v option when running the playbook

Update documentation to run the playbook for remote users in verbose mode by default

Configure system email service

What

Configure the installer to use sendmail instead of SMTP by default or configure postfix so the server can send emails automatically.

Alternatively, we could configure postfix on the server during the installation process.

Why

Until we configure delayed job properly, we're getting a nasty 500 error when trying to send emails using the installer's default SMTP settings.

Update Documentation regarding initial ssh connection

What

In order to make the initial connection to the remote server, the playbook expects access by the root user, using a public key

We should update the documentation to reflect that having the public_key in the server is a prerequisite to running the playbook

Error when installing Postgres extensions

I just tried to install Consul with Ansible on a Ubuntu 16.04 LTS machine and I got the following error message:

TASK [postgresql : Add PostgreSQL extensions] *********************************************************************************************************************************************************************
ok: [95.216.150.39] => (item=plpgsql) => changed=false 
  ansible_loop_var: item
  db: consul_production
  ext: plpgsql
  item: plpgsql
  queries: []
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OperationalError: could not open extension control file "/usr/share/postgresql/9.5/extension/unaccent.control": No such file or directory
failed: [95.216.150.39] (item=unaccent) => changed=false 
  ansible_loop_var: item
  item: unaccent
  msg: |-
    Database query failed: could not open extension control file "/usr/share/postgresql/9.5/extension/unaccent.control": No such file or directory
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OperationalError: could not open extension control file "/usr/share/postgresql/9.5/extension/pg_trgm.control": No such file or directory
failed: [95.216.150.39] (item=pg_trgm) => changed=false 
  ansible_loop_var: item
  item: pg_trgm
  msg: |-
    Database query failed: could not open extension control file "/usr/share/postgresql/9.5/extension/pg_trgm.control": No such file or directory

I solved it by manually installing postgresql-contrib:

sudo apt install postgresql-contrib

I didn't have any problem before so this is maybe due to the cleaning of #76. In any case, I suggest we add postgresql-contrib to the list of dependencies to avoid this error.

Context

This crash happened on a VPS on Ubuntu 16.04 hosted on Hetzner.

Unicorn can no longer be exetuted as deploy user

Hello,
today I deployed Consul on a new server. The installation worked straightforward, except that in my previous attempt. about a month ago, I was able to schedule unicorn to start at boot using user "deploy", in this new instance it can only be run as root.

If I "sudo chown -R deploy:wheel /home/deploy/consul" then I'm able to start it againg with user "deploy".

What's best?

Thank you for your help
Best regards

Make the playbook inmutable

What

If the script is only run once and everything goes smoothly there is no problem...
However if something goes wrong, some roles are not prepared to be run a second time

We should make the playbook inmutable, thus being able to run the playbook multiple times without seeing errors due t changes in the state of the server

Checkout:

Capistrano role

Task Install gems Fails

Scenario, libs and versions :

Local Machine:
Ubuntu 16.04.4 LTS
ruby 2.3
usuario local: user
simplejson 3.8.1
ansible 2.5.4

Remote server:
Ubuntu 16.04.4 LTS
ruby 2.3
user: deploy (created by ansible )
python-simplejson 3.8.1
Ubuntu 16.04 x64

TASK [capistrano : Install gems (this may take a few minutes)] *********************************************************************************************************************************************
fatal: [10.99.52.80]: FAILED! => {"changed": true, "cmd": "gem install bundler && bundle install", "delta": "0:00:17.052788", "end": "2018-06-27 12:06:38.257560", "msg": "non-zero return code", "rc": 5, "start": "2018-06-27 12:06:21.204772", "stderr": "", "stderr_lines": [], "stdout": "Successfully installed bundler-1.16.2\nParsing documentation for bundler-1.16.2\nDone installing documentation for bundler after 3 seconds\n1 gem installed\nDon't run Bundler as root. Bundler can ask for sudo if it is needed, and\ninstalling your bundle as root will break this application for all non-root\nusers on this machine.\nFetching gem metadata from https://rails-assets.org/..\nFetching gem metadata from https://rubygems.org/...........\nFetching gem metadata from https://rails-assets.org/..\nUsing rake 12.3.1\nUsing concurrent-ruby 1.0.5\nUsing i18n 0.9.1\nUsing minitest 5.11.1\nUsing thread_safe 0.3.6\nUsing tzinfo 1.2.4\nUsing activesupport 4.2.10\nUsing builder 3.2.3\nUsing erubis 2.7.0\nUsing mini_portile2 2.3.0\nFetching nokogiri 1.8.2\nInstalling nokogiri 1.8.2 with native extensions\nGem::Ext::BuildError: ERROR: Failed to build gem native extension.\n\n current directory: /var/lib/gems/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri\n/usr/bin/ruby2.3 -r ./siteconf20180627-6654-k2t6v1.rb extconf.rb\nmkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h\n\nextconf failed, exit code 1\n\nGem files will remain installed in /var/lib/gems/2.3.0/gems/nokogiri-1.8.2 for\ninspection.\nResults logged to\n/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out\n\nAn error occurred while installing nokogiri (1.8.2), and Bundler cannot\ncontinue.\nMake sure that gem install nokogiri -v '1.8.2' succeeds before bundling.\n\nIn Gemfile:\n graphiql-rails was resolved to 1.4.8, which depends on\n rails was resolved to 4.2.10, which depends on\n actionmailer was resolved to 4.2.10, which depends on\n actionpack was resolved to 4.2.10, which depends on\n actionview was resolved to 4.2.10, which depends on\n rails-dom-testing was resolved to 1.0.9, which depends on\n nokogiri", "stdout_lines": ["Successfully installed bundler-1.16.2", "Parsing documentation for bundler-1.16.2", "Done installing documentation for bundler after 3 seconds", "1 gem installed", "Don't run Bundler as root. Bundler can ask for sudo if it is needed, and", "installing your bundle as root will break this application for all non-root", "users on this machine.", "Fetching gem metadata from https://rails-assets.org/..", "Fetching gem metadata from https://rubygems.org/...........", "Fetching gem metadata from https://rails-assets.org/..", "Using rake 12.3.1", "Using concurrent-ruby 1.0.5", "Using i18n 0.9.1", "Using minitest 5.11.1", "Using thread_safe 0.3.6", "Using tzinfo 1.2.4", "Using activesupport 4.2.10", "Using builder 3.2.3", "Using erubis 2.7.0", "Using mini_portile2 2.3.0", "Fetching nokogiri 1.8.2", "Installing nokogiri 1.8.2 with native extensions", "Gem::Ext::BuildError: ERROR: Failed to build gem native extension.", "", " current directory: /var/lib/gems/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri", "/usr/bin/ruby2.3 -r ./siteconf20180627-6654-k2t6v1.rb extconf.rb", "mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h", "", "extconf failed, exit code 1", "", "Gem files will remain installed in /var/lib/gems/2.3.0/gems/nokogiri-1.8.2 for", "inspection.", "Results logged to", "/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out", "", "An error occurred while installing nokogiri (1.8.2), and Bundler cannot", "continue.", "Make sure that gem install nokogiri -v '1.8.2' succeeds before bundling.", "", "In Gemfile:", " graphiql-rails was resolved to 1.4.8, which depends on", " rails was resolved to 4.2.10, which depends on", " actionmailer was resolved to 4.2.10, which depends on", " actionpack was resolved to 4.2.10, which depends on", " actionview was resolved to 4.2.10, which depends on", " rails-dom-testing was resolved to 1.0.9, which depends on", " nokogiri"]}
to retry, use: --limit @/home/jesux/projects/installer/consul.retry

Generate a random secret token for each installation

What

After copying the file secrets.yml.example we should generate a random token for the production environment and use that as the secret token, instead of the default secret token that should be used only for development environments but it is temporarily used in the production environment too

Task Update rvm returs fatal

Hello, Im trying to deploy Consul on a vps.
While trying to run the playbook I get the following Error at the update rvm task:

TASK [galaxy/rvm_io.ruby : Update rvm] *****************************************
fatal: [149.210.191.114]: FAILED! => {"changed": false, "cmd": "/home/deploy/.rvm/bin/rvm get stable && /home/deploy/.rvm/bin/rvm reload", "delta": "0:00:00.233758", "end": "2018-10-16 13:08:11.563976", "msg": "non-zero return code", "rc": 1, "start": "2018-10-16 13:08:11.330218", "stderr": "Archives path '/usr/share/rvm/archives' not writable, aborting.", "stderr_lines": ["Archives path '/usr/share/rvm/archives' not writable, aborting."], "stdout": "", "stdout_lines": []}

My local machine runs Ubuntu 18.04.1 LTS
And my remote server runs: Ubuntu 16.04.5 LTS

I do not have direct Acces to the root of the remote server but I have Acces to an user with the same rights. So I adjusted the host file and the playbook to look like this:

Host:

[servers]
xxx.xxx.xxx.xxx ansible_ssh_extra_args='-o StrictHostKeyChecking=no' ansible_ssh_private_key_file='~/.ssh/id_rsa' ansible_port=22 ansible_ssh_user='consuladmin'

Playbook:

---

- hosts: all
  remote_user: consuladmin
  become: yes
  vars_files:
    - ./vars.yml
  roles:
    - postgresql

Any help?

Ansible & Ubuntu 18.04 incompatibilities

Original issue

We've found to incompatibilities between Ansible configuration and Ubuntu 18.04.

Default version of PostgreSQL in Ubuntu 18.04 is version 10, which raises the following error:

"UNENCRYPTED option is no longer supported in PG10"

This error has an easy solution, telling Ansible to encrypt user passwords in database

The other error raises when trying to install libcurl3, due to incompatibilities. Installing libcurl4 instead has worked fine so far for us.

Comments by @valdis

Error 1:

TASK [system : Add Node repository] ***************************************************************************************************
fatal: [157.230.119.69]: FAILED! => changed=false
  msg: 'W:GPG error: https://deb.nodesource.com/node_6.x bionic InRelease: The following signatures couldn''t be verified because the public key is not available: NO_PUBKEY 1655A0AB68576280, E:The repository ''https://deb.nodesource.com/node_6.x bionic InRelease'' is not signed.'

Manual solution:

root@ubuntu18-server:~# curl --silent https://deb.nodesource.com/gpgkey/nodesource.gpg.key | sudo apt-key add -

Error 2:

TASK [system : Install Tools] *********************************************************************************************************
fatal: [157.230.119.69]: FAILED! => changed=false
  cache_update_time: 1572695546
  cache_updated: true
  msg: |-
    '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"      install 'build-essential' 'zlib1g-dev' 'libssl-dev' 'libreadline-dev' 'libyaml-dev' 'libxml2-dev' 'libxslt1-dev' 'libffi-dev' 'libcurl3' 'libcurl4-openssl-dev' 'nodejs' 'libpq-dev' 'imagemagick' 'ruby-dev'' failed: E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
  rc: 100
  stderr: |-
    E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
  stderr_lines: <omitted>
  stdout: |-
    Reading package lists...
    Building dependency tree...
    Reading state information...
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    The following information may help to resolve the situation:

    The following packages have unmet dependencies:
     libcurl3 : Conflicts: libcurl4 but 7.58.0-2ubuntu3.8 is to be installed
     libcurl4 : Conflicts: libcurl3 but 7.58.0-2ubuntu2 is to be installed
  stdout_lines: <omitted>

Manual solution:

root@ubuntu18-server:~# apt-get remove libcurl4
root@ubuntu18-server:~# apt-get install libcurl3

Error 3:

TASK [postgresql : Create PostgreSQL database] ****************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'psycopg2'
fatal: [157.230.119.69]: FAILED! => changed=false
  msg: Failed to import the required Python library (psycopg2) on consul's Python /usr/bin/python3. Please read module documentation and install in the appropriate location

Manual solution:

root@ubuntu18-server:~# apt install python3-psycopg2

Error 4:

TASK [galaxy/rvm.ruby : Install rvm] **************************************************************************************************
fatal: [157.230.119.69]: FAILED! => changed=true
  cmd: |-
    /tmp/rvm-installer.sh stable --path ~/.rvm --auto-dotfiles --user-install
  delta: '0:00:00.046766'
  end: '2019-11-02 12:15:01.972168'
  msg: non-zero return code
  rc: 4
  start: '2019-11-02 12:15:01.925402'
  stderr: |-
    WARN: ...the preceeding error with code 200 occurred while fetching https://api.github.com/repos/rvm/rvm/tags
    WARN: ...the preceeding error with code 200 occurred while fetching https://api.bitbucket.org/2.0/repositories/mpapis/rvm/refs/tags?sort=-name&pagelen=20

    ERROR: Exhausted all sources trying to fetch version 'latest' of RVM!
  stderr_lines: <omitted>
  stdout: |-
    Turning on auto dotfiles mode.
    Turning on user install mode.
  stdout_lines: <omitted>

Manual solution:

root@ubuntu18-server:~# gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

Add to README how to install Ansible on Ubuntu

Why

Right now we only have documentation on how to install Ansible with brew, which is popular for mac users, but not very common for Linux users

What

Add documentation on how to setup Ansible for Linux, particularly Ubuntu to start with

How

Check out the @zenny's walkthrough as a reference 👌

Capistrano RVM task looks in root

Hello, not sure if I've configured the installer incorrectly, but Capistrano seems to be checking the /root/ directory for rvm:

cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as [email protected]: bundle exit status: 102
bundle stdout: Can not find rvm in '/root/.rvm'.
bundle stderr: Nothing written

I think this is in the deploy user's:

/home/deploy/.rvm

Any ideas?

Capistrano Taks Fails

Scenario, libs and versions :

Local Machine:
Ubuntu 16.04.4 LTS
ruby 2.3
usuario local: user
simplejson 3.8.1
ansible 2.5.4

Remote server:
Ubuntu 16.04.4 LTS
ruby 2.3
user: deploy (created by ansible )
python-simplejson 3.8.1
Ubuntu 16.04 x64

TASK [capistrano : Copy deploy.rb (locally)] ***************************************************************************************************************************************************************
fatal: [10.99.52.80]: FAILED! => {"changed": false, "checksum": "df14b6d8a11dd21c701813c32e1da2b6548c5e69", "dest": "/home/user/projects/installer/consul/config/deploy.rb", "gid": 1000, "group": "user", "mode": "0644", "msg": "chgrp failed: failed to look up group wheel", "owner": "deploy", "path": "/home/user/projects/installer/consul/config/deploy.rb", "size": 1922, "state": "file", "uid": 1001}
to retry, use: --limit @/home/user/projects/installer/consul.retry

PLAY RECAP *************************************************************************************************************************************************************************************************
10.99.52.80 : ok=47 changed=3 unreachable=0 failed=1

Add instructions to deploy from a fork

What

Update README with documentation/screencast on how to deploy the code in a fork

Cant rerun playbook

So regarding issue #57 I got stuck when running the playbook. I run the playbook with the method mentiond in issue #14. Now however I have another issue which keeps me from rerunning the playbook. On a fresh installed server I can run the playbook fine. But when I try to re run the playbook I get the following error:

TASK [unicorn : Kill running process] *****************************************************************************************************************************************************
changed: [127.0.0.1] => (item=22359)

TASK [unicorn : Start Unicorn] ************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "/home/deploy/.rvm/gems/ruby-2.3.2/wrappers/unicorn -c config/unicorn.rb -E production -D", "delta": "0:00:01.548114", "end": "2018-11-21 15:08:24.682874", "msg": "non-zero return code", "rc": 1, "start": "2018-11-21 15:08:23.134760", "stderr": "master failed to start, check stderr log for details", "stderr_lines": ["master failed to start, check stderr log for details"], "stdout": "", "stdout_lines": []}
	to retry, use: --limit @/root/installerold/consul.retry

However when I try to run the playbook for a third time it will work again. so it does for the fifth, seventh etc. But for the fourth and sixth time it gives me the error again. And so it goes on. All the even number of tries will fail. The only file I have changed is the group_vars/all.

Capistrano: "database.yml does not exist"

Trying out the new capistrano installer. Again, thank you for continuing to improve this awesome installer.

I'm seeing this:

00:02 deploy:check:linked_files
      ERROR linked file /var/www/consul/shared/config/database.yml does not exist

I don't know enough about the use of the shared folder. Did I miss a step? Thanks!

Add Unicorn init script

With the clean install there is not a unicorn init script, I think it is necessary for start Consul after reboot or to restart it easy.

How

This is an standar file to create a unicorn init script that could be used to make a template for Ansible and a task to deploy it.


### BEGIN INIT INFO
# Provides:          unicorn
# Required-Start:    $all
# Required-Stop:     $all
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the unicorn app server
# Description:       starts unicorn using start-stop-daemon
### END INIT INFO

set -e

USAGE="Usage: $0 <start|stop|restart|upgrade|rotate|force-stop>"

# app settings
USER="deploy"
APP_NAME="appname"
APP_ROOT="/home/$USER/$APP_NAME"
ENV="production"

# environment settings
PATH="/home/$USER/.rbenv/shims:/home/$USER/.rbenv/bin:$PATH"
CMD="cd $APP_ROOT && bundle exec unicorn -c config/unicorn.rb -E $ENV -D"
PID="$APP_ROOT/shared/pids/unicorn.pid"
OLD_PID="$PID.oldbin"

# make sure the app exists
cd $APP_ROOT || exit 1

sig () {
  test -s "$PID" && kill -$1 `cat $PID`
}

oldsig () {
  test -s $OLD_PID && kill -$1 `cat $OLD_PID`
}

case $1 in
  start)
    sig 0 && echo >&2 "Already running" && exit 0
    echo "Starting $APP_NAME"
    su - $USER -c "$CMD"
    ;;
  stop)
    echo "Stopping $APP_NAME"
    sig QUIT && exit 0
    echo >&2 "Not running"
    ;;
  force-stop)
    echo "Force stopping $APP_NAME"
    sig TERM && exit 0
    echo >&2 "Not running"
    ;;
  restart|reload|upgrade)
    sig USR2 && echo "reloaded $APP_NAME" && exit 0
    echo >&2 "Couldn't reload, starting '$CMD' instead"
    $CMD
    ;;
  rotate)
    sig USR1 && echo rotated logs OK && exit 0
    echo >&2 "Couldn't rotate logs" && exit 1
    ;;
  *)
    echo >&2 $USAGE
    exit 1
    ;;
esac```

Update ruby version to 2.3.3

What

COSNUL uses 2.3.3 but the playbook runs Ruby version 2.3.2
Make the playbook consistent with the current CONSUL version

Consolidate documentation

Right now, most of the documentation for production setup is in the README of this repository, while a bit lives in the documentation site. I suggest deciding on one place (for now - can change later), or at least linking between them.

500 | Internal server error on /valuation page

After deployment when I try tot access the valuation page in the admin panel I get a 500 internal server error.

Edit: This only happens when there are no participatory budget present.

Add instructions on how to copy a public ssh key to the remote server

What

Add instructions on how to copy a public ssh key to the remote server

How

Using ssh-copy-id

Use secrets.yml to store SMTP credentials

What

Update CONSUL's secrets.yml.example to include SMTP credentials
Update installer correspondingly to set this variables in the production server's secrets.yml

Why

Right now the installer is hardcoding these values in the production.rb file, but with a new deploy these values will be overwritten by the default production.rb file

Proposal wont go over HTTPS

Hey there,

I have Consul deployed on an Ubuntu server. I made the site secure with SSL with the use of certbot (Let's Encrypt certificate) The whole site is encrypted except for any of the proposal pages.
To be exact the following pages are not secure: /proposals/new, /legislation/processes/../proposals/new and /budgets/../investments

If I set [force_ssl] https://github.com/consul/consul/blob/master/config/environments/production.rb#L45) to true ill recieve an ERR_TOO_MANY_REDIRECTS error in browser.

these are my nginx conf files:
/etc/nginx/sites-enabled/default

upstream app {
        server unix:/home/deploy/consul/sockets/unicorn.sock;
}

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /home/deploy/consul/public;

        server_name _;

        try_files $uri/index.html $uri @app;
        location @app {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;
                proxy_pass http://app;
        }
}

server {

        root /home/deploy/consul/public;
    server_name mydomain.com; # managed by Certbot

        try_files $uri/index.html $uri @app;
        location @app {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;
                proxy_pass http://app;
        }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = consulemmen.ddns.net) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

        listen 80 ;
        listen [::]:80 ;
    server_name consulemmen.ddns.net;
    return 404; # managed by Certbot
}

/etc/nginx/sites-available/default

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /var/www/html;

        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;

        server_name _;

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }
}

Clarify workflow and differences between the Installer and Capistrano

Context

Some users are executing the Installer again to upload new versions of CONSUL to the production servers.

Objectives

Clarify in the documentation that the installer is only meant to me run once.
Future changes to the code in the production server should be done using Capistrano

Notes

CONSUL updates for new Releases should also be done using Capistrano.

However, we still have to decide how to deal with system updates. For the moment, system administrators are expected to do this job, but maybe in the future the Installer can handle upgrading system libraries.

ERROR 500 | Creating new budget investment proposal

Hey!,

Regarding issue #70 I have SSL working now but when creating a new proposal for a budget investment I receive a 500 error in browser. The new proposal is created anyway.

This is what I found in the log file:

W, [2019-01-17T09:42:28.265116 #23197]  WARN -- : [ActiveJob] [Rollbar] Got unexpected status code from Rollbar api: 400
W, [2019-01-17T09:42:28.484569 #23197]  WARN -- : [Rollbar] Got unexpected status code from Rollbar api: 400
F, [2019-01-17T09:42:28.488860 #23197] FATAL -- :
SocketError (getaddrinfo: Name or service not known):
  app/controllers/budgets/investments_controller.rb:62:in `create'

Add imagemagick

What

To upload images the required library should be installed: imagemagick

consuldemocracy / installer Goto Github PK

installer's Introduction

CONSUL DEMOCRACY

Documentation

CONSUL DEMOCRACY Foundation and project website

Configuration for development and test environments

Configuration for production environments

Current state

License

Contributions

installer's People

Contributors

Stargazers

Watchers

Forkers

installer's Issues

What

Why

How

Notes

What

What

Context

What

Why

How

What

Context

Objetives

Refereces

Context

Objectives

Context

What

Context

How to fix

What

What

Context

Objectives

How

What

What

Why

What

Context

What

What

Original issue

Comments by @valdis

Why

What

How

What

How

What

What

How

What

Why

Context

Objectives

Notes

What

Recommend Projects

Recommend Topics

Recommend Org