Giter VIP home page Giter VIP logo

eqpress's Introduction

Installing

A host with a working Ansible installation is required. Read this:

http://docs.ansible.com/ansible/intro_installation.html

For Debian 9+

apt-get install ansible

Clone this repo:

git clone https://github.com/equalitie/eqpress.git

Environment Initialization

Initializing the ansible environment is required before any other playbooks can be executed. Change into the equpress directory, create the inventory file and then run the init playbook:

cd eqpress 
touch hosts
vi hosts (add your target hosts including localhost)

Example inventory

[local]
localhost ansible_connection=local

[production]
eqwp1.equalit.ie
eqwp2.equalit.ie

[masters]
eqwp1.equalit.ie

[slaves]
eqwp2.equalit.ie

[eqwp]
eqwp1.equalit.ie
eqwp2.equalit.ie

Play to run

ansible-playbook -i hosts play-init-env.yml -v

The following settings can be specified. Accepting the defaults are enough to configure the environment so the other playbooks will work but making them unique to your environment is best.

SSL/TLS certificate attributes are required to generate the self-signed certificates used for MySQL replication.

  • Enter organization name
  • Root certificate country
  • Root certificate state/province
  • Root certificate city
  • Root certificate orginazational unit
  • Root certificate common name
  • Root certificate email address

Mandrill and Sendgrid are email delivery services offering free accounts for moderate levels of email traffic. These are not required but recommended for reliable email delivery.

  • Mandrill username - Sign up for an account here. Free plan allows 12,000 sent emails per month.
  • Mandrill password
  • Sendgrid username - Sign up for a free account here. Free plan allows 400 sent emails per day.
  • Sendgrid password
  • Default email service
  • Monitoring email address - Where all alerts are sent to.
  • Timezone
  • Use DNS lookup for host IP assignment - y or n

"y" if you want to use dns lookup for IP and "n" to use ansible gathered facts for IP

Note: If your VM or server has a private interal IP address and a network server supplies your public IP address (floating IP), choose "y" or your IP addresses will be the private addresses and in most cases will not work.

Server Configuration Initialization

To build a redundant pair of servers there are some ansible variables that need to be set for the playbooks to work. Run the initialization playbook to create the group and host variables:

ansible-playbook -i hosts play-init-servers.yml -v

The following settings must be specified:

  • Nginx worker processes - Should equal c - 2 where c is number of CPU cores. If c is < 4 then worker procs should equal 2 .
  • PHP-FPM max children - default is typically fine
  • PHP-FPM start servers - default is typically fine
  • PHP-FPM min spare - default is typically fine
  • PHP-FPM max spare - default is typically fine
  • PHP-FPM max requests - default prevents processes from eating too much RAM. Increase to 64 if server is very busy.
  • PHP-FPM opcache memory size - increase default if more than 20 sites are hosted on the same server
  • MySQL root user password - click here for long random strings
  • MySQL InnoDB buffer pool size - default good for servers with RAM <= 1GB. Set to 1536M for servers with 4GB RAM. Don't forget the K, M or G after the number
  • MySQL InnoDB log file size - default is fine for servers < 4GB RAM
  • **MySQL replication user password - click here for long random strings
  • MySQL Admin user password - mysqladmin user has process rights for monitoring replication status. click here for long random strings
  • MySQL webstats user password - webstats user writes to webstats DB to store HTTP access log data. click here for long random strings
  • MySQL Server ID for master - must be unique, don't accept the default
  • MySQL Server ID for slave - must be unique, don't accept the default
  • Master server hostname - using a fully qualified domain name is best.
  • Slave server hostname - using a fully qualified domain name is best.
  • Ansible group name - the group that these hosts will be uniquely identified by within the hosts file and variables stored in a file in the group_vars directory

Building a Replicated Pair of Servers

Minimum Requirements

Managed Nodes
  • The servers that will be used for creating the replicated pair must be running Debian 7 (Wheezy).
  • The debian packages python and python-simplejson must be installed for ansible to work.

Play to run

ansible-playbook -i hosts play-fullstack.yml -u root -l eqwp

When the servers are ready for production then it's time to deploy the cron jobs

ansible-playbook -i hosts play-go-live.yml -u root -l eqwp

Manual Host and Group Configuration

You can build the host and group files manually instead of running the play-init-servers.yml playbook. Create an alias in the ansible hosts file with the names of the new server pairs below:

[eqpress-test]
eqpress-test1.equalit.ie  
eqpress-test2.equalit.ie

Create a group YAML file in the group_vars directory (copy an existing one). Name the group file the same as the alias entered in the ansible hosts file:

group_vars/eqpress-test.yml

Edit this new group file and minimally change the following variables:

easypress_server_id
mysql_root_db_pass
mysql_repl_creds: password
mysql_admin_user: password
mysql_webstats: password

Create a host YAML file in the host_vars directory (copy an existing primary and replica). Name the host files the same as what was entered in the ansible hosts file:

host_vars/eqpress-test1.equalit.ie.yml
host_vars/eqpress-test2.equalit.ie.yml

Edit these new host files and minmally change the following variables:

mysql_server_id
mysql_repl_slave
mysql_repl_master

Ansible Plays

  • ansible-playbook -i hosts play-fullstack.yml -u root -l eqpress-test

When the servers are ready for production then it's time to deploy the cron jobs

  • ansible-playbook -i hosts play-go-live.yml -u root -l eqpress-test

Common Failures and Remedies

  • MySQL fails to start
    1. Check if mysql is running on the host
    2. Re-run the play using the mysql or slaveon tag
      ansible-playbook -i hosts play-fullstack.yml -u root -l eqpress-test --tags slaveon
      ansible-playbook -i hosts play-fullstack.yml -u root -l eqpress-test --tags mysql

Role based updates

Update nginx and php-fpm config

  • ansible-playbook -i hosts play-fullstack.yml -u root -l eph --tags="nginx,php"

Console

Deploy changes to the easyPress Console must-use plugin and proxy code

  • ansible-playbook -i hosts play-fullstack.yml -u root -v -l production --tags console

Deploy easyPress console must-use plugin to all sites

  • ansible -i hosts masters -m command -a "/usr/local/sbin/ep_install_console.sh all" -u root

Testing New Console Code

ansible-playbook -i hosts play-fullstack.yml -u root -l jester.easypress.ca --tags console && ansible -i hosts jester.easypress.ca -m command -a "/usr/local/sbin/ep_install_console.sh wtj.boreal321.com" -u root

Add or Update System Users

  • ansible-playbook -i hosts play-fullstack.yml -u root -l production --tags users
  • ansible-playbook -i hosts play-add-user.yml -u root -l eqpress-test1.boreal321.com

Update nginx configs

All configs and reload nginx

  • ansible-playbook -i hosts play-fullstack.yml -u root -l masters --tags nginx_config

WordPress and plugin updates

Update a specific plugin on all master servers

  • ansible -i hosts masters -m command -a "/usr/local/sbin/wp_update_plugins.sh wordpress-seo" -u root -v

###Auto-Provision Documentation

eqpress's People

Contributors

abrahambosch avatar acataluddi avatar boreal321 avatar eqpress avatar graphiclunarkid avatar jeremy5189 avatar kheops2713 avatar kromcuich avatar uffhitech avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eqpress's Issues

Some playbooks do not use the configurable settings

play-init-provision.yml asks for "auto_provision_base_directory" setting, but it is not use in the main.yml task for the auto-provision role, which uses hardcoded /var/www/easypress.ca/provision. Is this normal?

Monit is installed twice

Monit is installed as a Debian package in roles/common/tasks/main.yml, but also in its binary form from the repo, to /usr/local/bin/. This is maybe a bit confusing.

Monit seems to create zombie processes

A quick look on an already-installed system showed several zombie processes created by monit:

root      5928  1.2  0.0      0     0 ?        Zs   10:14   0:00 [monit_ep_consol] <defunct>
root      5995  0.5  0.0      0     0 ?        Zs   10:14   0:00 [monit_ep_consol] <defunct>
root      6063  0.5  0.0      0     0 ?        Zs   10:14   0:00 [monit_connectio] <defunct>
root      6131  0.5  0.0      0     0 ?        Zs   10:14   0:00 [monit_check_mai] <defunct>

Some more digging may be required to reproduce this and understand how it happens.

/var/www/easypress.ca/q8z94biV2aEUQg6Zrc7yBCp.php does not have a random filename

This file's name is designed to be random in order to avoid being queried from outside. However as of now its name is hardcoded in the repo.

Instead, a random filename could be generated and stored in a local configuration file when executing a play-init-* playbook, and then used when actually deploying the servers. Some jinja2 code can be written to generate random strings. Would this be a good approach?

mysql-percona root password cannot be changed/reset

The mysql-percona-replication role sets the root password, but the task will fail if the root password has already been set.

Maybe debconf could be used to set the root password no matter its previous value.

nginx cache.conf file overwritten by play-fullstack.yml

files/etc/nginx/cache.conf is a dynamic file that is recreated on ever new site. Running the full play-fullstack.yml fails if you have live sites because of the references to the fastcgi_cache zone that get added to this file.

Not sure is we should split this file to cache_common.conf and cache.conf.

Needs looked into so we can run full play-fullstack.yml playbook without breaking the system.

Refactoring of "easypress.ca"

Hello @Boreal,
I'd like to improve the Eqpress API. For for what is my understanding, currently the /var/www/easypress.ca contains the deployment API.
To make the code and platform more undertandable, I suggest and would like to refactor the easypress.ca in something like eqpress-api.

Is my assumptions correct and do you agree with the refactoring?

Thanks
Adriano

Alternative ways of providing Monit

It may be possible to avoid including the monit binary in the git repo:

  • if the wheezy-backports version of monit (5.6) is recent enough for what we want
  • by downloading the precompiled binary from the official website
  • by downloading and compiling the source on the managed host, as it seems Monit's only notable dependancy is openssl

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.