Omnibus GitLab
This project creates full-stack platform-specific downloadable packages for GitLab. For other installation options please see the GitLab installation page.
Canonical source
The source of omnibus-gitlab is hosted on GitLab.com and there are mirrors to make contributing as easy as possible.
Documentation version
Please make sure you are viewing the documentation for the version of omnibus-gitlab you are using. In most cases this should be the highest numbered stable branch (example shown below).
GitLab CI
To setup GitLab CI please see the separate GitLab CI documentation.
Configuration options
GitLab and GitLab CI are configured by setting their relevant options in
/etc/gitlab/gitlab.rb
. For a complete list of available options, visit the
gitlab.rb.template. New installations starting from GitLab 7.6, will have
all the options of the template listed in /etc/gitlab/gitlab.rb
by default.
Installation
Please follow the steps on the downloads page.
After installation
Run sudo gitlab-ctl status
; the output should look like this:
run: nginx: (pid 972) 7s; run: log: (pid 971) 7s
run: postgresql: (pid 962) 7s; run: log: (pid 959) 7s
run: redis: (pid 964) 7s; run: log: (pid 963) 7s
run: sidekiq: (pid 967) 7s; run: log: (pid 966) 7s
run: unicorn: (pid 961) 7s; run: log: (pid 960) 7s
If any of the processes is not behaving like expected, try tailing their logs to see what is wrong.
sudo gitlab-ctl tail postgresql
Your GitLab instance should reachable over HTTP at the IP or hostname of your server.
You can login as an admin user with username root
and password 5iveL!fe
.
Common installation problems
GitLab is unreachable in my browser
Try specifying an external_url
in
/etc/gitlab/gitlab.rb
. Also check your firewall settings; port 80 (HTTP) or
443 (HTTPS) might be closed on your GitLab server.
Emails are not being delivered
To test email delivery you can create a new GitLab account for an email that is not used in your GitLab instance yet.
If necessary, you can modify the 'From' field of the emails sent by GitLab with
the following setting in /etc/gitlab/gitlab.rb
:
gitlab_rails['gitlab_email_from'] = '[email protected]'
Run sudo gitlab-ctl reconfigure
for the change to take effect.
Reconfigure freezes at ruby_block[supervise_redis_sleep] action run
During the first gitlab-ctl reconfigure
run, omnibus-gitlab needs to figure
out if your Linux server is using SysV Init, Upstart or Systemd so that it can
install and activate the gitlab-runsvdir
service. If gitlab-ctl reconfigure
makes the wrong decision, it will later hang at
ruby_block[supervise_redis_sleep] action run
.
The choice of init system is currently made in the embedded Runit
cookbook by essentially
looking at the output of uname -a
, /etc/issue
and others. This mechanism
can make the wrong decision in situations such as:
- your OS release looks like 'Debian 7' but it is really some variant which uses Upstart instead of SysV Init;
- your OS release is unknown to the Runit cookbook (e.g. ClearOS 6.5).
Solving problems like this would require changes to the embedded Runit
cookbook; Merge Requests are welcome. Until this problem is fixed, you can work
around it by manually performing the appropriate installation steps for your
particular init system. For instance, to manually set up gitlab-runsvdir
with
Upstart, you can do the following:
sudo cp /opt/gitlab/embedded/cookbooks/runit/files/default/gitlab-runsvdir.conf /etc/init/
sudo initctl start gitlab-runsvdir
sudo gitlab-ctl reconfigure # Resume gitlab-ctl reconfigure
TCP ports for GitLab services are already taken
By default, Unicorn listens at TCP address 127.0.0.1:8080. Nginx listens on port 80 (HTTP) and/or 443 (HTTPS) on all interfaces.
The ports for Redis, PostgreSQL and Unicorn can be overriden in
/etc/gitlab/gitlab.rb
as follows:
redis['port'] = 1234
postgresql['port'] = 2345
unicorn['port'] = 3456
For Nginx port changes please see doc/settings/nginx.md.
Git SSH access stops working on SELinux-enabled systems
On SELinux-enabled systems the git user's .ssh
directory or its contents can
get their security context messed up. You can fix this by running sudo gitlab-ctl reconfigure
, which will run a chcon --recursive
command on
/var/opt/gitlab/.ssh
.
Postgres error 'FATAL: could not create shared memory segment: Cannot allocate memory'
The bundled Postgres instance will try to allocate 25% of total memory as
shared memory. On some Linux (virtual) servers, there is less shared memory
available, which will prevent Postgres from starting. In
/var/log/gitlab/postgresql/current
:
1885 2014-08-08_16:28:43.71000 FATAL: could not create shared memory segment: Cannot allocate memory
1886 2014-08-08_16:28:43.71002 DETAIL: Failed system call was shmget(key=5432001, size=1126563840, 03600).
1887 2014-08-08_16:28:43.71003 HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory or swap space, or exceeded your kernel's SHMALL parameter. You can either reduce the request size or reconfigure the kernel with larger SHMALL. To reduce the request size (currently 1126563840 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
1888 2014-08-08_16:28:43.71004 The PostgreSQL documentation contains more information about shared memory configuration.
You can manually lower the amount of shared memory Postgres tries to allocate
in /etc/gitlab/gitlab.rb
:
postgresql['shared_buffers'] = "100MB"
Run sudo gitlab-ctl reconfigure
for the change to take effect.
Reconfigure complains about the GLIBC version
$ gitlab-ctl reconfigure
/opt/gitlab/embedded/bin/ruby: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /opt/gitlab/embedded/lib/libruby.so.2.1)
/opt/gitlab/embedded/bin/ruby: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by /opt/gitlab/embedded/lib/libruby.so.2.1)
This can happen if the omnibus package you installed was built for a different OS release than the one on your server. Double-check that you downloaded and installed the correct omnibus-gitlab package for your operating system.
Reconfigure fails to create the git user
This can happen if you run sudo gitlab-ctl reconfigure
as the git user.
Switch to another user.
More importantly: do not give sudo rights to the git user or to any of the other users used by omnibus-gitlab. Bestowing unnecessary privileges on a system user weakens the security of your system.
Failed to modify kernel parameters with sysctl
If sysctl cannot modify the kernel parameters you could possibly get an error with the following stack trace:
* execute[sysctl] action run
================================================================================
Error executing action `run` on resource 'execute[sysctl]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '255'
---- Begin output of /sbin/sysctl -p /etc/sysctl.conf ----
This is unlikely to happen with non virtualized machines but on a VPS with virtualization like openVZ, container might not have the required module enabled or container doesn't have access to kernel parameters.
Try enabling the module on which sysctl errored out, on how to enable the module see example here.
There is a reported workaround described in this issue which requires editing the GitLab' internal recipe by supplying the switch which will ignore failures. Ignoring errors can have unexpected side effects on performance of your GitLab server so it is not recommended to do so.
Uninstalling omnibus-gitlab
To uninstall omnibus-gitlab, preserving your data (repositories, database, configuration), run the following commands.
# Stop gitlab and remove its supervision process
sudo gitlab-ctl uninstall
# Debian/Ubuntu
sudo dpkg -r gitlab
# Redhat/Centos
sudo rpm -e gitlab
To remove all omnibus-gitlab data use sudo gitlab-ctl cleanse
.
To remove all users and groups created by omnibus-gitlab, before removing the gitlab package (with dpkg or yum) run sudo gitlab-ctl remove_users
. Note All gitlab processes need to be stopped before runnign the command.
Updating
Instructions for updating your Omnibus installation and upgrading from a manual installation are in the update doc.
Starting and stopping
After omnibus-gitlab is installed and configured, your server will have a Runit
service directory (runsvdir
) process running that gets started at boot via
/etc/inittab
or the /etc/init/gitlab-runsvdir.conf
Upstart resource. You
should not have to deal with the runsvdir
process directly; you can use the
gitlab-ctl
front-end instead.
You can start, stop or restart GitLab and all of its components with the following commands.
# Start all GitLab components
sudo gitlab-ctl start
# Stop all GitLab components
sudo gitlab-ctl stop
# Restart all GitLab components
sudo gitlab-ctl restart
Note that on a single-core server it may take up to a minute to restart Unicorn and Sidekiq. Your GitLab instance will give a 502 error until Unicorn is up again.
It is also possible to start, stop or restart individual components.
sudo gitlab-ctl restart sidekiq
Unicorn supports zero-downtime reloads. These can be triggered as follows:
sudo gitlab-ctl hup unicorn
Note that you cannot use a Unicorn reload to update the Ruby runtime.
Configuration
Backup and restore omnibus-gitlab configuration
All configuration for omnibus-gitlab is stored in /etc/gitlab
. To backup your
configuration, just backup this directory.
# Example backup command for /etc/gitlab:
# Create a time-stamped .tar file in the current directory.
# The .tar file will be readable only to root.
sudo sh -c 'umask 0077; tar -cf $(date "+etc-gitlab-%s.tar") -C / etc/gitlab'
You can extract the .tar file as follows.
# Rename the existing /etc/gitlab, if any
sudo mv /etc/gitlab /etc/gitlab.$(date +%s)
# Change the example timestamp below for your configuration backup
sudo tar -xf etc-gitlab-1399948539.tar -C /
Remember to run sudo gitlab-ctl reconfigure
after restoring a configuration
backup.
NOTE: Your machines SSH host keys are stored in a separate location at /etc/ssh/
. Be sure to also backup and restore those keys to avoid man-in-the-middle attack warnings if you have to perform a full machine restore.
Configuring the external URL for GitLab
In order for GitLab to display correct repository clone links to your users
it needs to know the URL under which it is reached by your users, e.g.
http://gitlab.example.com
. Add or edit the following line in
/etc/gitlab/gitlab.rb
:
external_url "http://gitlab.example.com"
Run sudo gitlab-ctl reconfigure
for the change to take effect.
Storing Git data in an alternative directory
By default, omnibus-gitlab stores Git repository data under
/var/opt/gitlab/git-data
: repositories are stored in
/var/opt/gitlab/git-data/repositories
, and satellites in
/var/opt/gitlab/git-data/gitlab-satellites
. You can change the location of
the git-data
parent directory by adding the following line to
/etc/gitlab/gitlab.rb
.
git_data_dir "/mnt/nas/git-data"
Run sudo gitlab-ctl reconfigure
for the change to take effect.
If you already have existing Git repositories in /var/opt/gitlab/git-data
you
can move them to the new location as follows:
# Prevent users from writing to the repositories while you move them.
sudo gitlab-ctl stop
# Only move 'repositories'; 'gitlab-satellites' will be recreated
# automatically. Note there is _no_ slash behind 'repositories', but there _is_ a
# slash behind 'git-data'.
sudo rsync -av /var/opt/gitlab/git-data/repositories /mnt/nas/git-data/
# Fix permissions if necessary
sudo gitlab-ctl reconfigure
# Double-check directory layout in /mnt/nas/git-data. Expected output:
# gitlab-satellites repositories
sudo ls /mnt/nas/git-data/
# Done! Start GitLab and verify that you can browse through the repositories in
# the web interface.
sudo gitlab-ctl start
Changing the name of the Git user / group
By default, omnibus-gitlab uses the user name git
for Git gitlab-shell login,
ownership of the Git data itself, and SSH URL generation on the web interface.
Similarly, git
group is used for group ownership of the Git data. You can
change the user and group by adding the following lines to
/etc/gitlab/gitlab.rb
.
user['username'] = "gitlab"
user['group'] = "gitlab"
Run sudo gitlab-ctl reconfigure
for the change to take effect.
Setting up LDAP sign-in
See doc/settings/ldap.md.
Enable HTTPS
Redirect HTTP
requests to HTTPS
.
Change the default port and the ssl certificate locations.
Use non-bundled web-server
Adding ENV Vars to the Gitlab Runtime Environment
See doc/settings/environment-variables.md.
Changing gitlab.yml settings
See doc/settings/gitlab.yml.md.
Specify numeric user and group identifiers
Omnibus-gitlab creates users for GitLab, PostgreSQL, Redis and NGINX. You can
specify the numeric identifiers for these users in /etc/gitlab/gitlab.rb
as
follows.
user['uid'] = 1234
user['gid'] = 1234
postgresql['uid'] = 1235
postgresql['gid'] = 1235
redis['uid'] = 1236
redis['gid'] = 1236
web_server['uid'] = 1237
web_server['gid'] = 1237
Sending application email via SMTP
See doc/settings/smtp.md.
Omniauth (Google, Twitter, GitHub login)
Omniauth configuration is documented on
doc.gitlab.com. To effect
the necessary changes in gitlab.yml
, use the following syntax in
/etc/gitlab/gitlab.rb
. Note that the providers are specified as an array of
Ruby hashes.
gitlab_rails['omniauth_enabled'] = true
gitlab_rails['omniauth_providers'] = [
{
"name" => "google_oauth2",
"app_id" => "YOUR APP ID",
"app_secret" => "YOUR APP SECRET",
"args" => { "access_type" => "offline", "approval_prompt" => "" }
}
]
Adjusting Unicorn settings
Setting the NGINX listen address or addresses
Inserting custom NGINX settings into the GitLab server block
Inserting custom settings into the NGINX config
Backups
Creating an application backup
To create a backup of your repositories and GitLab metadata, run the following command.
Note that GitLab CI currently does not have a backup script.
sudo gitlab-rake gitlab:backup:create
This will store a tar file in /var/opt/gitlab/backups
. The filename will look like
1393513186_gitlab_backup.tar
, where 1393513186 is a timestamp.
If you want to store your GitLab backups in a different directory, add the
following setting to /etc/gitlab/gitlab.rb
and run sudo gitlab-ctl reconfigure
:
gitlab_rails['backup_path'] = '/mnt/backups'
Upload backups to remote (cloud) storage
For details check backup restore document of GitLab CE.
Scheduling a backup
To schedule a cron job that backs up your repositories and GitLab metadata, use the root user:
sudo su -
crontab -e
There, add the following line to schedule the backup for everyday at 2 AM:
0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create
You may also want to set a limited lifetime for backups to prevent regular
backups using all your disk space. To do this add the following lines to
/etc/gitlab/gitlab.rb
and reconfigure:-
# limit backup lifetime to 7 days - 604800 seconds
gitlab_rails['backup_keep_time'] = 604800
Restoring an application backup
We will assume that you have installed GitLab from an omnibus package and run
sudo gitlab-ctl reconfigure
at least once.
First make sure your backup tar file is in /var/opt/gitlab/backups
.
sudo cp 1393513186_gitlab_backup.tar /var/opt/gitlab/backups/
Next, restore the backup by running the restore command. You need to specify the timestamp of the backup you are restoring.
# Stop processes that are connected to the database
sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
# This command will overwrite the contents of your GitLab database!
sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186
# Start GitLab
sudo gitlab-ctl start
If there is a GitLab version mismatch between your backup tar file and the installed version of GitLab, the restore command will abort with an error. Install a package for the required version and try again.
Invoking Rake tasks
To invoke a GitLab Rake task, use gitlab-rake
(for GitLab) or
gitlab-ci-rake
(for GitLab CI). For example:
sudo gitlab-rake gitlab:check
sudo gitlab-ci-rake -T
Contrary to with a traditional GitLab installation, there is no need to change
the user or the RAILS_ENV
environment variable; this is taken care of by the
gitlab-rake
and gitlab-ci-rake
wrapper scripts.
Directory structure
Omnibus-gitlab uses four different directories.
/opt/gitlab
holds application code for GitLab and its dependencies./var/opt/gitlab
holds application data and configuration files thatgitlab-ctl reconfigure
writes to./etc/gitlab
holds configuration files for omnibus-gitlab. These are the only files that you should ever have to edit manually./var/log/gitlab
contains all log data generated by components of omnibus-gitlab.
Omnibus-gitlab and SELinux
Although omnibus-gitlab runs on systems that have SELinux enabled, it does not use SELinux confinement features:
- omnibus-gitlab creates unconfined system users;
- omnibus-gitlab services run in an unconfined context.
The correct operation of Git access via SSH depends on the labeling of
/var/opt/gitlab/.ssh
. If needed you can restore this labeling by running
sudo gitlab-ctl reconfigure
.
Depending on your platform, gitlab-ctl reconfigure
will install SELinux
modules required to make GitLab work. These modules are listed in
files/gitlab-selinux/README.md.
NSA, if you're reading this, we'd really appreciate it if you could contribute back a SELinux profile for omnibus-gitlab :) Of course, if anyone else is reading this, you're welcome to contribute the SELinux profile too.
Logs
Tail logs in a console on the server
If you want to 'tail', i.e. view live log updates of GitLab logs you can use
gitlab-ctl tail
.
# Tail all logs; press Ctrl-C to exit
sudo gitlab-ctl tail
# Drill down to a sub-directory of /var/log/gitlab
sudo gitlab-ctl tail gitlab-rails
# Drill down to an individual file
sudo gitlab-ctl tail nginx/gitlab_error.log
Runit logs
The Runit-managed services in omnibus-gitlab generate log data using svlogd. See the svlogd documentation for more information about the files it generates.
You can modify svlogd settings via /etc/gitlab/gitlab.rb
with the following settings:
# Below are the default values
logging['svlogd_size'] = 200 * 1024 * 1024 # rotate after 200 MB of log data
logging['svlogd_num'] = 30 # keep 30 rotated log files
logging['svlogd_timeout'] = 24 * 60 * 60 # rotate after 24 hours
logging['svlogd_filter'] = "gzip" # compress logs with gzip
logging['svlogd_udp'] = nil # transmit log messages via UDP
logging['svlogd_prefix'] = nil # custom prefix for log messages
# Optionally, you can override the prefix for e.g. Nginx
nginx['svlogd_prefix'] = "nginx"
Logrotate
Starting with omnibus-gitlab 7.4 there is a built-in logrotate service in
omnibus-gitlab. This service will rotate, compress and eventually delete the
log data that is not captured by Runit, such as gitlab-rails/production.log
and nginx/gitlab_access.log
. You can configure logrotate via
/etc/gitlab/gitlab.rb
.
# Below are some of the default settings
logging['logrotate_frequency'] = "daily" # rotate logs daily
logging['logrotate_size'] = nil # do not rotate by size by default
logging['logrotate_rotate'] = 30 # keep 30 rotated logs
logging['logrotate_compress'] = "compress" # see 'man logrotate'
logging['logrotate_method'] = "copytruncate" # see 'man logrotate'
logging['logrotate_postrotate'] = nil # no postrotate command by default
# You can add overrides per service
nginx['logrotate_frequency'] = nil
nginx['logrotate_size'] = "200M"
# You can also disable the built-in logrotate service if you want
logrotate['enable'] = false
UDP log shipping (GitLab Enterprise Edition only)
You can configure omnibus-gitlab to send syslog-ish log messages via UDP.
logging['udp_log_shipping_host'] = '1.2.3.4' # Your syslog server
logging['udp_log_shipping_port'] = 1514 # Optional, defaults to 514 (syslog)
Example log messages:
<13>Jun 26 06:33:46 ubuntu1204-test production.log: Started GET "/root/my-project/import" for 127.0.0.1 at 2014-06-26 06:33:46 -0700
<13>Jun 26 06:33:46 ubuntu1204-test production.log: Processing by ProjectsController#import as HTML
<13>Jun 26 06:33:46 ubuntu1204-test production.log: Parameters: {"id"=>"root/my-project"}
<13>Jun 26 06:33:46 ubuntu1204-test production.log: Completed 200 OK in 122ms (Views: 71.9ms | ActiveRecord: 12.2ms)
<13>Jun 26 06:33:46 ubuntu1204-test gitlab_access.log: 172.16.228.1 - - [26/Jun/2014:06:33:46 -0700] "GET /root/my-project/import HTTP/1.1" 200 5775 "https://172.16.228.169/root/my-project/import" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
2014-06-26_13:33:46.49866 ubuntu1204-test sidekiq: 2014-06-26T13:33:46Z 18107 TID-7nbj0 Sidekiq::Extensions::DelayedMailer JID-bbfb118dd1db20f6c39f5b50 INFO: start
2014-06-26_13:33:46.52608 ubuntu1204-test sidekiq: 2014-06-26T13:33:46Z 18107 TID-7muoc RepositoryImportWorker JID-57ee926c3655fcfa062338ae INFO: start
Starting a Rails console session
If you need access to a Rails production console for your GitLab installation you can start one with the command below. Please be warned that it is very easy to inadvertently modify, corrupt or destroy data from the console.
# start a Rails console for GitLab
sudo gitlab-rails console
# start a Rails console for GitLab CI
sudo gitlab-ci-rails console
This will only work after you have run gitlab-ctl reconfigure
at least once.
Using a MySQL database management server (Enterprise Edition only)
Create a user and database for GitLab
Configure omnibus-gitlab to connect to it
Seed the database (fresh installs only)
Using a non-packaged PostgreSQL database management server
Using a non-packaged Redis instance
Only start omnibus-gitlab services after a given filesystem is mounted
If you want to prevent omnibus-gitlab services (nginx, redis, unicorn etc.)
from starting before a given filesystem is mounted, add the following to
/etc/gitlab/gitlab.rb
:
# wait for /var/opt/gitlab to be mounted
high_availability['mountpoint'] = '/var/opt/gitlab'
Using an existing Passenger/Nginx installation
Building your own package
See the separate build documentation.
Running a custom GitLab version
It is not recommended to make changes to any of the files in /opt/gitlab
after installing omnibus-gitlab: they will either conflict with or be
overwritten by future updates. If you want to run a custom version of GitLab
you can build your own package or use another installation
method.
Acknowledgments
This omnibus installer project is based on the awesome work done by Chef in omnibus-chef-server.