aimkiray / blog Goto Github PK

🍩 Taste with Gitlab CI and Docker

0. Prepare the system environment

0.1. Configuring Proxy Server Access

export http_proxy=http://your_proxy_ip:port
export https_proxy=http://your_proxy_ip:port
export ALL_PROXY=http://your_proxy_ip:port

Optional: To enable all yum operations to use a proxy server, specify the proxy server details in /etc/yum.conf. The proxy setting must specify the proxy server as a complete URL, including the TCP port number. If your proxy server requires a username and password, specify these by adding proxy_username and proxy_password settings.

# The proxy server - proxy server:port number
proxy=http://your_proxy_ip:port
# The account details for yum connections (No need here)
# proxy_username=yum-user
# proxy_password=qwerty

0.2. Replace rhel mirror with centos mirror

First backup /etc/yum.repos.d/rhel-7-x86_64.repo

mv /etc/yum.repos.d/rhel-7-x86_64.repo /etc/yum.repos.d/rhel-7-x86_64.repo.backup

Download the corresponding version of the repo file and put it in /etc/yum.repos.d/

• CentOS7
• CentOS6

Run the following command to generate the cache

yum clean all
yum makecache

1. Install docker first

Only Centos 7 and above is supported (linux kernel version >= 3.10)

1.1. Set up the repository

Install required packages. yum-utils provides the yum-config-manager utility, and device-mapper-persistent-data and lvm2 are required by the devicemapper storage driver.

$ sudo yum install -y yum-utils \
  device-mapper-persistent-data \
  lvm2

Use the following command to set up the stable repository. You always need the stable repository, even if you want to install builds from the edge or test repositories as well.

$ sudo yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

Optional: Enable the edge and test repositories. These repositories are included in the docker.repo file above but are disabled by default. You can enable them alongside the stable repository.

$ sudo yum-config-manager --enable docker-ce-edge

$ sudo yum-config-manager --enable docker-ce-test

1.2. Install docker ce

Install the latest version of Docker CE, or go to the next step to install a specific version:

$ sudo yum install docker-ce

Restart the Docker service:

systemctl restart docker
# or
service docker stop

Then move the entire /var/lib/docker directory to the destination path:

mkdir /opt/data/docker
mv /var/lib/docker /opt/data/docker
ln -s /opt/data/docker /var/lib/docker

2. Use Docker China Mirror

Centos 6

$ vim /etc/sysconfig/docker

Add a row:

other_args="--registry-mirror=https://registry.docker-cn.com"

Centos 7

$ vim /etc/docker/daemon.json

After adding:

{
    "registry-mirrors": ["https://registry.docker-cn.com"]
}

or

{
  "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"]
}

Restart the Docker service:

systemctl restart docker
# or
service docker restart

3. Let docker use the proxy

Centos 7

First, create a systemd drop-in directory for the docker service:

mkdir /etc/systemd/system/docker.service.d

Now create a file called /etc/systemd/system/docker.service.d/http-proxy.conf that adds the HTTP_PROXY environment variable:

[Service]
Environment="HTTP_PROXY=http://your_proxy_ip:port"
Environment="NO_PROXY=localhost,127.0.0.0"

Flush changes:

$ sudo systemctl daemon-reload

Verify that the configuration has been loaded:

$ sudo systemctl show --property Environment docker
Environment=HTTP_PROXY=http://your_proxy_ip:port

Restart Docker:

$ sudo systemctl restart docker

Centos 6

vim /etc/sysconfig/docker
# insert
export http_proxy=http://your_proxy_ip:port
export https_proxy=http://your_proxy_ip:port
export NO_PROXY=localhost,127.0.0.0
# restart
service docker restart

4. Create a local docker mirror repository

Install and run docker-registry

$ docker run -d -p 5000:5000 --name registry registry:2

This will use the official registry image to start the private repository. By default, the repository is created in the container's /var/lib/registry directory. , You can use the -v parameter to store the image in a local path. For example, the following example places the uploaded image in the local /opt/data/registry directory.

$ docker run -d --name 02-registry --restart always \
    -p 5000:5000 \
    -v /opt/data/registry:/var/lib/registry \
    --name registry \
    registry:2

5. Installing Gitlab CI based on a docker image

Pull the latest version:

docker pull gitlab/gitlab-runner:latest

You need to mount a config volume into the gitlab-runner container to be used for configs and other resources:

docker run -d --name table-runner --restart always \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

Only registered runners can connect to Gitlab CI Server, so a registration process is required:

docker exec -it gitlab-runner gitlab-runner register

Then it will ask for some necessary authentication information, as follows:

Running in system-mode.

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://your_gitlab_ip/
Please enter the gitlab-ci token for this runner:
xxxxxxxxxxxxxxx
Please enter the gitlab-ci description for this runner:
[xxxxxxxx]: docker-runner
Please enter the gitlab-ci tags for this runner (comma separated):
table
Whether to run untagged builds [true/false]:
[false]:
Whether to lock the Runner to current project [true/false]:
[true]: false
Registering runner... succeeded                     runner=xxxxxx
Please enter the executor: docker, shell, virtualbox, kubernetes, docker-ssh, parallels, ssh, docker+machine, docker-ssh+machine:
docker
Please enter the default Docker image (e.g. ruby:2.1):
maven:3.5.3-jdk-8
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

One-line registration command:

docker exec -it table-runner gitlab-runner register \
  --non-interactive \
  --executor "docker" \
  --docker-image docker \
  --url "https://your_gitlab_ip/" \
  --registration-token "xxxxxxxxxxxxxx" \
  --description "docker-runner" \
  --tag-list "master" \
  --run-untagged false \
  --locked="true"

You can find them in your project page, Settings -> Pipeline -> Specific Runners

If successful, you can see the Runner just configured.

6. Let gitlab-runner use the proxy

Centos 7 & 6

You need to edit /srv/gitlab-runner/config/config.toml and add the following to the [[runners]] section:

environment = ["HTTPS_PROXY=http://your_proxy_ip:port", "HTTP_PROXY=http://your_proxy_ip:port", "NO_PROXY=localhost,127.0.0.0"]

Centos 7 (Not recommended)

Create a systemd drop-in directory for the gitlab-runner service:

mkdir /etc/systemd/system/gitlab-runner.service.d

Create a file called /etc/systemd/system/gitlab-runner.service.d/http-proxy.conf that adds the HTTP_PROXY environment variable(s):

[Service]
Environment="HTTP_PROXY=http://your_proxy_ip:port"
Environment="HTTPS_PROXY=http://your_proxy_ip:port"

Save the file and flush changes:

systemctl daemon-reload

Restart GitLab Runner:

sudo systemctl restart gitlab-runner

7. Using the host maven repository

Each time the build is in a separate container, maven needs to download the dependent jar from the maven repository again. This is actually not necessary.

We can use docker's volume to share .m2 files among multiple containers.

In addition, each time the runner will pull the dependent image on the docker hub, this is not necessary too.

From this we need to make some changes, to open the /srv/gitlab-runner/config/config.toml file. Add the maven .m2 directory to volumes and add mirror pull rules, as follows

concurrent = 1
check_interval = 0

[[runners]]
  name = "docker-runner"
  url = "https://your_gitlab_ip/"
  token = "xxxxxxxxxxxxxxxxxxxxxxx"
  executor = "docker"
  environment = ["HTTPS_PROXY=http://your_proxy_ip:port", "HTTP_PROXY=http://your_proxy_ip:port", "NO_PROXY=localhost,127.0.0.0"]
  [runners.docker]
    tls_verify = false
    image = "docker"
    privileged = true
    disable_cache = false
    shm_size = 0
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock", "/usr/.m2:/root/.m2"]
    pull_policy = "if-not-present"
  [runners.cache]

Restart runner

docker restart gitlab-runner

8. Install Kubernetes

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

Kubernetes relies on the etcd service to maintain the status of all primary nodes.

8.1. Start the Etcd service

docker run --net=host -d gcr.io/google_containers/etcd:v2.3.8 /usr/local/bin/etcd --addr=127.0.0.1:4001 --bind-addr=0.0.0.0:4001 --data-dir=/var/etcd/data

8.2. Start the master node (kubelet)

docker run --net=host -d -v /var/run/docker.sock:/var/run/docker.sock  gcr.io/google_containers/hyperkube:latest /hyperkube kubelet --api_servers=http://localhost:8080 --v=2 --address=0.0.0.0 --enable_server --hostname_override=127.0.0.1 --config=/etc/kubernetes/manifests

8.3. Start Service Agent

docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:latest /hyperkube proxy --master

9. Configuration of your jobs with .gitlab-ci.yml

See https://docs.gitlab.com/ee/ci/yaml/

10. Start with gitlab CI/CD

Push the modified code to gitlab and see the build process in the project's pipeline.

11. Dockerfile

12. Package the project into a Docker image

FAQ

Batch start Exited docker container

docker start $(docker ps -a | awk '{ print $1}' | tail -n +2)

Device or resource busy

cat /proc/mounts | grep "mapper/docker" | awk '{print $2}' | xargs umount

kill defunct process

ps -ef | grep defunct | grep -v grep | awk '{print $2}' | xargs kill -9

remove exist container

docker ps -a | grep table-sandbox | awk '{print $1}' | xargs docker stop | xargs docker rm

docker log

docker logs --tail 50 --follow --timestamps container_name