Giter VIP home page Giter VIP logo

ska-docker's Introduction

TANGO Docker containers

This project defines a set of Docker images and Docker compose files that are useful for TANGO control system development. See the documentation in the 'docs' folder for build and usage instructions.

Docker hierarchy and release tagging

When updating Dockerfiles, and especially the tags in the .release files, it is useful to know the hierarchy. All downstream images must have the release tags updated.

The release tags should match the underlying dependencies used. E.g., for the first release of cppTango 9.3.4-rc4, use the tag 9.3.4-rc4. If there are subsequent modifications to a Dockerfile, but still using that cppTango release, add a suffix, e.g., 9.3.4-rc4.1. Further changes would then incremenent that suffix: 9.3.4-rc4.2, etc.

  • ubuntu
    • deploy/Dockerfile:FROM ubuntu:18.04
    • tango-builder/Dockerfile:FROM ubuntu:18.04
  • debian-buster-slim
    • tango-dependencies/Dockerfile:FROM debian:buster-slim as buildenv
    • tango-dependencies/Dockerfile:FROM debian:buster-slim
      • tango-java/Dockerfile:FROM {nexus}/tango-dependencies:latest
        • hdbpp_viewer/Dockerfile:FROM {nexus}/tango-java:latest
        • tango-jive/Dockerfile:FROM {nexus}/tango-java
        • tango-pogo/Dockerfile:FROM {nexus}/tango-java:latest
        • tango-rest/Dockerfile:FROM {nexus}/tango-dependencies:latest as buildenv
        • tango-rest/Dockerfile:FROM {nexus}/tango-java:latest
        • tango-vnc/Dockerfile:FROM {nexus}/tango-java:latest
      • tango-cpp/Dockerfile:FROM {nexus}/tango-dependencies:latest as buildenv
      • tango-cpp/Dockerfile:FROM debian:buster-slim
        • tango-archiver/Dockerfile:FROM {nexus}/tango-cpp:latest
        • tango-dsconfig/Dockerfile:FROM {nexus}/tango-cpp:latest
        • tango-libtango/Dockerfile:FROM {nexus}/tango-cpp
          • tango-admin/Dockerfile:FROM {nexus}/tango-libtango:latest
          • tango-test/Dockerfile:FROM {nexus}/tango-libtango:latest
          • tango-databaseds/Dockerfile:FROM {nexus}/tango-libtango:latest
        • ska-python-buildenv/Dockerfile:FROM {nexus}/tango-cpp:latest
          • ska-python-runtime/Dockerfile:FROM {nexus}/ska-python-buildenv:latest as buildenv
          • ska-python-runtime/Dockerfile:FROM {nexus}/tango-cpp:latest
            • tango-itango/Dockerfile:FROM {nexus}/ska-python-buildenv:latest as buildenv
            • tango-itango/Dockerfile:FROM {nexus}/ska-python-runtime:latest
            • tango-pytango/Dockerfile:FROM {nexus}/ska-python-buildenv:latest as buildenv
            • tango-pytango/Dockerfile:FROM {nexus}/ska-python-runtime:latest
            • tango-vscode/Dockerfile:FROM {nexus}/ska-python-buildenv:latest as buildenv
            • tango-vscode/Dockerfile:FROM {nexus}/ska-python-runtime:latest
        • tango-starter/Dockerfile:FROM {nexus}/tango-cpp:latest
  • mariadb
    • tango-db/Dockerfile:FROM mariadb:10
      • mariadb_hdbpp/Dockerfile:FROM {nexus}/tango-db:latest

HDB++ Archiver Implementation

Documentation Status

HDB++ is a archiver available in TANGO. It is configured to subscribe to TANGO attributes to be archived. MariaDB provides the archival database. HDB++ default schema is used to store the archived data. It consists of the following components: The Hdbpp-es is a event suscriber device which subscribes to the Tango attributes. Multiple instances of the event subscriber are deployed to support large number of attributes. These attributes could belong to multiple TANGO Devices. The Hdbpp-cm provides the configuration manager. It allows to configure the attributes to be archived and defines which Event Subscriber is responsible for a set of Tango attributes to be archived. Hdbpp-viewer shows the graphical representation of the archived data, which is stored in the data archive.

The archiever solution comprises of the following Docker images:

Tango-archiver(hdbpp-es and hdbpp-cm), Mariadb and Hdb++ viewer

Containers like mariadb, databaseds, hdbpp-es and hdbpp-cm should be up and running in order to work HDB++ archiver solution.

Important links

ska-docker's People

Contributors

ajoubertza avatar andrew-bolin avatar apurvapatkar avatar bartashevich avatar bravostuzero avatar digvijaymali92 avatar flyingfrog81 avatar gerhardlr avatar hfribeiro avatar jayantkumbhar avatar kalyanithigale avatar matteo04052017 avatar piersharding avatar shraddhabajare avatar

Stargazers

avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

ingvord reszelaz

ska-docker's Issues

cppTango 9.3.4rc5 docker image

Hi,
We would like to try if our Taurus CI spurious failures are due to some eventual Tango bugs which are already fixed in the latest 9.3.4.rc5. Currently we use tangocs/tango-cs:9.3.2-alpha.1-no-tango-test docker image. Do you have already prepared the 9.3.4.rc5 image? If not could you give us some starting hint on how to prepare it based on your work?
Many thanks in advance!

Cannot pull images (nexus.engageska-portugal.pt unavailable)

My CI scripts trying to pull the tango docker images from nexus.engageska-portugal.pt started to fail today.

When I try accessing https://nexus.engageska-portugal.pt/ on a browser, I get a "certificate expired error" (and indeed the certificate appears expired since 2020-06-09). But even if choose to ignore the alert, then I get:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /.

Reason: Error reading from remote server

PD: Sorry if this is not the appropriate place, but since I could not connect to https://nexus.engageska-portugal.pt/ , I could not find a better contact to report this

cannot start GUI apps inside vagrant VM

Hello,
I have created a vagrant VM using: https://github.com/ska-telescope/ansible-playbooks
I cannot start the GUI programs via the Makefile.
This appears to be caused by the XAUTHORITY_MOUNT variable.

If I change the line:
XAUTHORITY_MOUNT := /tmp/.X11-unix:/tmp/.X11-unix
to the one used in the 'Darwin' branch of the if statement:
XAUTHORITY_MOUNT := $(HOME)/.Xauthority:/hosthome/.Xauthority:ro

I am then able to use make start jive and so on in my VM.

Not sure if this would break other use cases?
(I can't get make start jive to work at all on my 'bare metal' PC, not sure of the problem there yet)

failure while building

Hi, I was trying to build the images locally and I got the following error (maybe an outdated repo gpg signature?):

ska-docker/docker(master)$ sudo make DOCKER_REGISTRY_USER=cpascual build
(...)
docker tag nexus.engageska-portugal.pt/cpascual/tango-dependencies:9.3.3 nexus.engageska-portugal.pt/cpascual/tango-dependencies:latest
do some stuff after the docker build
make[2]: Leaving directory '/home/cpascual/src/ska-docker/docker/tango/tango-dependencies'
make -C tango-db build
make[2]: Entering directory '/home/cpascual/src/ska-docker/docker/tango/tango-db'
do some stuff before the docker build
docker build  -t nexus.engageska-portugal.pt/cpascual/tango-db:10.4.10 . -f Dockerfile --build-arg DOCKER_REGISTRY_HOST=nexus.engageska-portugal.pt --build-arg DOCKER_REGISTRY_USER=cpascual
Sending build context to Docker daemon  14.85kB
Step 1/4 : FROM mariadb:10
 ---> bea3d0aeb501
Step 2/4 : ENV TANGO_DOWNLOAD_URL=https://nexus.engageska-portugal.pt/repository/raw/tango-cs/tango-9.3.3-rc2.tar.gz
 ---> Using cache
 ---> 86432dc2ab8a
Step 3/4 : RUN buildDeps='curl ca-certificates'     && DEBIAN_FRONTEND=noninteractive apt-get update     && apt-get -y install --no-install-recommends $buildDeps     && rm -rf /var/lib/apt/lists/*     && mkdir -p /usr/src/tango     && cd /usr/src/tango     && curl -fsSL "$TANGO_DOWNLOAD_URL" -o tango.tar.gz     && tar xf tango.tar.gz -C /usr/src/tango --strip-components=1     && mkdir -p dbinit/include     && cp cppserver/database/create_db.sql.in dbinit/create_db.sql     && cp cppserver/database/create_db_tables.sql.in dbinit/include/create_db_tables.sql     && cp cppserver/database/stored_proc.sql.in dbinit/include/stored_proc.sql     && sed -i "s|@TANGO_DB_NAME@|tango|g" dbinit/create_db.sql     && sed -i "s|@TANGO_DB_NAME@|tango|g" dbinit/include/create_db_tables.sql     && sed -i "s|@TANGO_DB_NAME@|tango|g" dbinit/include/stored_proc.sql     && sed -i "s|^source create_db_tables.sql$|source /docker-entrypoint-initdb.d/include/create_db_tables.sql|g" dbinit/create_db.sql     && sed -i "s|^source stored_proc.sql$|source /docker-entrypoint-initdb.d/include/stored_proc.sql|g" dbinit/create_db.sql     && sed -i "/CREATE DATABASE tango;/d" dbinit/create_db.sql     && cp -r dbinit/* /docker-entrypoint-initdb.d     && apt-get purge -y --auto-remove $buildDeps     && rm -r /usr/src/tango
 ---> Running in ae35e5ff0962
Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:5 http://ftp.osuosl.org/pub/mariadb/repo/10.4/ubuntu bionic InRelease [6265 B]
Err:1 http://archive.ubuntu.com/ubuntu bionic InRelease
  At least one invalid signature was encountered.
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
  At least one invalid signature was encountered.
Err:5 http://ftp.osuosl.org/pub/mariadb/repo/10.4/ubuntu bionic InRelease
  At least one invalid signature was encountered.
Err:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
  At least one invalid signature was encountered.
Err:2 http://security.ubuntu.com/ubuntu bionic-security InRelease
  At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://archive.ubuntu.com/ubuntu bionic InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu bionic InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu bionic-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu bionic-updates InRelease' is not signed.
W: GPG error: http://ftp.osuosl.org/pub/mariadb/repo/10.4/ubuntu bionic InRelease: At least one invalid signature was encountered.
E: The repository 'http://ftp.osuosl.org/pub/mariadb/repo/10.4/ubuntu bionic InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu bionic-backports InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu bionic-backports InRelease' is not signed.
W: GPG error: http://security.ubuntu.com/ubuntu bionic-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.ubuntu.com/ubuntu bionic-security InRelease' is not signed.

link to Important links got 404 error

Important links
https://gitlab.com/ska-telescope/ska-docker/tree/story_AT1-422/docker/tango/mariadb_hdbpp
....
and all other links at the bottom of this page failed.

I am interest to learn how to get "HDB++ Archiver Implementation" working.
Anybody have a guide for implement this HDB++ archiving on Docker?
Thanks,
Tac.

got this error when running "make start archiver"

make start archiver
photonlab@photonlab-FBG:~/ska-docker/docker-compose$ make start archiver
Network mode cannot be host for the archiver! It won't work unless you set the env var CI_JOB_ID=local
DISPLAY=localhost:10.0 XAUTHORITY=/hosthome/.Xauthority TANGO_HOST=photonlab-FBG:10000 NETWORK_MODE=host XAUTHORITY_MOUNT=/tmp/.X11-unix:/tmp/.X11-unix MYSQL_HOST=photonlab-FBG:3306 CONTAINER_NAME_PREFIX= COMPOSE_IGNORE_ORPHANS=true docker-compose -f tango.yml up -d
WARNING: Found orphan containers (tangotest) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
tangodb is up-to-date
databaseds is up-to-date
DISPLAY=localhost:10.0 XAUTHORITY=/hosthome/.Xauthority TANGO_HOST=photonlab-FBG:10000 NETWORK_MODE=host XAUTHORITY_MOUNT=/tmp/.X11-unix:/tmp/.X11-unix MYSQL_HOST=photonlab-FBG:3306 CONTAINER_NAME_PREFIX= COMPOSE_IGNORE_ORPHANS=true docker-compose -f jive.yml -f pogo.yml -f itango.yml -f astor.yml -f tango.yml -f logviewer.yml -f hdbpp_viewer.yml -f tangotest.yml -f archiver.yml -f rest.yml up --no-start
tangodb is up-to-date
tangotest is up-to-date
databaseds is up-to-date
DISPLAY=localhost:10.0 XAUTHORITY=/hosthome/.Xauthority TANGO_HOST=photonlab-FBG:10000 NETWORK_MODE=host XAUTHORITY_MOUNT=/tmp/.X11-unix:/tmp/.X11-unix MYSQL_HOST=photonlab-FBG:3306 CONTAINER_NAME_PREFIX= COMPOSE_IGNORE_ORPHANS=true docker-compose -f jive.yml -f pogo.yml -f itango.yml -f astor.yml -f tango.yml -f logviewer.yml -f hdbpp_viewer.yml -f tangotest.yml -f archiver.yml -f rest.yml start archiver
ERROR: No such service: archiver
Makefile:106: recipe for target 'start' failed
make: *** [start] Error 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.