Giter VIP home page Giter VIP logo

im-dashboard's Introduction

IM-Dashboard

Tests Codacy Badge Codacy Badge License Docs SQAaaS badge

Infrastructure Manager - Simple Graphical UI (based on INDIGO PaaS Orchestrator Dashboard)

Functionalities:

  • OIDC authentication
  • Display user's infrastructures
  • Display infrastructure details, template and log
  • Delete infrastructure
  • Create new infrastructure

The im-dashboard is a Python application built with the Flask microframework; Flask-Dance is used for Openid-Connect/OAuth2 integration.

The docker image uses Gunicorn as WSGI HTTP server to serve the Flask Application.

Achievements

SQAaaS badge

This software has received a gold badge according to the Software Quality Baseline criteria defined by the EOSC-Synergy project.

How to deploy the dashboard

Register a client in an OIDC server with the following properties:

  • redirect uri: https://<DASHBOARD_HOST>:<PORT>/login/oidc/authorized
  • scopes: 'openid', 'email', 'profile', 'offline_access' ('eduperson_entitlement' in EGI Check-In optional)
  • introspection endpoint enabled

Create the config.json file (see the example) setting the following variables:

Parameter name Description Mandatory (Y/N) Default Value
OIDC_CLIENT_ID OIDC client ID Y N/A
OIDC_CLIENT_SECRET OIDC client Secret Y N/A
OIDC_BASE_URL OIDC service URL Y N/A
OIDC_GROUP_MEMBERSHIP List of OIDC groups to be checked for allowing access N []
OIDC_SCOPES OIDC scopes Y N/A
TOSCA_TEMPLATES_DIR Absolute path where the TOSCA templates are stored Y N/A
TOSCA_PARAMETERS_DIR Absolute path where the TOSCA parameters are stored Y N/A
IM_URL Infrastructure Manager service URL Y N/A
IM_TIMEOUT Infrastructure Manager service calls timeout N 60
SUPPORT_EMAIL Email address that will be shown in case of errors N ""
EXTERNAL_LINKS List of dictionaries ({ "url": "example.com" , "menu_item_name": "Example link"}) specifying links that will be shown under the "External Links" menu N []
LOG_LEVEL Set Logging level N info
DB_URL URL to the DB to store dashboard data N sqlite:///creds.db
ANALYTICS_TAG Google Analytic Tag N ""
STATIC_SITES List of static sites added to the AppDB ones ([{"name": "static_site_name", "url": "static_site_url", "id": "static_id", "vos": {"vo": "stprojectid"}}]) N []
STATIC_SITES_URL URL of a JSON file with the list of static sites added to the AppDB ones N ""
APPDB_CACHE_TIMEOUT AppDB cache TTL N 3600
CHECK_TOSCA_CHANGES_TIME Interval to look for changes in TOSCA templates N 120
VAULT_URL Vault service URL to store Cloud credentials N None

You need to run the IM dashboard on HTTPS (otherwise you will get an error); you can choose between

  • enabling the HTTPS support
  • using an HTTPS proxy

Details are provided in the next paragraphs.

Enabling Credentials encryption

To enable the encryption of the Cloud providers credentials (sensitive data), you have to set the CREDS_KEY environment varible with a valid key used to encrypt/decrypt de data. To get a valid one you can use this python code (you will nedd Cryptography library):

from cryptography.fernet import Fernet
key = Fernet.generate_key()
print(key.decode())

TOSCA Template Metadata

The IM dashboard can exploit some optional information provided in the TOSCA templates for rendering the cards describing the type of applications/services or virtual infrastructure that a user can deploy.

In particular, the following tags are supported:

Tag name Description
description Used for showing the card description
metadata.display_name Used for the card title. If not pro
metadata.icon . Used for showing the card image. If no image URL is provided, the dashboard will load this icon.
metadata.display_name Used for the card title. If not provided, the template name will be used
metadata.tag Used for the card ribbon (displayed on the right bottom corner)

Example of template metadata:

tosca_definitions_version: tosca_simple_yaml_1_0

imports:
  - indigo_custom_types: https://raw.githubusercontent.com/indigo-dc/tosca-types/v4.0.0/custom_types.yaml

description: Deploy a Mesos Cluster (with Marathon and Chronos frameworks) on top of Virtual machines

metadata:
  display_name: Deploy a Mesos cluster
  icon: https://indigo-paas.cloud.ba.infn.it/public/images/apache-mesos-icon.png

topology_template:

....

You can find the set of available TOSCA templates in the following repo.

Enabling HTTPS

You would need to provide

  • a pair certificate/key that the container will read from the container paths /certs/cert.pem and /certs/key.pem;
  • the environment variable ENABLE_HTTPS set to True

Run the docker container:

docker run -d -p 443:5001 --name='im-dashboard' \
           -e ENABLE_HTTPS=True \
           -v $PWD/cert.pem:/certs/cert.pem \
           -v $PWD/key.pem:/certs/key.pem \
           -v $PWD/config.json:/app/app/config.json \
           -v $PWD/tosca-templates:/opt/tosca-templates \
           grycap/im-dashboard:latest

Access the dashboard at https://<DASHBOARD_HOST>/

Using an HTTPS Proxy

Example of configuration for nginx:

server {
      listen         80;
      server_name    YOUR_SERVER_NAME;
      return         301 https://$server_name$request_uri;
}

server {
  listen        443 ssl;
  server_name   YOUR_SERVER_NAME;
  access_log    /var/log/nginx/proxy-paas.access.log  combined;

  ssl on;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_certificate           /etc/nginx/cert.pem;
  ssl_certificate_key       /etc/nginx/key.pem;
  ssl_trusted_certificate   /etc/nginx/trusted_ca_cert.pem;

  location / {
                # Pass the request to Gunicorn
                proxy_pass http://127.0.0.1:5001/;

                proxy_set_header        X-Real-IP $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header        X-Forwarded-Proto https;
                proxy_set_header        Host $http_host;
                proxy_redirect          http:// https://;
                proxy_buffering         off;
  }

}

Run the docker container:

docker run -d -p 5001:5001 --name='im-dashboard' \
           -v $PWD/config.json:/app/app/config.json \
           -v $PWD/tosca-templates:/opt/tosca-templates \
           grycap/im-dashboard:latest

⚠️ Remember to update the redirect uri in the OIDC client to https://<PROXY_HOST>/login/oidc/authorized

Access the dashboard at https://<PROXY_HOST>/

Performance tuning

You can change the number of gunicorn worker processes using the environment variable WORKERS. E.g. if you want to use 2 workers, launch the container with the option -e WORKERS=2 Check the documentation for ideas on tuning this parameter.

im-dashboard's People

Contributors

micafer avatar maricaantonacci avatar mtangaro avatar sebastian-luna-valero avatar gmolto avatar alfonpd avatar mariojmdavid avatar miklosban avatar antoniosanch3z avatar aidaph avatar josemifv avatar danielmartinez avatar marcvs avatar vykozlov avatar

Stargazers

 avatar Sebastián Risco avatar Baptiste Grenier avatar  avatar  avatar  avatar

Watchers

James Cloos avatar  avatar  avatar  avatar

im-dashboard's Issues

Some infrastructures stay listed after deleting

Hi,

After trying to deploy some infrastructures, (with failure due to site issues), I cannot delete the infrastructures from "My Infrastructures" list.

Screenshot from 2020-06-02 10-01-24

After presing "Delete" button, the "infrastructure uuid" stills there.
BR, Borja

Get SHA256 as output parameter from TOSCA

Improvement idea:

Issue:

When connecting with ssh, normally the first time the authenticity cannot be established, for example:

ssh -i key.pem [email protected]
The authenticity of host '193.144.46.239 (193.144.46.239)' can't be established.
ECDSA key fingerprint is SHA256:XoSh3FqD8E+oB1VVP/i8U8oiSzHrDlDWB0w/53aZe58.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '193.144.46.239' (ECDSA) to the list of known hosts.

Proposal:

Have an output normally which indicates (just as the Ip for example) the SHA256 of the key, in the example above:

State: | running
-- | --
OpenStack: fedcloud-osservices.egi.cesga.es:5000
0: 192.168.9.6 1: 193.144.46.239
1 CPUs, 2 GB of RAM , 10 GB of HD
Username: cloudadm
Download
SHA256:XoSh3FqD8E+oB1VVP/i8U8oiSzHrDlDWB0w/53aZe58

Wrong VO-Project target when creating infrastructure

When deploying a K8 cluster, I selected VO: "o3as"

However the master was created in the project scope "eosc-synergy" wich I noticed it is the one configured at my service credentials.

The application should not deploy the infrastructure on a different VO project as it would be difficult to find out where it was created (when you belong to several VO).

Also, when I try to edit my project at service credentials, it does not allow me to select my other VO.
image

Extra info

I belong to:
image

Related #33

Block deployment if "Site Resource Usage" is not enoguh

I can see that the IM Dashboard displays not the "Site Resource Usage"! That is really nice!

I think it would be added value if the application would rise you a warning when trying to deploy something where resources are not enough. For example, when trying to deploy a k8 cluster where all the Floating IP are occupied, rise something like:
Note this site has no available Floating IPs for your deployment. Would you like to continue? YES/NO

Error getting VO sites

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/app/app/__init__.py", line 432, in getsites
    appdb_sites = appdb.get_sites(vo)
  File "/app/app/appdb.py", line 108, in get_sites
    if check_supported_VOs(site, vo):
  File "/app/app/appdb.py", line 68, in check_supported_VOs
    if '@voname' in os_tpl and vo in os_tpl['@voname']:
TypeError: string indices must be integers

Infrastructure state Loading in mobile vertical format

Using the IM Dashboard from the mobile (iOS - Google Chrome), when placing the mobile in landscape format the status of the infrastructure is displayed correctly. Being vertical, it always stays in the "Loading" state.

Error loading images: Project __ not found

Hi,

I get the following error on the field "Select Site image: " when deploying a VM:

Error loading site images: Project eosc-synergy.eu not found!

Screenshot from 2020-06-02 10-12-51

After checking with @mariojmdavid, it seems the name configured for the project was "eosc-synergy" not "eosc-synergy.eu".

I supose a way to solve it would be to change the project name on OpenStack, however it looks this is not always possible.

Would it be an alternative to fix this issue?

Discard providers that would not work with your selected VO

According to #33, when you select a VO and a Provider to deploy a TOSCA, it might happen that you get the error "Error loading images: Project __ not found".

This is quite confusing for users, normally they do not know that they HAVE to configure/change the project-site at the "Service Credentials" page.

Would it be possible to?

  1. Configure the field "Select provider" to display ONLY the sites where you configured the selected VO.

  2. Create a note+link bellow "Select provider" to open a new tab on "Service Credentials", something similat to:
    If your provider is not listed, please configure your "Service Credentials" for the site.

  3. If a wrong VO was selected, rise an error! A log only saying "Error image not found" is not sufficient for the user.

I think that would simplify the user experience (at list for those with multiple VO).

How to reproduce

  1. You need to belong to at least 2 VO (let's call VO1 & VO2)
  2. You need to have at least 1 site supporting those 2 VO (let's call it YOUR-SITE)
  3. Go to Service Credentials and EDIT "Manage your credential" at the YOUR-SITE to work with VO1.
    4 Select any TOSCA templete to configure a deployment (Any)
  4. Do a normal configuration that would normally work.
  5. In the tab "Scheduling" select the VO2 and the provider "YOUR-SITE".
  6. Submit.

Deployment log example:

2020-11-12 09:06:38.223942: All VMs failed. No contextualize.
2020-11-12 09:06:44.295493: Error launching the VMs of type front to cloud ID ost3 of type OpenStack. Attempt 1: Error: 404 Not Found {'message': 'No image found with ID 8a32deca-1e5c-4d46-919d-b01b1f52e9b9

\n\n\n', 'code': '404 Not Found', 'title': 'Not Found'}
Attempt 2: Error: 404 Not Found {'message': 'No image found with ID 8a32deca-1e5c-4d46-919d-b01b1f52e9b9

\n\n\n', 'code': '404 Not Found', 'title': 'Not Found'}
Attempt 3: Error: 404 Not Found {'message': 'No image found with ID 8a32deca-1e5c-4d46-919d-b01b1f52e9b9

\n\n\n', 'code': '404 Not Found', 'title': 'Not Found'}

Error launching the VMs of type wn to cloud ID ost3 of type OpenStack. Attempt 1: Error: 404 Not Found {'message': 'No image found with ID 8a32deca-1e5c-4d46-919d-b01b1f52e9b9

\n\n\n', 'code': '404 Not Found', 'title': 'Not Found'}
Attempt 2: Error: 404 Not Found {'message': 'No image found with ID 8a32deca-1e5c-4d46-919d-b01b1f52e9b9

\n\n\n', 'code': '404 Not Found', 'title': 'Not Found'}
Attempt 3: Error: 404 Not Found {'message': 'No image found with ID 8a32deca-1e5c-4d46-919d-b01b1f52e9b9

\n\n\n', 'code': '404 Not Found', 'title': 'Not Found'}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.