The purpose of the quality dashboard is to collect the status of AppStudio services:
- Github repositories data and action status
- Code coverage from codecov
- Build status and test types
- Openshift CI jobs and their statistics
- Jira issues impacting RHTAP (in progress, blockers)
Quality Dashboard implements a Golang-based backed and stores data in a PostgreSQL database, modelled and queried with entgo framework.
Different specific connectors are developed to pull data from different sources:
- Github connector: to pull data from github, such as repositories information and actions status
- Codecov connector: to pull code coverage data from Codecov
- ProwJobs connector: to pull automatically data about prow jobs executions impacting the repositories
- Jira connector: to pull issues from Jira
The database will retain last 10 days of CI job executions.
Ent is an Object Relational Mapping (ORM) framework for modeling any database schema as Go objects. The only thing you need to do is to define a schema and Ent will handle the rest. Your schema will be validated first, then Ent will generate a well-typed and idiomatic API. The generated API is used to manage the data and will contain:
- Client objects used to interact with the database
- CRUD builders for each schema type
- Entity object (Go struct) for each the schema type
You can use such generated code to build your endpoints and manipulate the database in an easy and programmatic way.
The schema for Quality Dashboard data types is located here. You can refer to entgo documentation for syntax details.
After adding new data types to the schema (or editing the existing ones), you have to execute the following command in backend/pkg/storage/ent
to re-build the model:
go run -mod=mod entgo.io/ent/cmd/ent generate ./schema --target ./db --feature sql/upsert
The generated code will be saved into the backend/pkg/storage/ent/db
folder.
The backend/pkg/storage/ent/client
package implements the database client used to interact with the database.
In turn, the database client package implements the storage interface used by the server.
The backend server exposes a set of APIs to interact with data. The implementation of the API server is located at backend/api
and uses a basic HTTP router configuration.
The frontend component is a React web application that uses patternfly project to build the UI. It interacts with the backend via HTTP api endpoints.
You need to be logged in to an OpenShift cluster first.
Example (oc command): oc login -u <user> -p <password> <oc_api_url>.
The install script will deploy quality dashboard and all resources to your OpenShift cluster.
The script will create a namespace appstudio-qe
and deploy backend and frontend using kustomize
To run the script:
# Run install.sh from hack folder to deploy the dashboard
/bin/bash hack/install.sh --storage-user <username> --storage-password <password> --github-token <token> --jira-token <token>
When running the install script, you need to specify these parameters:
Parameter Name | Description | Required | Example |
---|---|---|---|
github-token |
Github token to read repositories | yes | --github-token ghp_xxxxx |
jira-token |
Jira token to read jira issues | yes | --jira-token xxxxx |
storage-user |
Database user name | yes | --storage-user admin |
storage-password |
Database user password | yes | --storage-password adminPassword |
To install quality dashboard locally (for development purposes) you will need to run both backend and frontend by your own.
First, you need to have a PostgreSQL instance running to host local data. You can start one with your favourite container engine (docker or podman)
podman run -p 5432:5432 --name some-postgres -e POSTGRES_PASSWORD=postgres -d postgres
After that, you need to build the backend binaries. To do that you can follow the backend instructions.
Once built, run the backend server in a terminal:
# from the backend folder
./bin/server-runtime
If you specified some different values for the database container, you can override the default values by exporting the following environment variables:
POSTGRES_ENT_HOST
POSTGRES_ENT_PORT
POSTGRES_ENT_DATABASE
POSTGRES_ENT_USER
POSTGRES_ENT_PASSWORD
GITHUB_TOKEN
The server runtime will take care of initializing the database structure and pull the data.
Open a new terminal, navigate to the frontend folder, install dependencies and run:
cd frontend
yarn
yarn start:dev
or with npm:
cd frontend
npm install
npm run start:dev
All data is organized by Teams: a team groups a set of repositories to show data in a more concise manner and acts as a global filter. All the teams that have been created will be listed in a table in the Teams page, where they can also be managed. Switching a team from the main toolbar, will update the context for the whole view in the dashboard.
The Config page provides a quick way of adding multiple teams and repositories in the DB. It presents a code editor where you can set teams and its corresponding JIRA projects and repositories, by downloading an existing config or starting from scratch.
Please, note that:
- different teams can not have the same repository
- different teams can not have the same description
The config should conform to the following template:
teams:
- name: team-example
description: description-example
jira_projects:
- STONE
repositories:
- name: e2e-tests
organization: redhat-appstudio
- name: quality-dashboard
organization: redhat-appstudio
The Openshift CI connector will collect and show an overview of the last 10 days of jobs execution, by repository and job type. Current job types are: presubmit, periodic and postsubmit. If more than one job per repository and job type is there, the connector will collect all of them. The dashboard will present the last 10 days of data in a chart, for day to day inspection, and the averages of the whole period of time. Also, just for periodic jobs, we show the test suites output of the last executed job.
The Github connector will pull data from Github, such has repositories info and action status.
The codecov connector will pull code coverage data from Codecov.
The Jira connector will pull data from Jira. We gather Jira issues that are impacting RHTAP (such as blockers, in progress, etc.) and present them in the dashboard for quick reference.