Giter VIP home page Giter VIP logo

qshift-templates's People

Contributors

aureamunoz avatar cmoulliard avatar iocanel avatar

Stargazers

avatar

Watchers

avatar

Forkers

ch007m iocanel aureamunoz

qshift-templates's Issues

Enable to build a quarkus native image

Todo

Enable to build a quarkus native image
Changes:

  • Add a new parameter within the template: Native build: yes/no
  • Modify the maven package step
  • Add a new task to run native-image
  • Adapt the logic of the pipeline

Set a default value for the backstage owner

TODO

Set a default value for the backstage owner

        owner:
          title: Owner
          type: string
          ui:field: EntityPicker
          ui:options:
            catalogFilter:
              kind:
                - Group
                - User
          default: user:guest

User "system:serviceaccount:tqvarnst:pipeline" cannot get resource "virtualmachineinstances" in API group "kubevirt.io

TODO

Update the ClusterRoleBinding of the sa:pipeline to give it access to kubevirt.io API within the namespace of the application AND NOT as this is the case under qshift-test namespace.

Example

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubevirt.io:view
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubevirt.io:view
subjects:
- kind: ServiceAccount
  name: pipeline
  namespace: qshift-test

Add condition within the template to install postgresql helm chart and its argocd app

Todo

Add condition to install postgresql helm chart and its argocd app

Question:
What should be the parameters ?

  • DB_USERNAME=${DB_USERNAME:-healthy}
  • DB_PASSWORD=${DB_PASSWORD:-healthy}
  • DB_DATABASE=${DB_DATABASE:-fruits_database}
  • DB_NAMESPACE=$(DB_NAMESPACE:-)
    to set the helm values when we install the instance ?

Then helm chart could be installed as such

See: https://github.com/halkyonio/primaza-poc/blob/main/scripts/atomic-fruits.sh#L54-L59
helm install $DB_RELEASE_NAME bitnami/postgresql \
    --version $DB_VERSION \
    --set auth.username=$DB_USERNAME \
    --set auth.password=$DB_PASSWORD \
    --set auth.database=$DB_DATABASE \
    --create-namespace \
    -n ${DB_NAMESPACE}"

For the demo, we could define within the template the default values (using template parameters) that we pass to the argocd: create action.

It is also needed to update the application.properties to use the parameters OR that we pass instead ENV vars to the Quarkus App Deployment manifest instead.

WDYT ? @iocanel @aureamunoz

argocd action Error: Duplicate project detected. Cannot overwrite existing

Issue

When we replay the scaffolding using the template of the PR #5 then process fails

792024-02-09T11:37:40.000Z Finished step Generating GitOps Resources
802024-02-09T11:37:40.000Z Beginning step Publishing to Code Source Repository - Github
812024-02-09T11:37:41.000Z info: Init git repository {dir=/var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/f3344935-b1b0-4e02-8359-ad5c6dfeaf38}
822024-02-09T11:37:41.000Z info: Adding file {dir=/var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/f3344935-b1b0-4e02-8359-ad5c6dfeaf38,filepath=.}
832024-02-09T11:37:41.000Z info: Committing file to repo {dir=/var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/f3344935-b1b0-4e02-8359-ad5c6dfeaf38,message=initial commit}
842024-02-09T11:37:41.000Z info: Creating new remote {dir=/var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/f3344935-b1b0-4e02-8359-ad5c6dfeaf38,remote=origin,url=https://github.com/ch007m/my-quarkus-app.git}
852024-02-09T11:37:41.000Z info: Pushing directory to remote {dir=/var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/f3344935-b1b0-4e02-8359-ad5c6dfeaf38,remote=origin}
862024-02-09T11:37:42.000Z Finished step Publishing to Code Source Repository - Github
...
882024-02-09T11:29:00.000Z Error: Duplicate project detected. Cannot overwrite existing.89    at 
ArgoService.createArgoProject (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/@roadiehq/backstage-plugin-argo-cd-backend/src/service/argocd.service.ts:264:13)90    at 
process.processTicksAndRejections (node:internal/process/task_queues:95:5)91    at 
ArgoService.createArgoResources (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/@roadiehq/backstage-plugin-argo-cd-backend/src/service/argocd.service.ts:686:5)92    at 
Object.handler (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/@roadiehq/scaffolder-backend-argocd/src/actions/run/argocd.ts:59:7)93    at 
NunjucksWorkflowRunner.executeStep (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/@backstage/plugin-scaffolder-backend/src/scaffolder/tasks/NunjucksWorkflowRunner.ts:343:9)94    at 
NunjucksWorkflowRunner.execute (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/@backstage/plugin-scaffolder-backend/src/scaffolder/tasks/NunjucksWorkflowRunner.ts:431:9)95    at 
TaskWorker.runOneTask (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/@backstage/plugin-scaffolder-backend/src/scaffolder/tasks/TaskWorker.ts:145:26)96    at 
async run (/Users/cmoullia/code/openshift/fork-backstage-playground/node_modules/p-queue/dist/index.js:163:29)

Error occurs when the argocd plugin checks if the project already exists

https://github.com/RoadieHQ/roadie-backstage-plugins/blob/f96282c5430fa57f2a1953cf6cc7c66815cdf5ee/plugins/backend/backstage-plugin-argo-cd-backend/src/service/argocd.service.ts#L330-L361
  }: CreateArgoProjectProps): Promise<object> {
    const data = {
      project: this.buildArgoProjectPayload({
        projectName,
        namespace,
        sourceRepo,
        destinationServer,
      }),
    };

    const options: RequestInit = {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${argoToken}`,
      },
      body: JSON.stringify(data),
    };
    const resp = await fetch(`${baseUrl}/api/v1/projects`, options);
    const responseData = await resp.json();
    if (resp.status === 403) {
      throw new Error(responseData.message);
    } else if (resp.status === 404) {
      return resp.json();
    } else if (
      JSON.stringify(responseData).includes(
        'existing project spec is different',
      )
    ) {
      throw new Error('Duplicate project detected. Cannot overwrite existing.');
    }
    return responseData;
  }

Screenshot 2024-02-09 at 12 33 08

Duplicate code in quarkus app generates endpoint clashes

The following project https://github.com/ch007m/ch007m-quarkus-app generated via backstage Quarkus plugin contains duplicated entities and resources. It comes from the recently generator code added for hibernate-orm-rest-data extension in Quarkus.

Duplicated code can be found in

  • src/main/java/io/quarkus/MyEntity.java and src/main/java/io/quarkus/demo/MyEntity.java
  • src/main/java/io/quarkus/MyEntityResource.java and src/main/java/io/quarkus/demo/MyEntityResource.java

The following error is launched when the application is started with mvn quarkus:devcommand:

io.quarkus.demo.MyEntityResourceJaxRs_6379958a2e660cd15fde7de382353a8d80f383ae#delete consumes *, produces *
io.quarkus.MyEntityResourceJaxRs_95036a537c257516229b105d011e9f48bae2e7cc#delete consumes *, produces *
POST /my-entity is declared by :
io.quarkus.demo.MyEntityResourceJaxRs_6379958a2e660cd15fde7de382353a8d80f383ae#add consumes application/json, produces application/json
io.quarkus.MyEntityResourceJaxRs_95036a537c257516229b105d011e9f48bae2e7cc#add consumes application/json, produces application/json
GET /my-entity is declared by :
io.quarkus.demo.MyEntityResourceJaxRs_6379958a2e660cd15fde7de382353a8d80f383ae#list consumes *, produces application/json
io.quarkus.MyEntityResourceJaxRs_95036a537c257516229b105d011e9f48bae2e7cc#list consumes *, produces application/json
GET /my-entity is declared by :
io.quarkus.demo.MyEntityResourceJaxRs_6379958a2e660cd15fde7de382353a8d80f383ae#get consumes *, produces application/json
io.quarkus.MyEntityResourceJaxRs_95036a537c257516229b105d011e9f48bae2e7cc#get consumes *, produces application/json
PUT /my-entity is declared by :
io.quarkus.demo.MyEntityResourceJaxRs_6379958a2e660cd15fde7de382353a8d80f383ae#update consumes application/json, produces application/json
io.quarkus.MyEntityResourceJaxRs_95036a537c257516229b105d011e9f48bae2e7cc#update consumes application/json, produces application/json
GET /my-entity is declared by :
io.quarkus.demo.MyEntityResourceJaxRs_6379958a2e660cd15fde7de382353a8d80f383ae#count consumes *, produces application/json
io.quarkus.MyEntityResourceJaxRs_95036a537c257516229b105d011e9f48bae2e7cc#count consumes *, produces application/json

        at io.quarkus.resteasy.reactive.server.deployment.ResteasyReactiveProcessor.checkForDuplicateEndpoint(ResteasyReactiveProcessor.java:1379)
        at io.quarkus.resteasy.reactive.server.deployment.ResteasyReactiveProcessor.setupEndpoints(ResteasyReactiveProcessor.java:671)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)

cc @iocanel @cmoulliard

Merge the 2 pipelines into one

Todo

Merge the 2 pipelines into one as the step to build the image should only take place if the task: mvn test => accessing remote container succeeded.

Generated project dont include the proper application.properties & import.sql

Issue

When we generate a project using the option starter=true with code.quarkus.io, then the generated project don't include:

  • application.properties
{% if values.database == 'quarkus-jdbc-postgresql' %}
# Kubernetes Service name to access the DB should be as defined within the helm chart => postgresql:5432
# apiVersion: v1
# kind: Service
# metadata:
#  name: {{ include "postgresql.primary.fullname" . }}
#  see: https://github.com/bitnami/charts/blob/9ba6ba51cf7c094e457396c423f5843df46023a2/bitnami/postgresql/templates/primary/svc.yaml#L4
%prod.quarkus.datasource.jdbc.url=jdbc:postgresql://${{ values.database_name }}-postgresql:5432/${{ values.database_name }}
%prod.quarkus.datasource.username=${{ values.database_username }}
{% elif values.database == 'quarkus-jdbc-mysql' %}
%prod.quarkus.datasource.jdbc.url=jdbc:postgresql://${{ values.component_id }}-db:3306/${{values.component_id}}-db
%prod.quarkus.datasource.username=root
{% elif values.database == 'quarkus-jdbc-h2' %}
%prod.quarkus.datasource.db-kind=h2
%prod.quarkus.datasource.jdbc.url=jdbc:h2:mem:quarkus
{% endif %}

%prod.quarkus.hibernate-orm.sql-load-script=import.sql
%prod.quarkus.hibernate-orm.database.generation=drop-and-create
  • import.sql file matching the entity
-- This file allow to write SQL commands that will be emitted in test and dev.
insert into myentity (id, field) values(1, 'field-1');
insert into myentity (id, field) values(2, 'field-2');
insert into myentity (id, field) values(3, 'field-3');
alter sequence myentity_seq restart with 4;

Screenshot 2024-04-17 at 10 05 08

Query: https://code.quarkus.io/?e=rest&e=hibernate-orm-rest-data-panache&e=jdbc-postgresql&extension-search=postgres

PipelineRun name is not unique

Issue

The PipelineRun name is not unique when we populate the resource using the helm chart

// source: https://github.com/q-shift/qshift-templates/blob/main/qshift/templates/quarkus-application/manifests/helm/build-test-push/templates/03-pipelinerun-quarkus-build-test-push.yaml

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: quarkus-build-test-push
  namespace: {{ .Values.app.namespace }}
  labels:
    {{- include "backstage.labels" . | nindent 4 }}

By consequence, it is not possible within a namespace to create several quarkus applications

Suggestion: Add the link of the Openshift route exposing the quarkus application

Suggestion

Add the link/url of the OpenShift Route exposing the quarkus application within the Component

See doc: https://backstage.io/docs/features/software-catalog/descriptor-format/#links-optional

Example of links:

kind: Component
metadata:
  name: "my-quarkus-app"
  description: "A cool quarkus app"
  annotations:
    argocd/app-name: my-quarkus-app-dev
    backstage.io/kubernetes-id: my-quarkus-app
    backstage.io/kubernetes-namespace: my-quarkus-app-dev
    janus-idp.io/tekton: my-quarkus-app
    backstage.io/source-location: url:https://gitlab-gitlab.apps.cluster-8z65j.sandbox1430.opentlc.com/development/my-quarkus-app
    backstage.io/techdocs-ref: url:https://gitlab-gitlab.apps.cluster-8z65j.sandbox1430.opentlc.com/development/my-quarkus-app
    gitlab.com/project-slug: development/my-quarkus-app
    quay.io/repository-slug: "quayadmin/my-quarkus-app"
  tags:
    - java
    - quarkus
  links:
    - url: https://devspaces.apps.cluster-8z65j.sandbox1430.opentlc.com/#https://gitlab-gitlab.apps.cluster-8z65j.sandbox1430.opentlc.com/development/my-quarkus-app
      title: OpenShift Dev Spaces (VS Code)
      icon: web
    - url: https://devspaces.apps.cluster-8z65j.sandbox1430.opentlc.com/#https://gitlab-gitlab.apps.cluster-8z65j.sandbox1430.opentlc.com/development/my-quarkus-app?che-editor=che-incubator/che-idea/latest
      title: OpenShift Dev Spaces (JetBrains IntelliJ)
      icon: web

WDYT ? @iocanel @aureamunoz

template.yaml, YAMLParseError: Map keys must be unique

Issue

When the template is deployed on RHDH iusing helm chart version 1.1, we got this error

2024-03-26T15:14:37.960Z catalog warn YAML error at url:https://github.com/q-shift/qshift-templates/tree/main/qshift/templates/quarkus-application/template.yaml, YAMLParseError: Map keys must be unique at line 44, column 11:
- User
default: user:guest
^

as default: user:guest is defined twice

        owner:
          title: Owner
          type: string
          default: user:guest <== HERE
          ui:field: EntityPicker
          ui:options:
            catalogFilter:
              kind:
                - Group
                - User
          default: user:guest <== AND HERE

Name of the resources generated by the Helm chart is too long

TODO

The name of the resources: deployment, service, route, etc generated by the Helm chart is too long:

Example:
When we scaffold a project using as application name (= component_id): my-quarkus-app
then the deployment name becomes: my-quarkus-app-deploy-quarkus-deploy

Chart name is: quarkus-deploy
Chart Release name: my-quarkus-app-deploy

The following logic should be reviewed as currently it concatenates: <Chart Release name>-<Chart name> to produce a very long name OR we should use as chart name, the component_id

Remark: Apparently the argocd:create action create the chart release name using the Argocd App name which is in our case: my-quarkus-app-deploy or `<component_id>-deploy

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "quarkus-template.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

argocd + helm: error calling include: template: no template "quarkus-template.annotations"

Issue

Argocd fails to generate the Helm template due to

Failed to load target state: failed to generate manifest for source 1 of 1: r
pc error: code = Unknown desc = `helm template . --name-template my-quarkus-app-deploy --namespace openshift-gitops --kube-version 1.27 --values /tmp/9f46bb0b-0982-4bdb-8a46-52c507662f6e <api versions removed> --include-crds` 
failed exit status 1: Error: template: quarkus-deploy/templates/deployment.yaml:7:8: 
executing "quarkus-deploy/templates/deployment.yaml" at <include "quarkus-template.annotations" .>: 
error calling include: template: no template "quarkus-template.annotations" associated with template "gotpl" Use --debug flag to render out invalid YAML

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.