as a DevOps engineer I would like to have legion-airflow CI pipeline and relative Jenkins jobs so I can trigger CI testing of the component

Description
When the user clicks the button to delete a DAG, they are redirected to the 404 Not Found page.

Airflow plugin page: https://<-- cluster-url -->/airflow/admin/
Redirectes to: https://<-- cluster-url -->/admin/

Actual behavior
When the user clicks the button to delete a DAG, they are redirected to the 404 Not Found page and the DAG is removed.

Expected behavior
When the user clicks the button to delete a DAG, they stay on the same Airflow plugin page and the DAG is removed.

Attachment

ENV
ODAHU version: 1.6.0

Airflow should authenticate via OAuth to requests to API

Required due to critical fix blow:
apache/airflow#12341

Airflow Operators support templating

We should support this feature at least for important spec fields such as data input, image, etc.

So, we need:

1. Determine a set of spec fields for ModelTraining, ModelPackaging, ModelDeployment, BatchInference that should be templated
2. Then implement this templating by adding an appropriate feature to our plugin's operators

Use case:

Consider a DAG that runs every day and should train on some set of data that relate to the execution date's day

training_id, training = resource("""
kind: ModelTraining
id: airflow-wine-from-yamls
spec:
  model:
    name: wine
    version: 1.0
  toolchain: mlflow
  entrypoint: main
  workDir: mlflow/sklearn/wine
  hyperParameters:
    alpha: "1.0"
  data:
    - connName: wine
      localPath: mlflow/sklearn/wine/wine-quality-2020-10-10.csv
  resources:
    limits:
      cpu: 4
      memory: 4Gi
    requests:
      cpu: 2
      memory: 2Gi
  vcsName: odahu-flow-examples""")

with dag:
    train = TrainingOperator(
        task_id="training",
        api_connection_id=api_connection_id,
        training=training,
        default_args=default_args
    )

In this example, DAG trains using hardcoded mlflow/sklearn/wine/wine-quality-2020-10-10.csv file but it parameter should depend on DAG execution date such as mlflow/sklearn/wine/wine-quality-{{ execution_date }}.csv

As operations engineer I want to move odahuflow_api Airflow connection sensitive data to password field that will mask data in connection edit mode. This will require plugin changes.

Pass DagRun as PipelineRunID into ODAHU API
after Lineage service will be done

The new major Airflow version is released.
Need to migrate plugin to it.

Some users faced the version 1.10.12 issue described here: apache/airflow#12341.
We must update Airflow version to latest.

We should support the creation of the Legion resources from yaml files inside an Airflow and provide examples and guidelines.