Scaffold the wego controller

https://kubebuilder.io/

Also Rename Team Workspaces

Question
We will have the MCCP UI, a team workspaces UI and the flux UI. How should we integrate these UIs so they appear seamless to the user?

Timebox
1 week

Outcome
Writeup your findings here. Please include the following (if any have been created) :

• Screenshots
• Design/architecture diagrams
• WEP
• PR/FAQ

Provide the ability to create an application via calls to the Wego Api Server. Eventually, both the CLI and UI will interact with the system via the Api Server.

Short description
As a App developer I want to use wego with a repo created by my corp IT so that wego doesn't require permissions to create a repo

Acceptance criteria

• blank repo before running wego
• post running wego, my repo has the proper directory layout
• add an app to the repo, it gets deployed into my cluster.

Work breakdown
(optional)Task breakdown of the work

• create X
• design X
• ...

As a platform operator I want to be able to use a helm chart to install services into my cluster for use by all the application teams.

Example:
https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack

Acceptance Criteria:

• Validate PR (that it exists and contains the helm chart/release)
• Library support for PR approval, merge, etc.
• Chart/Release shows up in cluster after commit and is healthy

In order to decide what actions to take when deploying to a cluster, we need to know if the cluster is:

• unmodified
• has flux2 installed
• has wego installed

We need a library call that will return one of: "unmodified", "flux2", "wego" (or similar choices)

Short description
As an Application Developer, I want to run my application in multiple k8s clusters so that I can ensure the same repository can be used across target clusters and that WeGO works correctly in minikube and k3s.

Given a kind cluster with WeGO installed
And a WeGO app added to the cluster
And a k3s cluster
And a minikube cluster
When I install the WeGO GORT into the k3s cluster
And I install the GORT into the minikube cluster
Then I can add the same app to the new clusters from the same Git repository
And it will function normally in all three clusters

In order to pull and install the correct dependencies, we need a description of all dependency locations and versions. We can use the dependencies.toml model from wkp but it would likely be worthwhile to see if the state of the art has moved on.

Short description
As a SRE I want to see what configuration wego would apply to the cluster for a particular commit -- without it actually performing any actions so that I can verify what manifests, repositories, secrets, etc will be created

Acceptance criteria

• issuing --dry-run doesn't modify my cluster
• issuing --dry-run doesn't create or modify any repositories
• displays manifests that will be added to my cluster and repo
• prints the steps that would be performed without the --dry-run flag

Design materials

• Screenshots
• Design diagrams
• WEP
• PR/FAQ

Work breakdown
(optional)Task breakdown of the work

• Separate out generation of manifests so that they can be displayed rather than applied
• Display output

Short description
As a SRE I want to deploy MCCP into dev using sqlite and into UAT with postgress so that I can use gitops across environments

Acceptance criteria

• Deploy MCCP v1 in dev with slqlite and using a shared config repo deploy in staging with external DB
• single configuration repo with both environments
• all manifests in git

Design materials

• Screenshots
• Design diagrams
• WEP
• PR/FAQ

Work breakdown
(optional)Task breakdown of the work

• create X
• design X
• ...

Setup Github workflows to install and create a kind cluster.

The initial test can be a simple kubectl get pods -A

Define, manage, and update weave gitops platform definitions in git repository:

start by extending a weve gitops app repository, by adding a new /platform directory

Connect a new or existing repostory to a namespace

If there is a platform repo, this should be done by committing config to that repo.

Short description
As an SRE I want to query metrics from gitops so that I can use it as an aid in troubleshooting my wego environment

Acceptance criteria

• prometheus is installed
• user can query metrics from SC, HC, NC, KC
• manifest(s) for the installed prometheus are in the platform repo (or folder in the monorepo case) ("platform apps"?)

Design materials

• WEP See TBD

Work breakdown
(optional)Task breakdown of the work

• create X
• design X
• ...

Short description
As an application operator, I would like to go the core UI and see the status of all my applications -- whether they are in sync, healthy, or there are errors.
After I've deployed something, I would like to see the status of the deployment.
(See weave cloud deploy page for related example)

Acceptance criteria

• ?
• ?

Work breakdown
(optional)Task breakdown of the work

• create X
• design X
• ...

Question
What are 2 or 3 ways WeGO can have a positive impact on users Deployment Frequency, Lead Time For changes, MTTR, Change Failure Rate.

Short description
DORA identified the following metrics as key attributes for measuring a team/product health:

• Deployment Frequency
• Lead Time For changes
• MTTR
• Change Failure Rate.

Timebox
3 days

Questions

• Most of the benefit is provided by GitOps in general. What, if anything, does wego add above just GitOps.
• Is this already covered by existing white papers and work Cornelia is already doing?

Outcome
Writeup your findings here. Please include the following (if any have been created) :

• Screenshots
• Design/architecture diagrams
• WEP
• PR/FAQ

Connect a repository to an "empty" cluster. Via a running weave gitops runtime.

Prerequisites:

"empty" cluster: Not configured with gitops runtime

Architecture Design:

Acceptance Criteria:

GitOps runtime is installed into a cluster
Connected to a repo wego creates in github

Future:

• need to be able to attach to existing repo
• need to be able to recreate this install from config stored in git (reconstitute the cluster)

Based on a declarative configuration

• abstraction layering is an issue
• generate an "weave gitops install spec" ?
• can be used to reconstitute the system

Define and manage Applications in the apps directory of a git repo, deployed into a k8s cluster

Currently in the ci we are using our keys for dockerhub, aws and a github token for changelog. It would be nice to have keys from accounts that wont disappear when an employee leaves.

The wego add command should add a new workload repo into the cluster.

1. call flux twice as following:

flux create source git podinfo \
  --url=https://github.com/stefanprodan/podinfo \
  --branch=master \
  --interval=30s \
  --export > ./clusters/my-cluster/podinfo-source.yaml

flux create kustomization podinfo \
  --source=podinfo \
  --path="./kustomize" \
  --prune=true \
  --validation=client \
  --interval=5m \
  --export > ./clusters/my-cluster/podinfo-kustomization.yaml

2. commit those files and create a pull request

helm option will be added in another task

TBD

• will we require the user to be in the cluster repo? if so how can we handle the wego add . suggestion mark made ?

Configuring an application in wego can require creating a repository (if an existing repository is not specified). The new repository should have the structure currently being defined for wego applications (TBD).

We should be able to create repositories via clone or fork.

Short description
As a SRE I want to see what wego will do without it actually performing any actions so that I can verify what manifests, repositories, secrets, etc will be created

Acceptance criteria

• using dry run mode doesn't modify my cluster
• using dry run mode doesn't create or modify any repositories
• displays manifests that will be added to my cluster and repo
• prints the steps that would be performed without using dry run mode

Design materials

• Screenshots
• Design diagrams
• WEP
• PR/FAQ

Work breakdown
(optional)Task breakdown of the work

• create X
• design X
• ...

Short description

As a SRE I want to update wego/flux k8s cluster so that I'm running the latest code.

Flux does this by downloading new version and reinstalling...

wego upgrade

For now we will only support one previous version when upgrading. A PR will be created in the wego platform repository (assuming only one repository needs to be updated). The cluster/target will be upgraded when the PR is merged.

Discussion

Upgrading a Weave GitOps installation

The process of upgrading Weave GitOps for a single cluster consists of two pieces:

• Upgrading flux controller resources
• Upgrading resources generated by Weave GitOps

Flux Controller Resources

Flux controller resources should not constitute a significant problem since flux supports upgrading them by simply downloading a new flux version and re-installing flux. As we don't expect users to modify the core flux manifests, the only reason we store them at all is so that upgrading can be done via pull request. We'll simply need to run flux install --export and overwrite the existing flux manifests in the platform repository with the new ones generated by the install.

Resources Generated by Weave GitOps

Resources generated by wego app add are potentially a bit more complicated in that we expect users may modify them (to change the sync interval, for example). It would be a much better user experience if we were to keep any such changes during the upgrade. According to the flux migration timetable, any breaking changes to the CRDs will come with associated conversion mechanisms. So, we should be able to run/apply such mechanisms during an upgrade when such a change occurs. As of yet, it isn't clear what such mechanisms will look like so we will have to plan to be flexible if/when breaking changes occur. In the absence of such breaking changes, we should be able to leave flux-generated resources alone. The only resource we may have to update is our own app resource. That should be a relatively straightforward change, though, since the app resource has no user-serviceable parts.

Pull Requests

An unfortunate side effect of our choice to allow configuration to be stored in multiple places is that we can't always upgrade everything with a single pull request. For flux-only upgrades without breaking changes, we will only need to create a PR for the platform repository. However, if we change the app resource or a breaking change occurs in the flux CRDs, we'll need to extract the config URL from each app managed by Weave GitOps and create a PR in each referenced repository with the necessary changes. The UI or CLI will have to inform the user of all the repositories that will need PRs reviewed and merged. (It's possible that this issue will be go away if the directory structure changes play out as currently discussed)

Multi-cluster

In a multi-cluster environment there will be applications that are deployed in multiple clusters. The GitOps management resources for the applications (other than the app resource), however, will be segregated by cluster. So, wego upgrade should be able to work just fine on an individual cluster. The only place in which that might be problematic is the app resource, which is shared. This suggests that we'll need to ensure all app changes are backward-compatible (for some number of releases) so that we can upgrade the app resources during an upgrade for a single cluster.

Short description
As a developer, I want to push a tag and a draft release gets created into GitHub.

Acceptance criteria

• there is a GitHub action that runs the tests, builds the artifacts, collects the changelogs, and creates a release with it.

Short description
As a platform operator, I want to install wego into a cluster

Acceptance criteria

• user is able to specify the branch, the owner, the repo name and the provider flux will use to bootstrap.

Show the status of all the objects in the application from the CLI.

wego status application <name> or something like that should show the current deployment status of each of the gitops goals that are a part of that application.

Acceptance criteria

• wego CLI status call shows if my app has been sync'd
• wego CLI status call shows if my app has been processed through the kustomize controller (if my app is kustomize based)
• wego CLI status call shows if my app has been processed through the helm controller (if my app is helm based)
• wego CLI displays any errors my app received in the GitOps runtime
• wego CLI displays last time my app was successfully deployed

Design materials

• UX

Work breakdown
(optional)Task breakdown of the work

• Check status of all flux components driving application in cluster (source, helm, kustomize controllers)
• Display results to user via CLI

Posslbily Later:
Task: Determine which resources in the system were installed by WeGO
Task: Extract current status from each resource - deployment successful, health checks successful
Task: Display results to user

I believe hooks for monitoring the gitops runtime via Prometheus/Grafana already exist.

I expect we therefore need to provide a base platform stack that includes prometheus, graphana, and alert manager.

As soon as there is a testable/usable version of the flux UI, I expect we will want to install that as well.

Spike goal is to determine what components need to be added, and create initial stories for weave gitops cluster observability.

Short description
As a SRE I need to deploy the existing MCCP cluster observability tools with sqlite in dev and postgres in UAT so that the UAT environment data lives longer

Given two kind clusters (dev and UAT) with Weave GitOps installed
And the Helm Chart for the MCCP cluster observability tools is in a git repository
And there are two Values.yaml files in distinct directories within the git repository
And the values file for dev defines the db as sqlite
And the values file for UAT defines the db as postgreSQL
When I wego app add the Helm Chart to each cluster
And the paths for the apps point to the distinct directories
Then the MCCP tools are deployed to the dev cluster using sqlite
And the MCCP tools are deployed to the UAT cluster using postgreSQL

To make it easier to contribute to weave-gitops, we need a make target that will install necessary dependencies.

Short description
As a cluster operator I want to install weave gitops into my kubernetes cluster so that application teams can have a gitops repository to manage deployments of that software.

Acceptance criteria

• user is able to specify the branch, the owner, the repo name and the provider flux will use to bootstrap.

Setup the test automation framework to run with CI

CLI commands: App list and app status create equivalent pages for UI

View status of (and errors encountered) in the gitops delivery pipeline.

https://miro.com/app/board/o9J_lVcp-Zc=/?moveToWidget=3074457354294370398&cot=14

Setup the initial automation testing framework for WEGO

Question
We will have the MCCP UI, a team workspaces UI and the flux UI. How should we integrate these UIs so they appear seamless to the user?

Timebox
1 week

Outcome
Writeup your findings here. Please include the following (if any have been created) :

• Screenshots
• Design/architecture diagrams
• WEP
• PR/FAQ

Currently when the publish-s3-artifact job that runs on master merge is broken and can't push the binary to s3

Flux can be installed into an environment using terraform - see https://github.com/fluxcd/terraform-provider-flux

Should Weave GitOps also support this type of install?

As an application operator I would like to be able to put a helm chart in the repository and have the software properly installed in my namespace in the cluster.

Example: Postgres

Acceptance Criteria:

• Validate PR (that it exists and contains the helm chart/release)
• Library support for PR approval, merge, etc.
• Chart/Release shows up in cluster after commit and is healthy

• Mental model where workloads are in their own folder, and pipelines are kept separately.
• Flux follows "minimalAPI" we should provide a more full featured api.
• Application primitive

Short description
As a application dev I want to use GitOps to deploy the sock shop demo so that I can learn about k8s and gitops.

Sockshop repo: https://github.com/microservices-demo/microservices-demo
Version: 0.0.12
Path: deploy/kubernetes/complete-demo.yaml

Acceptance criteria

• k8s running sock shop
• There is a doc page showing this as an example helm chart based deployment
• cluster repo has a source controller pointing at the repo ref
• make sure it works...

We are going to be using the flux2 binary from within the wego binary. We need to embed the binary and add a wego flux command that allows invoking arbitrary flux commands that won't be exposed through our opinionated interface.

Setup test cases for WEGO user-facing scenarios.