Task to set up basic releasing workflow, should include pushing docker image.

When someone tries to create a profile thats invalid it should not get added to the catalog

Example:

The below catalog entry should get rejected because it does not have a valid version

apiVersion: weave.works/v1alpha1
kind: ProfileCatalogSource
metadata:
  name: nginx-catalog
spec:
  profiles:
    - name: weaveworks-nginx
      description: This installs nginx.
      version: totally!not?semver
      url: https://github.com/weaveworks/profiles-examples
      maintainer: weaveworks (https://github.com/weaveworks/profiles)
      prerequisites:
      - Kubernetes 1.18+

Not sure what fields outside of version we want to check 🤷

Implementation thoughts

Two reasonable approaches:

1. Validating webhook, checks each entry and rejects any that are malformed. This lets the user get immeditate feedback about the failures
2. The controller simply ignores any malformed profiles
3. Regex matching as part of the CRD https://book.kubebuilder.io/reference/markers/crd-validation.html

What feature/behavior/change do you want?

$ pctl search nginx
weaveworks/nginx-ingress - NGINX Ingress Controller
thirdparty/nginx - NGINX server for static content

Why do you want this feature?

I want to be able to find profiles for installation.

This is simply about querying a cached index of known profiles, indexing these profiles from a simple, static catalog source.

As part of #108 we agreed on an implementation for multi-profile repos and how to tag them. We need to document how users can setup a profile repository and can setup an example repo that contains multiple profiles & tags.

Outcomes:

• I can read docs on how to setup a repo containing multi profiles and tags
• I have a repo I can go to to see an example

... and is that a problem?

could i end up with 2 x cat1/prof1 in my catalog?

What feature/behavior/change do you want?

Access to a private repository to access a profile.

Why do you want this feature?
Some profiles will not be public, and we want access to access them.

As a user, I want to be able to see all my subscriptions, and confirm that they have been deployed.

Run kubectl to get the object and check that the created object status is available.

Scenario 1

Deleting the helm release post-installation should cause the profile subscription status to update

The API needs to handle removing things from the profiles array which are associated with that catalog.

This would be useful to have sooner rather than later, because right now the integration tests keep failing when run multiple times as the in memory list persists between runs

We have no deletion logic in the reconciler yet.

All created child resources need to be scrapped if the DeletionTimestamp != nil.

If we set the owner on child objects, when we delete the pSub resource, they should go.

What feature/behavior/change do you want?

Install a profile by generating a ProfileSubscription, that can install Profile with a secret or configmap. (assuming config map exists in cluster.)

$ pctl install weaveworks/nginx-ingress --namespace europe --config-secret=service-config-secret
apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
...

I can then apply that yaml to my cluster and see that my profile is installed with my values set.

Why do you want this feature?

I want to be able to create a subscription to a specific profile.

It's not nice for several reasons:

Latest Tag on image pull and updating

Leaving the tag at least will significantly hinder the update process since we don't set image pull policy to always.

Image tag is always latest

This also means that on each tag release the tag version for the controller manager is actually always latest and not the version that we are publishing. That means if a user pulls / prepares a version say, v0.0.1, they will get a manifest file for which the tag will be latest and not actually the version they wanted.

Example UX:

$ pctl outdated --namespaces dev-pesto,dev-pitch-black
dev-pesto       weaveworks/nginx-ingress 1.10.2 [upgradable from: 1.10.1] auto (6hrs)
dev-pitch-black weaveworks/nginx-ingress 1.10.2 [upgradable from: 1.10.1] manual

Outcomes for this ticket:
The aim here is to investigate/play around/spike and come up with a good solution (or several potential solutions) for how we discover when profiles fall out of date.

Add any ideas with pros and cons in this thread (or link to a googledoc if the discussion and investigation could get lengthy).

This ticket is complete when a follow up user feature has been created to actually implement the work.

What feature/behavior/change do you want?

Currently, we support charts that are embedded in the profile itself, but we should be able to support upstream charts from chart repositories.

Why do you want this feature?

To simplify the installation of a provided Helm Chart.

This should use a HelmRepository rather than a GitRepository.

From #45 (comment)

how do we want to handle helm values now that the profile can contain multiple helmchart artifacts? Currently its 1 top-level set of values passed into both helmcharts, which obviously causes some limitations. Do we want to start scoping the variables per-artifacts?

Currently we set the default branch in the controller, let's have a webhook do that instead. We can add further things to this as we like.

Problem

Changes in profiles can break things in pctl. There are two types of breakages and changes:

Compile time

Some changes, like type renaming, api changes that are directly used and imported by pctl will break pctl's compile.

Logical

Changes to output, changes in profiles name or things in the YAML files which are installed and tested by pctl.

Solution 1

Version the usage of profile in pctl so breaking changes aren't so apparent. Versioning also has the benefit of setting up testing and overwriting a version to a latest tag. This requires two things.

Versioning in Profiles

We already do this now somewhat with tags and releases. It's just not happening every time now which is what we need. Publish more tags and release more versions.

Pin in Go modules

In pctl we need a pin to a specific version and add an ability to say update-modules.
Example:
go.mod

github.com/weaveworks/profiles v0.0.1

Testing

The test then can use a replace target call before running the test to update to the latest version:

go mod -replace github.com/weaveworks/[email protected]=github.com/weaveworks/profiles

As a user I want to subscribe to a Profile that has raw Kubernetes files in an artifacts directory.

This can be a simple deployment.yaml.

Currently,

To check your subscription was successful, you can inspect the nginx pod: kubectl describe pod [-n <namespace>] <pod-name>. The pod name will be comprised of profileSubscriptionName-profileDefinitionName-artifactName-xxxx

The separation character - is always the same, whether the sections have semantic meaning, or not.

Is there some advantage (besides convention) to separating artifactName and xxxx?
I feel like profileSubscriptionName-profileDefinitionName-artifactNamexxxx without the last - is already an improvement because it's 3 items (instead of 4) for a human to parse.

Could we use other UTF-8 characters here like
profileSubscriptionName::profileDefinitionName-artifactNamexxxx
or
profileSubscriptionName<>profileDefinitionName-artifactNamexxxx

What do other people think?

What feature/behavior/change do you want?
Add basic support for having a profile as an artifact

e.g. Profile A:

apiVersion: profiles.fluxcd.io/v1alpha1
kind: Profile
metadata:
  name: profile-A
spec:
  description: Profile for deploying nginx
  version: v0.0.1
  artifacts:
    - name: nginx-server
      path: nginx/chart
      kind: HelmChart

Profile B:

apiVersion: profiles.fluxcd.io/v1alpha1
kind: Profile
metadata:
  name: profile-B
spec:
  description: Profile for deploying nginx
  version: v0.0.1
  artifacts:
    - name: nginx-server
      profile:
        url: github.com/weaveworks/profile-a
        branch: main
      kind: Profile
    - name: nginx-consumer
      path: some/helm/chart
      kind: HelmChart

Why do you want this feature?
So that I can write more granular re-usable profiles

What feature/behavior/change do you want?

Show profile details

$ pctl show weaveworks/nginx-ingress --catalog-url foo-bar.com
Version: 1.10.1
Catalog: weaveworks (https://github.com/weaveworks/profiles)
URL: https://github.com/weaveworks/nginx-profile
Prerequisites: Kubernetes 1.18+
Description: NGINX Ingress Controller
Maintainer: WeaveWorks <[email protected]>

Why do you want this feature?

Before installing a profile, I want to make sure that it will work with my cluster and that it provides what I expect.

TODO more discussion needed

Kustomize CRD: https://toolkit.fluxcd.io/components/kustomize/kustomization/

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: nginx-profile-test
  namespace: default
spec:
  profileURL: https://github.com/weaveworks/nginx-profile
  branch: yaml
  patchesJson6902:
  - target:
      version: v1
      kind: Deployment
      name: podinfo
    patch:
    - op: add
      path: /metadata/annotations/key
      value: value

What feature/behavior/change do you want?
I want to be able to associate what catalog, profile & version a subscription is linked to.

Currently a ProfileSubscription spec looks like:

spec
  profileURL: https://github.com/weaveworks/nginx-profile
  branch: main

Using pctl you can install a profile from a catalog entry, e.g. :

pctl install nginx-catalog/weaveworks-nginx # generates profile_subscription.yaml

cat profile_subscription
...
spec:
  branch: main
  profileURL: https://github.com/weaveworks/nginx-profile

This doesn't leave any connection to what catalog entry was used to generate the profile. I want to be able to get the catalog name and version from the subscription.

Proposal

You can keep the current way of specifying a profile subscription directly:

spec
  profileURL: https://github.com/weaveworks/nginx-profile
  branch: main

Or alternatively you can just provide the catalog name, version and profile name:

spec
  profileName: nginx
  catalog: weaveworks-catalog
  version: 1.0

Which would get mutated by a mutating webhook to fill in the profileURL and branch by looking up the catalog entry, e.g.:

spec
  profileName: nginx
  catalog: weaveworks-catalog
  version: 1.0
  profileURL: https://github.com/weaveworks/nginx-profile
  branch: main

This might be a good opportunity to think about renaming the spec fields, and possibly grouping them to something like:

spec
  name: nginx
  catalog: weaveworks-catalog
  version: 1.0
  src: # something better than this
    url: https://github.com/weaveworks/nginx-profile
    branch: main # maybe "ref"?

Why do you want this feature?
so that I can clearly see what catalog & profile my profile subscription was generated from.

The pctl show commands shows details of a profile:

pctl --catalog-url http://localhost:8000 show <catalog-name>/<profile-name>

The catalog name here comes from metadata.Name of the ProfileCatalogSource resource, but one can add multiple ProfileCatalogSource resources with the same name to different namespaces. It's not clear what the show command should return in this case.

The API needs to handle altering profiles in the array which are associated with that catalog

Create a simple profile in a "profile repository".

apiVersion: profiles.fluxcd.io/v1alpha1
kind: Profile
metadata:
  name: nginx-profile
spec:
  description: <insert description>
  maturity: alpha
  owner: WeaveWorks <[email protected]>
  version: v0.0.1
  artifacts:
    # An artifact references a Helm chart, a local path, or a remote Profile.
    # version, url go together, and path is mutually exclusive with the url/version
    - name: nginx-server
      path: nginx/chart # <- relative to the profile.yaml, this gets the example nginx Helm chart we want to deploy.

This story involves creating a valid Helm chart and a profile.yaml that points to the Helm chart.

What feature/behavior/change do you want?
Currently we have a simple hard-coded set of profiles in a catalog source.

// ProfileCatalogSourceSpec defines the desired state of ProfileCatalogSource
type ProfileCatalogSourceSpec struct {
	// Profiles is the list of profiles exposed by the catalog
	Profiles []ProfileDescription `json:"profiles,omitempty"`
}

Possible solutions

GitRepository resource

This is a proposal to alternatively, parse a Flux GitRepository artifact for profiles, something like:

type GitRepositoryReference corev1.LocalObjectReference

// ProfileCatalogSourceSpec defines the desired state of ProfileCatalogSource
type ProfileCatalogSourceSpec struct {
	// Profiles is the list of profiles exposed by the catalog
	Profiles []ProfileDescription `json:"profiles,omitempty"`
        GitRepository *GitRepositoryReference `json:"gitRepository,omitempty"`
}

This would point at a Flux GitRepository in the same namespace (for simplicity of access).

We should grab the status from this object, which includes the SHA and the URL for downloading the source as a tarball.

We should look for profile.yaml files in the tarball, parse and index the data just as if it was an embedded profile.

Why do you want this feature?

This is necessary to access profiles stored in Git repositories.

I suspect we'll need more, to do versioning of the repo etc.

Outcomes

A set of proposals on how to solve this, with some (hacky) demos

What feature/behavior/change do you want?

As per story #26 but putting the profile subscription into a git repository.

As a user:

1. I have an existing gitops repo in Github. This repo is connected to Flux running in my cluster
2. I cd to a local directory which is my gitops repo
3. I run the pctl command to install a profile with secret values provided via a config map
4. I see output telling me that
   1. A new branch has been created in my repo
   2. A single commit has been added to that branch
   3. A PR has been opened on my gitops repo from that new branch against my default branch
   4. A link to the PR which I can merge
5. I can navigate to that PR in my gitops repo
6. I can see that the PR adds a single ProfileSubscription yaml file, which details the profile I requested in my initial install command.
7. I can merge that PR, and I should see that:
   1. My profile is installed into my cluster by Flux
   2. The values I have requested have been applied

Example UI:

$ pctl install weaveworks/nginx-ingress --namespace europe --config-secret=service-config-secret
Branch 'install-nginx-agf10' created with Profile installation.
Pull Request opened on GitHub repository https://github.com/repo.
You can merge this PR by visiting:
https://github.com/repo/pull/N

# user merges PR
# waits for flux to pick up

$ kubectl get pods # etc
# wooo such profile

Why do you want this feature?

We're GitOps!

Notes

Take a look at https://github.com/gitops-tools/pkg for creating a branch easily

All re-queuing is currently disabled.

We may want to do this for certain failures. Also nothing is currently checking that child resources have been created successfully. (This could be a separate story if needed).

What feature/behavior/change do you want?
After installing a profile I want to be notified when a newer version of a profile is available.

Example

If my catalog consists of:

apiVersion: weave.works/v1alpha1
kind: ProfileCatalogSource
metadata:
  name: nginx-catalog
spec:
  profiles:
    - name: nginx
      description: This installs nginx.
      version: 1.0.0
      url: https://github.com/weaveworks/nginx-profile
      maintainer: weaveworks (https://github.com/weaveworks/profiles)
      prerequisites:
      - Kubernetes 1.18+

and I deploy my subscription:

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: nginx-profile-test
  namespace: default
spec
  profileName: nginx
  catalog: nginx-catalog
  version: 1.0.0

and a new version 1.0.1 of nginx gets added to my catalog I should get a notification that my profile subscription is out of date.

How to notify the user

Lest start by updating the pctl get and pctl list to have a new column "Available Updates" that has a list of versions, e.g. "1.1.1, 1.2.0"
Why do you want this feature?
so that I can keep my subscription up to date with the latest updates

Motive

Currently the profile fetching in the Git implementation is not using authentication. We would like to provide private repository options as well, which means authentication needs to be implemented there.

Discussion

What kind of authentication that will be is up for debate and needs to be further discussed.

Proposition 1

A client factory.
TODO: flesh this out

Proposition 2

TODO: flesh this out

A chain pattern of providers.

As a user I want to subscribe to a Profile that has raw Kubernetes files in an artifacts directory.

This can be a simple deployment.yaml.

We should have a docs folder which contains something like:

• Architectural plans
• Feature roadmap checklist (basically something high level that anyone can look at to see where we are at and where are are going)

Feel free to add whatever else you think would be good. Make sure docs are linked from Readme.

We are generating a multi-arch docker build, but we pin the ARCH to amd64 in the Dockerfile. So they won't work basically.

Solution:

Use TARGETARCH ARG in the Dockerfile as input for the go build command. like:

ARG TARGETARCH
...
RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} ....

Documentation of the Argument: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope

Currently the HelmRepository resource name contains the branch, which feels slightly irrelevant. Lets stick to a similar naming convention that we have for HelmRelease

currently we only check 1 deep to see if the current profiles is artifacting itself. Lets add a check to ensure that no matter how many profiles down the recursion occurs it is caught.

E.g.:
Profile https://github.com/weaveworks/nginx-profile on branch main

apiVersion: profiles.fluxcd.io/v1alpha1
kind: Profile
metadata:
  name: nginx
spec:
  description: Profile for deploying nginx
  version: v0.0.1
  artifacts:
    - name: nested-profile
      kind: Profile
      profile:
        url: https://github.com/weaveworks/nginx-profile
        branch: helm-artifact     

Where helm-artifact is:

apiVersion: profiles.fluxcd.io/v1alpha1
kind: Profile
metadata:
  name: nginx
spec:
  description: Profile for deploying nginx
  version: v0.0.1
  artifacts:
    - name: nested-profile
      kind: Profile
      profile:
        url: https://github.com/weaveworks/nginx-profile
        branch: foo

Where foo is:

apiVersion: profiles.fluxcd.io/v1alpha1
kind: Profile
metadata:
  name: nginx
spec:
  description: Profile for deploying nginx
  version: v0.0.1
  artifacts:
    - name: nested-profile
      kind: Profile
      profile:
        url: https://github.com/weaveworks/nginx-profile
        branch: main

This should error out as we are installing profile https://github.com/weaveworks/nginx-profile on branch main that eventually contains itself as an artifact

As a user, I want to trigger the deployment of a Helm chart from a profile, by subscribing to a specific version of a profile.

apiVersion: ...
kind: ProfileSubscription
spec:
    profileURL: https://github.com/testing/testing.git # <- profile.yaml

Use kubectl to apply this yaml, and this should trigger a HelmRelease with chart from the profile.

This does not require any configuration.

What feature/behavior/change do you want?
After deploying my profile subscription with a set of values I want to be able to update the values and have the changes propagate to the running deployments

After #69 this might be a no-op for certain cases as changes to the subscription will result to changes in the artifacts. Regardless testing needs to be added + how this works when using a configmap instead of a list of values

Why do you want this feature?
So that I can update my deployments

Run pctl install <foo> then do ls -l. The generated files are all 0755. We should be generating 0644s.

What feature/behavior/change do you want?
Installing a profile.

This is about installing a profile to a "git repository".

Assuming that you have a cluster configuration repository, installation of a
profile should look like this:

This also assumes that the profile has helm-chart references.

$ pctl install https://github.com/weaveworks/nginx-profile.git

This should result in the generation of several files relative to the cluster
configuration repository root.

In the paths below, the nginx-profile part should come from the profile-name
(let's not worry too much about conflicts here).

/profiles/nginx-profile/profile.yaml
/profiles/nginx-profile/artifacts/helm-release.yaml
/profiles/nginx-profile/artifacts/helm-repository.yaml

To get there, I suspect that we need to rename the ProfileDefinition to a
Profile, there's no controller (initially) for this, but the CR is installed
into the cluster.

To deploy this, we'd be deploying the artifacts directory.

For this iteration, we need to side-step values configuration.

Why do you want this feature?

Add a Github Action which checks that PRs with the kind/feature labels have modified the README.md. The PR should not be mergeable without it. This may require a ticket/PR to corp to add this requirement to the branch protection rules

something along the lines of:

$ pctl show weaveworks/nginx-ingress
Version: 1.10.1
Catalog: weaveworks (https://github.com/weaveworks/profiles)
Prerequisites: Kubernetes 1.18+
Description: NGINX Ingress Controller
Maintainer: WeaveWorks <[email protected]>
Configurable values: containerPort, somethingElse...

not exact api, just illustrative

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: nginx-profile-test
  namespace: default
spec:
  profileURL: https://github.com/weaveworks/nginx-profile
  branch: main
  values:
     containerPort: 8080
     somethingElse: foo

What feature/behavior/change do you want?

Currently we cache the results, but we should only return results from ProfileCatalogSources that the "current user" has access to.

This probably involves querying for accessible sources, and then filtering the results based on the accessible elements, we don't need to reindex.

Why do you want this feature?

Enterprise access-control is used to control access to objects (via policies etc), and we should make it easy to prevent installation of profiles that user doesn't have access to.

Add a line to docs to tell users that to delete a profile they just need to remove the file in their gitiops repo and pr it.

We want users to be able to have multiple profiles in a repo. We also want users to use git tags to version profiles.

Lets document how users should layout there repo when it contains multiple profiles, and how to do tagging. Kustomize uses a relatively straight forward approach for tagging a repo that contains muiltiple products. For example it contains the following tags:

kyaml/v0.10.19
kyaml/v0.10.18
cmd/config/v0.9.11
cmd/config/v0.9.10

The prefix to the version references the component name and its path. So for example on the kyaml/v0.10.19 git tag, if you go to the kyaml directory it contains the kyaml source code for version v0.10.19.

The doc should:

• explaining our approach
• list out the required changes
  • e.g. Introducing tag field (placeholder #102)

What feature/behavior/change do you want?
If I create a profile that creates a HelmRelease resource, if a users tries to manually edit the spec the controller should overwrite any changes and it bring it back to the desired state. Essentially mirroring the same behaviour that would occur if you tried to modify a pod owned by a deployment.

The same should occur if a user deletes the HelmRelease, it should get recreated.

Why do you want this feature?
To prevent the artifact resources from drifting out of sync

As part of #108 we agreed how we will implement multi-profile repos and tagging. We now need to update the controller and profile subscription to pull from these repos. We now need to update the profile subscription to match:

Version

Version (git tag) workflow:

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: foo-test
  namespace: default
spec:
  profileURL: https://github.com/weaveworks/weaveworks-profiles
  version: foo/v0.1.2

This should result in the foo directory in git tag foo/v0.1.2 being the source of truth for this profile

Branch

Branch & path base workflow:

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: bar-test
  namespace: default
spec:
  profileURL: https://github.com/weaveworks/weaveworks-profiles
  branch: main
  path: bar/

This should result in the bar directory on branch main being the source of truth for this profile

What feature/behavior/change do you want?

Placeholder, may change depending on the output of #76

Follow up to #76, now we have documented and understand our approach for tagging we need to add support for getting a profile from a tag. Currently a subscription looks like:

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: nginx-profile-test
  namespace: default
spec:
  profileURL: https://github.com/weaveworks/nginx-profile
  branch: main

We use the profileURL and branch to fetch the profile. This story is to introduce a new alternative to the branch field, tag

apiVersion: weave.works/v1alpha1
kind: ProfileSubscription
metadata:
  name: nginx-profile-test
  namespace: default
spec:
  profileURL: https://github.com/weaveworks/nginx-profile
  tag: nginx/1.2.3

Why do you want this feature?
So I can install a profile from a tag

Currently in pkg/profile/artifact.go we simply get the first index of the artifacts array.

When deploying a Helm chart from a profile, I want to be able to provide configuration for the deployed chart.

apiVersion: ...
kind: ProfileSubscription
spec:
   profileURL: https://github.com/testing/testing.git
   values: # This should be taken from the config for the Nginx chart and applied to the HelmRelease
      foo: value1
      bar:
      baz: value2
      oof:
      - item1
      - item2
   valuesFrom:
   - configMapKeyRef:
       # Name of the config map
       name: default-values  # mandatory
       # Namespace of the config map
       namespace: my-ns      # optional; defaults to HelmRelease namespace
       # Key in the config map to get the values from
       key: values.yaml      # optional; defaults to values.yaml
       # If set to true successful retrieval of the values file is no
       # longer mandatory
       optional: false       # optional; defaults to false

  

This should work in Fluxv2.

Documentation for configuring HelmReleases https://github.com/fluxcd/helm-controller/blob/main/docs/spec/v2beta1/helmreleases.md#values-overrides

This is not concerned with Secrets or ConfigMaps, if a value is exposed by the Helm chart, this can configure it.

When a ProfileCatalogSource is removed via the HTTP API, its associated catalog continues to linger in the in-memory cache. The controller should handle this case and remove the catalog.