Giter VIP home page Giter VIP logo

korifi's Introduction

Korifi

Build Status Maintainability Test Coverage

This repository contains an experimental implementation of the Cloud Foundry V3 API that is backed entirely by Kubernetes custom resources.

For more information about what we're building, check out the Vision for CF on Kubernetes document.

Differences with Cloud Foundry for VMs

Korifi is very different from CF for VMs architecturally. Most core CF components have been replaced by more Kubernetes native equivalents. Check out our architecture docs to learn more.

Although we aim to preserve the core Cloud Foundry developer experience, there are some key behavior differences. We've listed some of these in our differences between Korifi and CF-for-VMs doc.

Additionally, we do not currently support all V3 CF APIs or filters. What we do support is tracked in our API endpoint docs.

Installation

Check our installation instructions.

Contributing

Please check our contributing guidelines and our good first issues.

This project follows Cloud Foundry Code of Conduct.

Hacking

Our hacking guide has instructions on how to work on the project locally.

License

This project is licensed under the Apache License, Version 2.0.

When using the Korifi or other Cloud Foundry logos be sure to follow the guidelines.

korifi's People

Contributors

acosta11 avatar akrishna90 avatar alperdedeoglu avatar angelachin avatar benjaminguttmann-avtq avatar beyhan avatar birdrock avatar clintyoshimura avatar danail-branekov avatar davewalter avatar dependabot[bot] avatar eghobo avatar emalm avatar gcapizzi avatar georgethebeatle avatar gnovv avatar julian-hj avatar kei-yamazaki avatar kieron-dev avatar kirederik avatar korifi-bot avatar marsteg avatar matt-royal avatar mnitchev avatar puremunky avatar saffronjam avatar spgreenberg avatar szeort avatar tcdowney avatar wanddynosios avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

korifi's Issues

[Feature]: Platform Operators can follow manual instructions to install kpack & create default Registry Credentials secret and ClusterBuilder, Stack, and Store for staging

Blockers/Dependencies

This should be done in serial with: #42

  • these stories interrelate
  • this story will define the initial conventions for configuring the ClusterBuilder, Registry Credentials, etc- like what their names should be

Background

As a Platform Operator
I want manual instructions to install kpack and create a default Registry Credentials secret and ClusterBuilder, Stack, and Store
So that Developers can stage applications using these as defaults

  • kpack CRs should be installed at the same time

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
WHEN I follow manual kpack install instructions included in the https://github.com/cloudfoundry/cf-k8s-controllers repo
THEN I see that a kpack secret, ClusterBuilder, ClusterStack, and ClusterStore have been created in the namespace I intended

Dev Notes

  • Point people to the kpack docs for kpack installation instructions
  • For now it is OK to have the Registry secrets to only work per-namespace and ask the Platform Operator to re-apply when new namespaces are added
  • Instructions for how to update Stack and Store are not in scope
  • Feel free to refer to the spike install script for inspiration

[Feature]: API Client can add a destination to a route via POST /v3/routes/<route-guid>/destinations

Blockers/Dependencies

  • Blocked on story to add DestinationGUIDs to CFRoute CR

Background

As a client of the API shim
I want to be able to add a destination to a Route
So that I can route traffic to an application

Acceptance Scenarios

GIVEN I have the API shim running locally
THEN I can make perform the following scenarios


Happy Path

GIVEN I have applied a CFRoute CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following request

curl "https://api.example.org/v3/routes/<route-guid>/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d '{
        "destinations": [
          {
            "app": {
              "guid": "1cb006ee-fb05-47e1-b541-c34179ddc446"
            }
          },
          {
            "app": {
              "guid": "01856e12-8ee8-11e9-98a5-bb397dbc818f",
              "process": {
                "type": "api"
              }
            },
            "port": 9000,
            "protocol": "http1"
          }
        ]
      }'

THEN I get a response that looks like the following

HTTP/1.1 200 OK
Content-Type: application/json

{
  "destinations": [
    {
      "guid": "89323d4e-2e84-43e7-83e9-adbf50a20c0e",
      "app": {
        "guid": "1cb006ee-fb05-47e1-b541-c34179ddc446",
        "process": {
          "type": "web"
        }
      },
      "weight": null,
      "port": 8080,
      "protocol": "http1"
    },
    {
      "guid": "fbef10a2-8ee7-11e9-aa2d-abeeaf7b83c5",
      "app": {
        "guid": "01856e12-8ee8-11e9-98a5-bb397dbc818f",
        "process": {
          "type": "api"
        }
      },
      "weight": null,
      "port": 9000,
      "protocol": "http1"
    }
  ],
  "links": {
    "self": {
      "href": "https://api.example.org/v3/routes/cbad697f-cac1-48f4-9017-ac08f39dfb31/destinations"
    },
    "route": {
      "href": "https://api.example.org/v3/routes/cbad697f-cac1-48f4-9017-ac08f39dfb31"
    }
  }
}

AND I can use kubectl to view the destinations in the CFRoute resource.


Route does not exist

GIVEN I have do not have a CFRoute with the GUID below
WHEN I make the following API request

curl "https://api.example.org/v3/routes/<non-existent-route-guid>/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d '{
        "destinations": [
          {
            "app": {
              "guid": "1cb006ee-fb05-47e1-b541-c34179ddc446"
            }
          },
          {
            "app": {
              "guid": "01856e12-8ee8-11e9-98a5-bb397dbc818f",
              "process": {
                "type": "api"
              }
            },
            "port": 9000,
            "protocol": "http1"
          }
        ]
      }'

THEN I get a response that looks like the following

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "Route is invalid. Ensure it exists and you have access to it.",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Invalid destination app (missing)

GIVEN I have applied a CFRoute CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following API request

curl "https://api.example.org/v3/routes/<non-existent-route-guid>/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d '{
        "destinations": [
          {
            "port": 9000,
            "protocol": "http1"
          }
        ]
      }'

THEN I get a response that looks like the following

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "Destinations[0]: must have an \"app\".",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Invalid destination app (missing GUID)

GIVEN I have applied a CFRoute CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following API request

curl "https://api.example.org/v3/routes/<non-existent-route-guid>/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d '{
        "destinations": [
          {
            "app": {}
            },
            "port": 9000,
            "protocol": "http1"
          }
        ]
      }'

THEN I get a response that looks like the following

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "Destinations[0]: app must have the structure {\"guid\": \"app_guid\"}",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Invalid destination app (missing process type)

GIVEN I have applied a CFRoute CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following API request

curl "https://api.example.org/v3/routes/<non-existent-route-guid>/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d '{
        "destinations": [
          {
            "app": {
              "guid": "01856e12-8ee8-11e9-98a5-bb397dbc818f",
              "process": {}
            },
            "port": 9000,
            "protocol": "http1"
          }
        ]
      }'

THEN I get a response that looks like the following

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "Destinations[0]: process must have the structure {\"type\": \"process_type\"}",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Invalid destination app (invalid protocol)

GIVEN I have applied a CFRoute CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following API request

curl "https://api.example.org/v3/routes/<non-existent-route-guid>/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d '{
        "destinations": [
          {
            "app": {
              "guid": "01856e12-8ee8-11e9-98a5-bb397dbc818f",
              "process": {
                "type": "api"
              }
            },
            "port": 9000,
            "protocol": "http"
          }
        ]
      }'

THEN I get a response that looks like the following

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "Destinations[0]: protocol must be 'http1'",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Documentation

Document this endpoint in our API docs

Dev Notes

[Feature]: Developer should only see spaces they belong to

Blockers

Background

At the moment, GET /v3/spaces returns the full list of spaces present on the foundation. We should only return the spaces the user making the request belongs to, e.g. spaces the user has some role in.

We want to keep this behaviour behind a feature flag for now, to avoid having a mix of authenticated and unauthenticated endpoints exposed.

Acceptance Criteria

GIVEN a cf root namespace exists
AND the cf/org1, cf/org2 and cf/org3 subnamespaces exist
AND the cf/org1/space1, cf/org2/space2 and cf/org3/space3 subnamespaces exist
AND user has RoleBindings (to any role for now) in org1 and org3
AND user has RoleBindings (to any role for now) in space1 and space3
WHEN I GET /v3/spaces with a correct Authorization header
THEN I only see space1 and space3 in the resources field

Dev Notes

  • Org Managers can list all spaces inside their org. While we don't need to implement the Org Manager role yet, let's take this into account while implementing this.
  • It is impossible in CF to have a space role without having a role in the corresponding org.

[Feature]: Developer can stop targeting a `cf-k8s` foundation via `cf api --unset`

Blockers/Dependencies

Background

cf api --unset allows users to remove any API information stored in their config without necessarily replace it with new information.

Acceptance Criteria

GIVEN I have targeted a cf-k8s foundation with cf api
WHEN I run cf api --unset
THEN I see all API information removed from $CF_HOME/config.json and CFOnK8s.Enabled set to false

Dev Notes

This should follow the same solution adopted for cloudfoundry/cf-k8s-api#70, but apply it to Actor.ClearTarget instead of Actor.SetTarget.

[Feature]: End user can configure controllers

Blockers/Dependencies

Blocked by #23

Blocked by a forthcoming RFC regardless choices of templating mechanism and tools.
Require input from:

  • European time zones
  • SAP

Background

As a end user
I want configuration options
So that modify the controller configuration

Acceptance Criteria

GIVEN a deployable controller set
WHEN I modify the configuration interface
THEN I see my changes reflected in on the running controller pod

  • Update README with conventions for configuration (defaults, structure, etc)

Dev Notes

This may be a combination of configuration options.

  • The image should be configurable, through some templating or patching solution (i.e.: kustomize, ytt, etc).
  • Any configuration that can be added by ConfigMaps and Secrets should be done that way, even if those are templated, as well.

`cf` can authenticate against a `cf-k8s` foundation using an `auth-provider` specified in `$KUBECONFIG`

$KUBECONFIG-compatible clients can use auth-provider plugins to generate bearer tokens. These plugins are configured using the auth-provider fields in AuthInfo. We want the cf CLI to be able to generate credentials via the same auth-provider plugins.

We should be able to accept this by using plain Kind, as it uses client certificates for authentication.

Dev notes

auth-provider plugins leverage hard-coded behaviour in client-go, but we know we can reuse them. cloudfoundry/cf-crd-explorations#64 has details on how to do it.

Blockers

[Feature]: Developer can switch to a different CF API via `cf api`

Blockers/Dependencies

Background

When switching from an API to the other, the CLI deletes the user authentication tokens so that the user is effectively logged out. When working against cf-k8s, the equivalent of those tokens is the reference to the auth-info in use, which in turn allows us to retrive the tokens/certs. When switching to a different API, this information should be deleted too.

Acceptance Criteria

GIVEN I am logged into a cf-k8s API (e.g. I have selected an auth-info)
WHEN I run cf api against a different cf-k8s API
THEN I see something like this in $CF_HOME/config.json:

"CFOnK8s": {
    "Enabled": true,
    "AuthInfo": ""
}

GIVEN I am logged into a cf-k8s API
WHEN I run cf api against a traditional CF API
THEN I see something like this in $CF_HOME/config.json:

"CFOnK8s": {
    "Enabled": false,
    "AuthInfo": ""
}

Dev Notes

There's a couple of ways we can achieve this.

  • One way is to add the Kubernetes specific behaviour to the existing one, as we know they're not going to interfere. This is what we do in the spike (although the functionality discussed in this story would be achieved by changing the SetTarget method, not ClearTarget).
  • Alternatively, we could consider adding a ClearTokenInformation method to the Config interface and have separate implementations. This would be more elegant but we would end up with an asymmetry: ClearTokenInformation would be used to undo SetTokenInformation in one case and to undo SetKubernetesUser in the other.

[Feature]: API Client can see that a CFRoute results in the creation of a Contour HTTPProxy

Blockers/Dependencies

Blocked by #56

Background

As an API Client
I want a CFRoute without any workload destinations to result in the creation of a Contour HTTPProxy CR
So that I can prepare to route traffic to my workload

The Goals of this issue are:

  • Implement Route Controller that reacts to CFRoutes and creates a Contour HTTPProxy based on its fields

Sample resources:

apiVersion: networking.cloudfoundry.org/v1alpha1
kind: CFDomain
metadata:
  name: <domain-guid>
  labels:
    networking.cloudfoundry.org/domainGUID: <domain-guid>
spec:
  name: vcap.me
apiVersion: networking.cloudfoundry.org/v1alpha1
kind: CFRoute
metadata:
  name: <route-guid>
  labels:
    networking.cloudfoundry.org/route-guid: <route-guid>
    networking.cloudfoundry.org/domain-guid: <domain-guid>
spec:
  host: subdomain
  path: /
  protocol: http
  domainRef:
    name: <domain-guid>
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: <route-guid>-proxy
  labels:
    networking.cloudfoundry.org/route-guid: <route-guid>
    networking.cloudfoundry.org/domain-guid: <domain-guid>
spec:
  virtualhost:
    fqdn: subdomain.vcap.me

Acceptance Criteria

GIVEN that I am targeting a K8s cluster
AND that I have followed the README instructions to install the Controllers
WHEN I apply the example CFDomain, and CFRoute resources in config/samples directory
THEN I should see all those operations succeed
WHEN I fetch the YAML for the Contour HTTPProxy CR from the K8s API
THEN I should see that it exists and its field values are populated based on its CFRoute's spec
AND I should see that the metadata.name field matches the metadata.name field of the corresponding CFRoute resource
AND I should see that the networking.cloudfoundry.org/domain-guid label has been set on the record and that it matches the spec.domainRef.name field of the corresponding CFRoute resource
AND I should see that the networking.cloudfoundry.org/route-guid label has been set on the record and that it matches the metadata.name field of the corresponding CFRoute resource

Dev Notes

  • For now, the routes section of the HTTPProxy resource will be empty
    • It will be populated by a later story that reacts to the addition of a destination to the corresponding CFRoute
  • We will need to configure RBAC to allow the creation of the HTTPProxy resource
  • If an HTTPProxy resource with the same GUID already exists, return success without creating a new HTTPProxy resource
  • HTTPProxy API Reference: https://projectcontour.io/docs/main/config/api/#projectcontour.io/v1.HTTPProxy

DUPLICATE: CI Placeholder

Blockers

Acceptance Criteria

AS a team member
WHEN I push a new commit to this repository
AND  that commit is successfully validated by our Continuous Integration system
THEN I should see that a new container image has been pushed to the team's public image registry (TBD?)
AND  I can follow instructions in the README to install the images into my cluster and see that they are running without error

Automate Image building & pushing for CF CRDs

Description

GitHub Actions allow running arbitrary commands on commits to branches.

Context

At the present time, we are optimizing for community engagement and simplicity. We should automate building and pushing images as part of the PR process that all contributors will follow.
Actions should be set up to build images for:

  • PR workflow
  • Commits to main

Acceptance Criteria

Given someone wants to introduce a change
When a PR is opened against the repository
Then an image is built and pushed to Dockerhub

Given main should be updated with changes from develop
When commits land on main
Then an image is built and pushed to Dockerhub

Dev Note

For now we will have all controller logic live in a single image. We may decide to separate them in the future. This image will not contain the webhook logic

API Client can retrieve a single domain via GET /v3/domains/:guid

Why

In support of routes in the CF V3 API, we would like implement read functionality for cf domain resources.

Blockers

Description

As a client of the CF API, I want to be able to retrieve a single CFDomain resource on Kubernetes via the API shim.

The behavior of this endpoint should match the existing behavior of theย GET /v3/domains/:guidย endpointย in CF for VMs as closely as possible -- including theย error cases.

For fields and relationships not included in the CFDomain, include the key but place an empty object in the value.

Acceptance Scenarios

GIVEN I have the API shim running locally
THEN I can make perform the following scenarios


Happy Path

GIVEN I have applied a CFRoute CR to the cluster (See cf-k8s-controllers samples) and know its "guid"
WHEN I make the following request

curl "https://api.example.org/v3/domains/[insert-guid-here]" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json"

THEN I get a response that looks like the following

HTTP/1.1 200 OK
Content-Type: application/json

{
  "guid": "3a5d3d89-3f89-4f05-8188-8a2b298c79d5",
  "created_at": "2019-03-08T01:06:19Z",
  "updated_at": "2019-03-08T01:06:19Z",
  "name": "test-domain.com",
  "internal": false,
  "router_group": null,
  "supported_protocols": ["http"],
  "metadata": {
    "labels": { },
    "annotations": { }
  },
  "relationships": {
    "organization": null,
    "shared_organizations": {
      "data": []
    }
  },
  "links": {
    "self": {
      "href": "https://api.example.org/v3/domains/3a5d3d89-3f89-4f05-8188-8a2b298c79d5"
    },
    "route_reservations": {
      "href": "https://api.example.org/v3/domains/3a5d3d89-3f89-4f05-8188-8a2b298c79d5/route_reservations"
    }
  }
}

Domain does not exist or cannot be found

WHEN I make the following request for an domain that does not exist

curl "https://api.example.org/v3/domains/[insert-guid-here]" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json"

THEN I get back the following response

HTTP/1.1 404 Not Found

{
  "errors": [
    {
      "detail": "Domain not found",
      "title": "CF-ResourceNotFound",
      "code": 10010
    }
  ]
}

Dev Notes

  • We're not supporting private domains right now so we can leave the fields around "shared private domains" empty. For context these are updated via the share/unshare domain endpoints: https://v3-apidocs.cloudfoundry.org/version/3.112.0/index.html#share-a-domain
  • We may not be able to generate a random domain for the end to end test, instead relying on the existing default domain guid in the installation as a subject of the test assertions until we implement the domain create api endpoint

[Feature]: Platform Operators can follow manual instructions to install Contour

Blockers/Dependencies

No response

Background

As a Platform Operator
I want manual instructions to install Contour
So that Developers can route traffic to workloads

Contour CRs should be installed at the same time

Acceptance Criteria

GIVEN that I am targeting a K8s cluster
WHEN I follow manual Contour install instructions included in the https://github.com/cloudfoundry/cf-k8s-controllers repo
THEN I see that Contour has been installed

Dev Notes

  • Point people to the Contour docs for installation instructions

[Feature]: API Client can see that Droplet app and build labels are set automatically

Blockers/Dependencies

Background

As an API Client
I want the appGUID and buildGUID labels on CFDroplet to be set automatically
So that I can look up CFDroplet records by appGUID and buildGUID

We encode the relationships between resources in two ways: via "ref" fields under spec and via metadata.labels. Rather than rely on our code and/or kubectl users to always remember both, we want to require the spec field and automatically set the label via a MutatingAdmissionWebhook

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the web hook
WHEN I apply the example CFApp, CFPackage, CFBuild and CFDroplet resource from the config/samples directory
THEN I should see all operations succeed
WHEN I fetch the yaml for the CFDroplet record from the k8s API
THEN I should see that the appGUID label has been set on the record and that it matches the spec.appRef.name field
AND I should see that the buildGUID label has been set on the record and that it matches the spec.buildRef.name field

Dev Notes

  • Remove the labels from the example resources if they are still present, and commit the change.
  • This should apply to creation and updates of records

[Feature]: App Developers have a default set of buildpacks that they can use

Blockers/Dependencies

BLOCKED BY: #43

Background

As a Developer
I want the default ClusterBuilder, Stack, and Buildpacks to be configurable by the Platform Operator
So that I can stage applications with a default set of buildpacks

The goal is to make a ConfigMap that contains references to a default ClusterBuilder, and Registry Credentials Secret that will be created as part of #45
Then the Build Controller should be modified to discover the ConfigMap and fill in the defaults on the kpack images it creates accordingly.

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
WHEN I follow manual kpack install instructions included in the https://github.com/cloudfoundry/cf-k8s-controllers repo
THEN I see that my CF Builds are staged using the default ClusterBuilder and Registry Credentials configured by the Platform Operator

Dev Notes

  • You can start with a ConfigMap with hardcoded name values and add templating to allow user-specified values later

Automatically set the appGUID label on CFApp records

We encode the relationships between resources in two ways: via "ref" fields under spec and via metadata.labels. Rather than rely on our code and/or kubectl users to always remember both, we want to require the spec field and automatically set the label via a MutatingAdmissionWebhook

Blockers

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND  that I have followed the README instructions to install the web hook
WHEN I apply the example CFApp resource in the repo (confirm ahead of time that it doesn't provide a GUID label)
THEN I should see it succeeds
WHEN I fetch the yaml for that record from the k8s API
THEN I should see that the GUID label has been set on the record and that it matches the `metadata.name` field

Dev Notes

  • We should be able to accomplish this via a MutatingAdmissionWebhook. The application that backs that hook should be the same that supports the ValidatingAdmissionWebhook from earlier issues (unless we have a good reason not to).
  • This should apply to creation and updates of records. One alternative to explore: can we only handle creation and then make the label immutable?

I can follow manual instructions to build and install validating webhooks

Blockers

  • #1 must be complete before picking this up

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
WHEN I read the README for this repo
THEN I see manual instructions for building an image for the validating web hook app and for installing it into my cluster
WHEN I follow those instructions
AND I create or modify a CFApp record in the k8s API
THEN I see log entries in the running validating web hook app that confirm it was called for the new or modified record
AND the record creation or modification should succeed 

NOTE: At his point the validating web hooks aren't ever rejecting updates, but they are properly wired in so they can do so in future stories

Dev Notes

For this story the webhook just needs to be running and wired into the k8s API. It will approve all requests. Also there is no behavior to test, so we can wait on writing specs

Register this webhook to handle CFApp resources only. We will add additional endpoints for other resources in the future.

This code should live in this repository. Also, the webhook server will also handle mutation in the future, not just validation

If we aren't able to configure the app to run insecurely, then do a one-off solution. For example, we could assume that the cert is in a secret and provide an example one to bootstrap the app, with the intention of properly generating it in the future. We'll solve the certificate generation issue properly in a follow-up story

[Feature]: API Client can see that CF Droplet process fields are set automatically

Blockers/Dependencies

BLOCKED BY: cloudfoundry/cf-crd-explorations#76
BLOCKED BY: #43

Background

As an API Client
I want a successful CF Build to have its process fields updated automatically based on the new kpack build image
So that I can configure a CF Process with the Droplet details

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the Controllers and set up the kpack Registry Credentials and staging CRs
WHEN I apply the example CFApp, CFPackage, and CFBuild resources in config/samples directory
THEN I should see all those operations succeed
WHEN I fetch the yaml for the CF Build CR after its kpack Build completes successfully and it contains the Droplet's image
THEN I should see that the CF Build also contains updated values for its processTypes and ports based on the OCI tags of the droplet image

Dev Notes

  • This update should happen at the same reconcile time that the build droplet image is updated
  • Feel free to refer to this implementation of the spike Droplet Controller for inspiration
  • For now, assume that the droplet image will not change once it is set, we will tackle this in a future issue
  • For this story, only integration testing is required, not end-to-end testing

[Feature]: End user can deploy controllers to cluster

Blockers/Dependencies

No response

Background

As an end user
I want to be able to deploy the controllers to a cluster
So that I can see resources reconciled

Acceptance Criteria

GIVEN I clone the repo and see the deployment yaml
WHEN I follow README steps to kubectl apply it
THEN I see a controller pod start up

Dev Notes

  • This should be a static deployment yaml.
  • We should think about how we keep the the static deployment in sync with our development process. Create a new story to automate this.
  • This may require some RBAC work as controllers are expanded.
  • The acceptance criteria specify README steps - these need to be updated.

[Feature]: API Client can see that Build app and package labels are set automatically

Blockers/Dependencies

Background

As an API Client
I want the appGUID and packageGUID labels on CFBuild to be set automatically
So that I can look up CFBuild records by appGUID and packageGUID

We encode the relationships between resources in two ways: via "ref" fields under spec and via metadata.labels. Rather than rely on our code and/or kubectl users to always remember both, we want to require the spec field and automatically set the label via a MutatingAdmissionWebhook

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the web hook
WHEN I apply the example CFApp, CFPackage, and CFBuild resources in config/samples directory
THEN I should see all those operations succeed
WHEN I fetch the yaml for the CFBuild record from the k8s API
THEN I should see that the appGUID label has been set on the record and that it matches the spec.appRef.name field
AND I should see that the packageGUID label has been set on the record and that it matches the spec.packageRef.name field

Dev Notes

  • Remove the labels from the example resources if they are still present, and commit the change.
  • This should apply to creation and updates of records

`cf` can authenticate against a `cf-k8s` foundation using the `client-certificate` and `client-key` in `$KUBECONFIG`

$KUBECONFIG can point to client certificate and key files via the top-level client-certificate and client-key fields in AuthInfo.

We want:

  • the cf CLI to be able to retrieve the certificate and key and pass them to the shim;
  • the shim to be able to use this to authenticate against the Kubernetes API.

We should be able to accept this by using plain Kind, as it uses client certificates for authentication.

Dev notes

  • Once this story and cloudfoundry/cf-k8s-api#13 are done, we should be able to use any combination of client-certificate/key and client-certificate/key-data
  • It's probably a good idea to reuse the Authorization header for this, but come up with a different type, e.g. Authorization: client-cert XXX.
  • The key could be either concatenated using a separator, or we could try to separate them: Authorization: client-cert XXX client-key YYY
  • Remember that Authorization types are case-insensitive!

Blockers

[Feature]: Merge CF Build and CF Droplet CRs

Background

After the explore to merge CF Build and Droplet, we created a proposal CR and spike implementation.

Please modify the CF Build CR to match the spec from the proposal, and delete the CF Droplet, then clean up any orphaned files.

Additionally, please update the sample CRs.

Example CR:

---
# Defines a build for the provided package. Triggers the staging process which results in a runnable container image.
# Successful CF Builds automatically receive CF Droplets.
apiVersion: apps.cloudfoundry.org/v1alpha1
kind: CFBuild
metadata:
  # apps.cloudfoundry.org/ labels are all managed by a mutating webhook
  labels:
    apps.cloudfoundry.org/appGuid: 14dcda7d-1fa1-4a91-b437-fbdba20e8c5a
    apps.cloudfoundry.org/packageGuid: ac85ad52-f52f-48e3-8c99-5e7badbe79c5
  name: 1591ee05-e208-4cf3-a662-1c2da42f20a7
  namespace: default
  # ownerReferences are managed by mutating webhook that looks at appRef
  ownerReferences:
    - apiVersion: apps.cloudfoundry.org/v1alpha1
      kind: CFApp
      name: 14dcda7d-1fa1-4a91-b437-fbdba20e8c5a
      uid: 2c5e9145-dee1-407f-b507-4ea84b25a7b4
spec:
  appRef:
    name: 14dcda7d-1fa1-4a91-b437-fbdba20e8c5a
  packageRef:
    name: ac85ad52-f52f-48e3-8c99-5e7badbe79c5
  stagingMemoryMB: 1024
  stagingDiskMB: 1024 # <----------- can we even do this?
  lifecycle:
    type: buildpack
    data:
      buildpacks: []
      stack: cflinuxfs3
status:
  conditions:
    - type: Succeeded
      status: "True"
      reason: Buildpack
      message: ""
    - type: Staging
      status: "False"
      reason: Succeeded
      message: ""
    - type: BuildReady
      status: "True"
      reason: Buildpack
      message: ""
    - type: DropletReady
      status: "True"
      reason: Buildpack
      message: ""
  droplet:
    ports: [80, 443] # spec.ports is the set of ports exposed on the Processes of the Droplet
    processTypes:
      - type: web
        command: bundle exec rackup config.ru -p $PORT -o 0.0.0.0
      - type: worker
        command: bundle exec rackup config.ru
    registry:
      image: gcr.io/cf-relint-greengrass/cf-crd-staging-spike/buildpack/14dcda7d-1fa1-4a91-b437-fbdba20e8c5a@sha256:17ef1315d87bb57657ee14f387394f56d6f4429151262d731a31e92e5497ad35
      imagePullSecrets:
      - name: app-registry-credentials

Action to take

Delete CF Droplet CR
Modify CF Build CR to align with proposal
Clean up & regenerate files
Add new example build CR to config/samples

Impact

We haven't implemented the controllers / endpoints for the Build/Droplet CR so there should be minimal changes required to the API shim and controllers.

Dev Notes

  • No tests should be required to make this change
  • Please ensure that the sample CR is apply-able with the new CF Build CRD
  • Leave the references on other CRs the same -- they can still say dropletRef. Eventually if we find there are other consumers of those CRs we can consider adding additional fields like kind and apiVersion if we want.
  • Memory and disk limits are present on CF Builds -- we are adding them last minute here. Think about how they're formatted and if disk makes sense in this context. We use them in cloudfoundry/cf-k8s-api#53

[Explore]: Packaging source code as Carvel's imgpkg

Background

In the CF API Spike implementation, the uploaded source code is converted into a Single Layer OCI image. One of the feedback comment received on this approach is to convert the uploaded source code zip into Carvel's imgpkg instead of a single layer OCI image.ย 

CF for VMs context
Duringย cf pushย the CLI will package up the local app source code into a zip file (referred to as appย bits) and send it to the Cloud Controller API. The Cloud Controller API will do some additional processing (a big portion of this is a feature calledย resource matching) and send the app bits to the packages bucket in theย blobstore. The associated CF Package for these bits will be updated to have the stateย READYย to indicate that the source code has been fully uploaded.

CF for K8s context
In CF for K8s we useย kpackย for application staging and provide the app source code as an OCI image. In this flow the CLI will still package the local app source code as a zip file and sent it to the Cloud Controller API. Instead of sending it to a blobstore, though, Cloud Controller will pass the zip off to another component calledย registry-buddyย whichย converts the zip file into a single-layer OCI imageย that contains the app source. It then pushes that image to an image registry. Once this is done the associated CF Package is updated to have the stateย READYย and Cloud Controller implicitly knows where the image was pushed to based on the configured image registry / package guid.

Some of the questions we would like to answer through this explore is to

  • Feasibility - We plan on using kpack for application staging in CF on K8s. Will moving to a imgpkg format break the feasibility of staging with kpack?

  • Complexity - Do we take up additional complexity to convert the source code to a imgpkg and to stage with kpack. How will re-staging work when there are stack updates?

  • Benefits - Do we see benefits of using imgpkg be of value to CF users.ย 

  • Are there any downsides?

Acceptance Criteria

By the end of this explore we should have a ADR of packaging tool - use of carvelsimgpkg as compared to use of single layer OCI image.

Number of days this exploration is expected to take. This is what is โ€œpointedโ€.

Time box: 2 days

Dev Notes

No response

[Explore]: How should we structure k8s resource labels?

Background

For the CRD spike we used labels like apps.cloudfoundry.org/appGuid and apps.cloudfoundry.org/processGuid. These were copied over from past Eirini work, and we want to reconsider the correct domain and label structure. We also want to ensure that our labels conform with the kubernetes label recommendations

Acceptance Criteria

Write up an ADR as a PR with a proposed set of labels for the following cases:

  • app GUID
  • build GUID
  • package GUID
  • process GUID
  • process type

Number of days this exploration is expected to take. This is what is โ€œpointedโ€.

Timeboxed to 1 day

No response

Dev Notes

No response

[Feature]: API Client can see that Package app label is set automatically

Blockers/Dependencies

Background

As an API Client
I want the appGUID label on CFPackage to be set automatically
So that I can look up CFPackage records by appGUID

We encode the relationships between resources in two ways: via "ref" fields under spec and via metadata.labels. Rather than rely on our code and/or kubectl users to always remember both, we want to require the spec field and automatically set the label via a MutatingAdmissionWebhook

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the web hook
WHEN I apply the example CFApp resource in the repo
AND I apply the example CFPackage resource in the repo
THEN I should see both operations succeed
WHEN I fetch the yaml for the CFPackage record from the k8s API
THEN I should see that the appGUID label has been set on the record and that it matches the spec.appRef.name field

Dev Notes

  • Remove the labels from the example resources if they are still present, and commit the change.
  • This should apply to creation and updates of records
  • Prior art exists for the label on CFApp

Explore: What is the role of CF Metadata in CF on Kubernetes

Context

The CF API has the concept of metadata (labels and annotations) that can be applied to most V3 resources. For the most part, CF labels and annotations are equivalent to Kubernetes labels and annotations and are used the same way. CF users can filter V3 APIs using the same set of label selector criteria as they can on Kubernetes. For example, the following request could be used to only return apps that were labeled env=dev, not have a chageback-code label, and that had a tier label set to backend or worker.

cf curl /v3/apps?label_selector=env=dev,%21chargeback-code,tier%20in%20%28backend,worker%29

If we want to continue supporting the label_selector filter we may need to propagate these labels through to the underlying Kubernetes custom resources.

Security Concerns in cf-for-k8s

We did not propagate CF metadata to the underlying Kubernetes resources in cf-for-k8s out of security concerns. Since we did not intend on developers having full kubectl access to the cluster, there were concerns that the ability to propagate labels through the CF APIs to the backing K8s cluster might allow a malicious (or unwitting) user to bypass NetworkPolicy, break routing, or interfere with existing apps. Since permissions were handled by Cloud Controller and all apps shared the cf-workloads namespace, this could be problematic.

Update:
CF for K8s does have an allowlist of prometheus.io prefixed annotations that it will pass through for app metrics. Some prior discussion on that and some of these security concerns can be found here: cloudfoundry/eirini#82

In CF on Kubernetes, however, we are intending to support direct kubectl access and will rely on Kubernetes RBAC to make authorization decisions which changes how strict we need to be.

What to explore

Explore if and how we should integrate CF Metadata in CF on Kubernetes and come up with a suggestion to share with the team. Some things to think about:

  • Are there security concerns still?
  • Is it ok to pass through all labels?
  • Do any of our controllers use labels internally? Is it alright if users can override these? Should we have a denylist of reserved label keys?
  • In CF we reserve the cloudfoundry.org prefix and prevent users from creating metadata under it. Should we continue doing that / should we make use of that?

Notes

[Feature]: API Client can see that a CF Build + kpack Image & kpack Build results in a CF Droplet

Blockers/Dependencies

BLOCKED BY: #42
BLOCKED BY: #52

Background

As an API Client
I want a CF Build along with its kpack Image & Build CRs to result in a CF Droplet automatically
So that I can look up the latest built image by inspecting the CF Build and its Droplet

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the Controllers and set up the kpack Registry Credentials and staging CRs
WHEN I apply the example CFApp, CFPackage, and CFBuild resources in config/samples directory
THEN I should see all those operations succeed
WHEN I fetch the yaml for the CF Build CR after its kpack Build completes successfully
THEN I should see that it now contains the Droplet's image

Dev Notes

  • Droplet processTypes and ports will be updated at the same time in a future story #44
  • Feel free to refer to the spike implementations of the Build Controller and the cf-to-kpack Build Controller for inspiration
  • Updating the Droplet's Process fields will be handled in a followup issue
  • For now ignore when a kpack Image is rebuilt and a new kpack Build is created after the first one, do not overwrite the droplet image

Prevent the creation of CFApps with spec.names that are taken

Data validation is something that we'll need to handle in both the CF API shim and when records are modified directly in the k8s API. Ideally we will minimize duplicate effort between codebase, so keep that in mind for this work.

Blockers

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND  that I have followed the instructions to install the web hook app into my cluster
WHEN I attempt to create a CFApp record that has the same spec.name as another CFApp in the same namespace
THEN I should see that the request to the k8s API fails with a validation failure message
AND  that the change was not applied 

Dev Note

  • Look into the standards for validation errors on k8s. If possible, use a machine-friendly code for the error so the API shim can react to it
  • Part of this story is defining the testing structure for webhooks
  • End-to-end testing is out of scope for this story

`cf` can authenticate against a `cf-k8s` foundation using the inline `client-certificate-data` and `client-key-data` in `$KUBECONFIG`

$KUBECONFIG can store an inline client certificate and key in the top-level client-certificate-data and client-key-data fields in AuthInfo.

We want:

  • the cf CLI to be able to retrieve the certificate and key and pass them to the shim;
  • the shim to be able to use this to authenticate against the Kubernetes API.

We should be able to accept this by using plain Kind, as it uses client certificates for authentication.

Dev notes

  • Once this story and #190 are done, we should be able to use any combination of client-certificate/key and client-certificate/key-data
  • It's probably a good idea to reuse the Authorization header for this, but come up with a different type, e.g. Authorization: client-cert XXX.
  • The key could be either concatenated using a separator, or we could try to separate them: Authorization: client-cert XXX client-key YYY
  • Remember that Authorization types are case-insensitive!

Blockers

[Feature]: API Client can see that CFRoute labels are set automatically

Blockers/Dependencies

No response

Background

As an API Client
I want the routeGUID and domainGUID labels on CFRoute resources to be set automatically
So that I can look up CFRoute records by these fields

We encode the relationships between resources in two ways: via "ref" fields under spec and via metadata.labels. Rather than rely on our code and/or kubectl users to always remember both, we want to require the spec field and automatically set the label via a MutatingAdmissionWebhook.

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the webhook
WHEN I apply the example CFDomain and CFRoute resources from the config/samples directory
THEN I should see all operations succeed
WHEN I fetch the YAML for the CFRoute record from the K8s API
THEN I should see that the networking.cloudfoundry.org/domain-guid label has been set on the record and that it matches the spec.domainRef.name field
AND I should see that the networking.cloudfoundry.org/route-guid label has been set on the record and that it matches the metadata.name field

Dev Notes

  • Remove the labels from the example resources if they are still present, and commit the change.
  • This should apply to creation and updates of records

[Feature]: API Client can list domains via GET /v3/domains (with filtering by name)

Blockers/Dependencies

Background

As a client of the API shim
I want to be able to list Domains
So that I can see all the domains that I have access to

As a client of the API shim
I want to be able to filter the list Domains by domain name
So that I can see a subset of the domains that I have access to

Acceptance Scenarios

GIVEN I have the API shim running locally
THEN I can make perform the following scenarios


Happy Path (when at least one domain exists)

GIVEN I have applied a CFDomain CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following request

curl "https://api.example.org/v3/domains" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \

THEN I get a response that looks like the following

HTTP/1.1 200 OK
Content-Type: application/json

{
  "pagination": {
    "total_results": 1,
    "total_pages": 1,
    "first": {
      "href": "https://api.example.org/v3/domains?page=1"
    },
    "last": {
      "href": "https://api.example.org/v3/domains?page=1"
    },
    "next": null,
    "previous": null
  },
  "resources": [
    {
      "guid": "5b5032ab-7fc8-4da5-b853-821fd1879201",
      "created_at": "2019-03-08T01:06:19Z",
      "updated_at": "2019-03-08T01:06:19Z",
      "name": "cf-apps.io",
      "internal": false,
      "router_group": null,
      "supported_protocols": ["http"],
      "metadata": {
        "labels": {},
        "annotations": {}
      },
      "relationships": {
        "organization": {
          "data": null
        },
        "shared_organizations": {
          "data": []
        }
      },
      "links": {
        "self": {
          "href": "https://api.example.org/v3/domains/5b5032ab-7fc8-4da5-b853-821fd1879201"
        },
        "route_reservations": {
          "href": "https://api.example.org/v3/domains/5b5032ab-7fc8-4da5-b853-821fd1879201/route_reservations"
        },
        "router_group": null
      }
    },
  ]
}

Happy Path (when no domains exist)

GIVEN that no CFDomain resources exist
WHEN I make the following request

curl "https://api.example.org/v3/domains" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \

THEN I get a response that looks like the following

HTTP/1.1 200 OK
Content-Type: application/json

{
  "pagination": {
    "total_results": 1,
    "total_pages": 1,
    "first": {
      "href": "https://api.example.org/v3/domains?page=1"
    },
    "last": {
      "href": "https://api.example.org/v3/domains?page=1"
    },
    "next": null,
    "previous": null
  },
  "resources": []
}

Happy Path (when at least one matching domain exists)

GIVEN I have applied a CFDomain CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following request

curl "https://api.example.org/v3/domains?names=cf-apps.io" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \

THEN I get a response that looks like the following

HTTP/1.1 200 OK
Content-Type: application/json

{
  "pagination": {
    "total_results": 1,
    "total_pages": 1,
    "first": {
      "href": "https://api.example.org/v3/domains?page=1&names=cf-apps.io"
    },
    "last": {
      "href": "https://api.example.org/v3/domains?page=1&names=cf-apps.io"
    },
    "next": null,
    "previous": null
  },
  "resources": [
    {
      "guid": "5b5032ab-7fc8-4da5-b853-821fd1879201",
      "created_at": "2019-03-08T01:06:19Z",
      "updated_at": "2019-03-08T01:06:19Z",
      "name": "cf-apps.io",
      "internal": false,
      "router_group": null,
      "supported_protocols": ["http"],
      "metadata": {
        "labels": {},
        "annotations": {}
      },
      "relationships": {
        "organization": {
          "data": null
        },
        "shared_organizations": {
          "data": []
        }
      },
      "links": {
        "self": {
          "href": "https://api.example.org/v3/domains/5b5032ab-7fc8-4da5-b853-821fd1879201"
        },
        "route_reservations": {
          "href": "https://api.example.org/v3/domains/5b5032ab-7fc8-4da5-b853-821fd1879201/route_reservations"
        },
        "router_group": null
      }
    },
  ]
}

Happy Path (when no matching domains exist)

GIVEN I have applied a CFDomain CR to the cluster (see cf-k8s-controllers samples)
WHEN I make the following request

curl "https://api.example.org/v3/domains?names=example.org" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \

THEN I get a response that looks like the following

HTTP/1.1 200 OK
Content-Type: application/json

{
  "pagination": {
    "total_results": 1,
    "total_pages": 1,
    "first": {
      "href": "https://api.example.org/v3/domains?page=1&names=example.org"
    },
    "last": {
      "href": "https://api.example.org/v3/domains?page=1&names=example.org"
    },
    "next": null,
    "previous": null
  },
  "resources": []
}

Sad Path (invalid query parameter)

WHEN I make the following request

curl "https://api.example.org/v3/domains?foo=bar" \
  -X GET \
  -H "Authorization: bearer <some-placeholder-token>" \

THEN I get a response that looks like the following

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
  "errors": [
    {
      "detail": "The query parameter is invalid: Valid parameters are: 'names'",
      "title": "CF-BadQueryParameter",
      "code": 10005
    }
  ]
}

Documentation

Document this endpoint in our API docs

Dev Notes

[Feature]: API Client can see that a CF Build results in kpack Image

Blockers/Dependencies

BLOCKED BY: cloudfoundry/cf-crd-explorations#76

Background

As an API Client
I want a CF Build to result in a kpack Image CR created automatically
So that I can build runnable images

The Goals of this issue are:

  • Implement Build Controller that reacts to CF Builds and creates kpack Image based on its fields

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the Controllers
WHEN I apply the example CFApp, CFPackage, and CFBuild resources in config/samples directory
THEN I should see all those operations succeed
WHEN I fetch the yaml for the kpack Image CR from the k8s API
THEN I should see that it exists and its field values are populated based on its CFBuild's spec

Dev Notes

  • For now please ignore the lifecycle block of the CF Build and always use a default ClusterBuilder
  • Labels should be added to the kpack Image for its app-guid and build-guid
    • OwnerReferences are out of scope for now
  • Feel free to refer to the spike implementation of the Build Controller for inspiration

[Explore]: Source code to Image conversion

Background

Blocked By
[Explore]: Packaging source code as Carvel's imgpkg

Based on the recommendation from cloudfoundry/cf-k8s-api#35, explore on how to convert source code into the recommended image format.

Things to remember

  • Ease of Scalability
  • Performance

Acceptance Criteria

The pair working on this story can decide b/w documenting the decision in an ADR or a proposal document.

Number of days this exploration is expected to take. This is what is โ€œpointedโ€.

No response

Dev Notes

  • One approach is to implement this as a separate service 'registry-service' or another way would be to make it part of the api-shim itself.

[Feature]: API client can assign an organisation role to a user with `POST /v3/roles`

Background

As an admin
I want to assign organisation roles to users
So that I can control permissions in the organisation

Acceptance Criteria

GIVEN an organisation exists
AND a user other than the admin exists
AND a ClusterRole exists
WHEN I make the following request:

curl "https://api.example.org/v3/roles" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
      "type": "<the ClusterRole name>",
      "relationships": {
        "user": {
          "data": {
            "username": "<the username>"
          }
        },
        "organization": {
          "data": {
            "guid": "<the org GUID>"
          }
        }
      }
    }'

THEN I get the following response:

HTTP/1.1 201 Created
Content-Type: application/json
{
   "guid": "40557c70-d1bd-4976-a2ab-a85f5e882418",
   "created_at": "2019-10-10T17:19:12Z",
   "updated_at": "2019-10-10T17:19:12Z",
   "type": "<the ClusterRole name>",
   "relationships": {
      "user": {
         "data": {
            "guid": "<the username>"
         }
      },
      "organization": {
         "data": {
            "guid": "<the org GUID>"
         }
      },
      "space": {
         "data": null
      }
   },
   "links": {
      "self": {
         "href": "https://api.example.org/v3/roles/40557c70-d1bd-4976-a2ab-a85f5e882418"
      },
      "organization": {
         "href": "https://api.example.org/v3/organizations/05c5da3b-6cbc-421c-87c3-20bb3c41ab7c"
      }
   }
}

AND I see a RoleBinding in the organisation namespace for the specified user

Dev Notes

[Feature]: Developer can `cf logout` from a `cf-k8s` foundation

Blockers/Dependencies

Background

cf login run against a cf-k8s foundation doesn't really perform a log in: we ask the user to select an auth-info which has been previously generated. For consistency and symmetry, we want to forget the selected auth-info on cf logout. cf logout does not remove any information about the the targeted API.

Acceptance Criteria

GIVEN I am logged into a cf-k8s foundation
WHEN I run cf logout
THEN I I should see something like this in $CF_HOME/config.json:

"CFOnK8s": {
    "Enabled": true,
    "AuthInfo": ""
}

Migrate cf-k8s-controllers to a multi-group kubebuilder project

Background

With the initial CRDs implemented in the single workloads.cloudfoundry.org group, we realized it may be better to organize the resources in separate groups. Because there is a relatively small blast radius to the change earlier on in the project, we should address the tech debt of a monolithic api group now.

See https://kubebuilder.io/migration/multi-group.html.

Follow the kubebuilder migration docs https://kubebuilder.io/migration/multi-group.html to reorganize the project to support multiple api groups.

Acceptance Criteria

Based directly on the multi-group kubebuilder documentation:
GIVEN/WHEN I view the github.com/cloudfoundry/cf-k8s-controller repository
THEN I see:

  • The CFRoute and CFDomain custom resources are in the networking.cloudfoundry.org group

  • The api directory is now an apis directory with workloads and networking subdirectories to hold the now separately organized resources

  • The controllers directory now contains workloads and networking subdirectories to hold the now separately organized controllers

  • The kubebuilder PROJECT file is marked as multigroup: true

AND WHEN I run tests
THEN I see the envtest based controllers/group.suite_test.go files correctly install CRDs to the test environment

Impact

Organizing custom resources and controllers more granularly in the controller repository allows for clearer separation of concerns between the different technical domains of the cf implementation.

Dev Notes

We expect the new directory structure to affect imports currently in use by the first set of cf api shim stories like:

workloadsv1alpha1 "code.cloudfoundry.org/cf-k8s-controllers/api/v1alpha1"

becomes:

workloadsv1alpha1 "code.cloudfoundry.org/cf-k8s-controllers/apis/workloads/v1alpha1"

Set status conditions on the CFApp resource

Blockers

  • #1 must be completed first

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND   the cluster has the CRDs from this project installed (via `make install`)
AND   I am running this app via `make run`
WHEN I create a new CFApp record
THEN I should see that it has a `status.conditions` entry with `type` "Running" and `status` "False"
AND  I should see a second `status.conditions` entry with `type` "Restarting" and `status` "False"

Also...

GIVEN that I have this repo checked out and it is my working directory
WHEN I run `go test ./...`
THEN I should see that tests of the CFApp controller run and pass

Dev Notes

We agreed as a team to use spec as our test framework and gomega as our matcher/expectation package.
The pair on this should explore whether unit testing is feasible or it envtest-driven integration tests are our best option.

[Feature]: API Client can add a destination to a route

Blockers/Dependencies

Blocked by cloudfoundry/cf-k8s-api#38.

Background

As a client of the CF API
I want to add a destination to a CFRoute resource
So that that I can route traffic to apps on CF on Kubernetes

The behavior of this endpoint should match the existing behavior of the POST /v3/routes/[guid]/destinations endpoint in CF for VMs as closely as possible -- including the error cases. Limit the scope of response data to exclude a distinct destination GUID for now, we would like to defer the decision of destination representation. For basic cases, copying the app guid should suffice, especially given the initial focus on single-process apps

Acceptance Criteria

GIVEN I have the API shim running locally according to the README
AND I have created the sample cfdomain.yaml, cfroute.yaml, and cfapp.yaml resources from cf-k8s-controllers on my target kubernetes cluster.
THEN I can perform the following scenarios

Happy Path

WHEN I make the following request

curl "https://api.example.org/v3/routes/84afc0f2-3dc2-4709-9fc8-8340be584a43/destinations" \
  -X POST \
  -H "Authorization: bearer [token]"
  -H "Content-type: application/json"
  -d '{
    "destinations": [
      {
        "app": {
          "guid": "14dcda7d-1fa1-4a91-b437-fbdba20e8c5a"
        }
      }
    ]
  }'

THEN I get the following response

HTTP/1.1 200 OK
Content-Type: application/json

  {
    "destinations": [
      {
        "guid": "14dcda7d-1fa1-4a91-b437-fbdba20e8c5a",
        "app": {
          "guid": "14dcda7d-1fa1-4a91-b437-fbdba20e8c5a",
          "process": {
            "type": "web"
          }
        },
        "weight": null,
        "port": 8080,
        "protocol": "http2"
      },
    ],
    "links": {
      "self": {
        "href": "https://api.example.org/v3/routes/84afc0f2-3dc2-4709-9fc8-8340be584a43/destinations"
      },
      "route": {
        "href": "https://api.example.org/v3/routes/84afc0f2-3dc2-4709-9fc8-8340be584a43"
      }
    }
  }

Request is invalid (invalid request body)

WHEN I make the following request

curl "https://api.example.org/v3/routes/[guid]/destinations" \
  -X POST \
  -H "Authorization: bearer <some-placeholder-token>" \
  -H "Content-type: application/json" \
  -d 'bad json'

THEN The CFRoute CR is not updated
AND I get back the following response

HTTP/1.1 400 Bad Request

{
  "errors": [
    {
      "detail": "Request invalid due to parse error: invalid request body",
      "title": "CF-MessageParseError",
      "code": 1001
    }
  ]
}

Request is invalid (route does not exist)

WHEN I make the following request

curl "https://api.example.org/v3/routes/not-a-route-guid/destinations" \
  -X POST \
  -H "Authorization: bearer [token]"
  -H "Content-type: application/json"
  -d '{
    "destinations": [
      {
        "app": {
          "guid": "not-an-app-guid"
        }
      }
    ]
  }'

THEN the CFRoute is not updated
AND I receive the following response

HTTP/1.1 404 Not Found

{
  "errors": [
    {
      "detail": "Route not found",
      "title": "CF-ResourceNotFound",
      "code": 10010
    }
  ]
}

Request is invalid (app does not exist )

WHEN I make the following request

curl "https://api.example.org/v3/routes/84afc0f2-3dc2-4709-9fc8-8340be584a43/destinations" \
  -X POST \
  -H "Authorization: bearer [token]"
  -H "Content-type: application/json"
  -d '{
    "destinations": [
      {
        "app": {
          "guid": "not-an-app-guid"
        }
      }
    ]
  }'

THEN the CFRoute is not updated
AND I receive the following response

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "App(s) with guid(s) \"not-an-app-guid\" do not exist or you do not have access.",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Request is invalid (app does not exist in the same space as the route)

WHEN I make the following request

curl "https://api.example.org/v3/routes/84afc0f2-3dc2-4709-9fc8-8340be584a43/destinations" \
  -X POST \
  -H "Authorization: bearer [token]"
  -H "Content-type: application/json"
  -d '{
    "destinations": [
      {
        "app": {
          "guid": "different-space-app-guid"
        }
      }
    ]
  }'

THEN the CFRoute is not updated
AND I receive the following response

HTTP/1.1 422 Unprocessable Entity

{
  "errors": [
    {
      "detail": "App(s) with guid(s) \"different-space-app-guid\" do not exist or you do not have access.",
      "title": "CF-UnprocessableEntity",
      "code": 10008
    }
  ]
}

Dev Notes

No response

Bootstrap this repo as a kubebuilder project and add the CRDs defined in previous work

To enable future work, we need this codebase to be set up as a kubebuilder project that contains the Custom Resource Definitions described in cloudfoundry/cf-crd-explorations#67

Acceptance Criteria

GIVEN that I am targeting a kubernetes cluster
AND   that I have this repository checked out locally
WHEN I run "make install" in this repo
AND  I run `kubectl api-resources`
THEN I see all of the CF-prefixed resources listed
WHEN I inspect each resource
THEN I see that it matches what we agreed upon in cf-crd-explorations pull request 67
AND  also I should see that there are example resource definitions in the `samples` directory for each CRD kind.

Dev Notes

Use the CRDs from cloudfoundry/cf-crd-explorations#67, but don't treat it being shared with the community as a blocker

No unit tests should be necessary for this work.

For the CRDs, mark which fields are required, valid enum types, and any other API schema constraints that apply.

**Moved** - [Feature]: Shim API can handle package uploads

Moving issue to cf-k8s-api repo

Background

The CLI will package the local app source code as a zip file and sends it to the shim API. This zip file is later converted to an artifact consumable by kpack for application staging.

As a CLI User
I want the POST /v3/packages/:guid/upload endpoint (see docs)
So that I can upload app source code as zip file

  • This endpoint should handle the file upload
  • App source zip should be uploaded as bits, if user doesn't provide any files or provides files of other type, or the bits key is missing return an appropriate error from error list. For all error cases to handle refer this and here

Acceptance Criteria

Happy Path

GIVEN that I am targeting a k8s cluster
WHEN I follow the instructions in README to deploy the shim and run the following curl command

    curl "http://api-shim.example.org/v3/packages/[guid]/upload" \
                  -X POST \
                  -F bits=@"path-to-app-source.zip" 

THEN I get a HTTP 200OK response

Dev Notes

  • REST API request would be a MULTIPART upload.

[Feature]: API client can assign a space role to a user with `POST /v3/roles`

Background

As an admin
I want to assign space roles to users
So that I can control permissions in the space

Acceptance Criteria

GIVEN a space exists
AND a user other than the admin exists
AND a ClusterRole exists
WHEN I make the following request:

curl "https://api.example.org/v3/roles" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
      "type": "<the ClusterRole name>",
      "relationships": {
        "user": {
          "data": {
            "guid": "<the username>"
          }
        },
        "space": {
          "data": {
            "guid": "<the space GUID>"
          }
        }
      }
    }'

THEN I get the following response:

HTTP/1.1 201 Created
Content-Type: application/json
{
   "guid": "40557c70-d1bd-4976-a2ab-a85f5e882418",
   "created_at": "2019-10-10T17:19:12Z",
   "updated_at": "2019-10-10T17:19:12Z",
   "type": "<the ClusterRole name>",
   "relationships": {
      "user": {
         "data": {
            "guid": "<the username>"
         }
      },
      "space": {
         "data": {
            "guid": "<the space GUID>"
         }
      },
      "organization": {
         "data": null
      }
   },
   "links": {
      "self": {
         "href": "https://api.example.org/v3/roles/40557c70-d1bd-4976-a2ab-a85f5e882418"
      },
      "space": {
         "href": "https://api.example.org/v3/spaces/05c5da3b-6cbc-421c-87c3-20bb3c41ab7c"
      }
   }
}

AND I see a RoleBinding in the space namespace for the specified user

Dev Notes

[Feature]: Platform Operator can follow manual instructions to configure ingress

Blockers/Dependencies

Depends on #56. Extends the requirements to also include configuration of a load balancer in front of a cluster and the initial manual configuration of CFDomain resources in the absence of a v3/domains endpoint and corresponding controller reconciliation loop.

Background

As a Platform Operator
I want manual instructions to configure my ingress and CF domains
So that I can receive external traffic to my platform

Acceptance Criteria

GIVEN I am targeting a K8s cluster with the cf shim and contour installed
WHEN I follow manual instructions for the load balancer and CF domain configuration
THEN I can externally access my cluster and see traffic from the load balancer routed on a real domain

Dev Notes

  • Assumes CF domain management is deferred
  • Assumes consumption of the gateway API for ingress configuration is deferred
  • We can start with instructions for GKE.

[Feature]: API Client can see that Process labels are set automatically

Blockers/Dependencies

Background

As an API Client
I want the appGUID, processGUID and processType labels on CFProcess to be set automatically
So that I can look up CFProcess records by these fields

We encode the relationships between resources in two ways: via "ref" fields under spec and via metadata.labels. Rather than rely on our code and/or kubectl users to always remember both, we want to require the spec field and automatically set the label via a MutatingAdmissionWebhook

Acceptance Criteria

GIVEN that I am targeting a k8s cluster
AND that I have followed the README instructions to install the web hook
WHEN I apply the example CFApp and CFProcess resources from the config/samples directory
THEN I should see all operations succeed
WHEN I fetch the yaml for the CFProcess record from the k8s API
THEN I should see that the appGUID label has been set on the record and that it matches the spec.appRef.name field
AND I should see that the processGUID label has been set on the record and that it matches the metadata.name field
AND I should see that the processType label has been set on the record and that it matches the spec.processType field

Dev Notes

  • Remove the labels from the example resources if they are still present, and commit the change.
  • This should apply to creation and updates of records

[Feature]: API Client can see that the addition of a destination to a CFRoute results in the update of the Contour HTTPProxy and the creation of a Service

Blockers/Dependencies

Blocked by #57

Background

As an API Client
I want a CFRoute with a workload destinations to result in the update of a Contour HTTPProxy CR and the creation of a Service CR
So that I can route traffic to my workload

The Goals of this issue are:

  • Modify Route Controller that reacts to CFRoutes, update the corresponding Contour HTTPProxy, and creates a Service based on its fields

Sample resources:

apiVersion: networking.cloudfoundry.org/v1alpha1
kind: CFDomain
metadata:
  name: <domain-guid>
  labels:
    networking.cloudfoundry.org/domainGUID: <domain-guid>
spec:
  name: vcap.me
apiVersion: networking.cloudfoundry.org/v1alpha1
kind: CFRoute
metadata:
  name: <route-guid>
  labels:
    networking.cloudfoundry.org/route-guid: <route-guid>
    networking.cloudfoundry.org/domain-guid: <domain-guid>
spec:
  host: subdomain
  path: /
  protocol: http
  domainRef:
    name: <domain-guid>
  destinations:
  - port: 8080
    appRef:
      name: <app-guid>
    processType: web
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: <route-guid>-proxy
  labels:
    networking.cloudfoundry.org/route-guid: <route-guid>
    networking.cloudfoundry.org/domain-guid: <domain-guid>
spec:
  virtualhost:
    fqdn: subdomain.vcap.me
  routes:
  - conditions:
    - prefix: /
    services:
      - name: <app-guid>-<route-guid>-service
        port: 80
apiVersion: v1
kind: Service
metadata:
  name: <app-guid>-<route-guid>-service
  labels:
    workloads.cloudfoundry.org/app-guid: <app-guid>
    networking.cloudfoundry.org/route-guid: <route-guid>
    networking.cloudfoundry.org/domain-guid: <domain-guid>
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    workloads.cloudfoundry.org/app-guid: <app-guid>
  sessionAffinity: None
  type: ClusterIP

Acceptance Criteria

GIVEN that I am targeting a K8s cluster
AND that I have followed the README instructions to install the Controllers
WHEN I apply the cfdomain.yaml and cfroute.yaml resources in config/samples directory
THEN I should see all those operations succeed
WHEN I fetch the YAML for the Contour root HTTPProxy resource from the K8s API with kubectl describe httpproxy my-host.cf-apps.io
THEN I should see that it exists and its includes field is populated with an entry matching the route-guid used as the name of the sub-HTTPProxy validated below
WHEN I fetch the YAML for the sub-HTTPProxy resource from the K8s API with kubectl describe httpproxy 84afc0f2-3dc2-4709-9fc8-8340be584a43
THEN I should see that it exists and its routes field is populated with a condition based on its CFRoute's destination path and a service entry with a name matching the service validated below
WHEN I fetch the YAML for the Service CR from the K8s API with kubectl describe service s-745cfe64-ac7a-4d4a-a627-c481e8f03be8-web
THEN I should see that it exists and its selector is populated based on its CFRoute's destination app-guid and process-type
AND I should see that the networking.cloudfoundry.org/domain-guid label has been set on the Service record and that it matches the spec.domainRef.name field of the corresponding CFRoute resource
AND I should see that the networking.cloudfoundry.org/route-guid label has been set on the Service record and that it matches the metadata.name field of the corresponding CFRoute resource

Dev Notes

  • We will need to configure RBAC to allow the creation of the Service resource
  • This story target the "update" flow, where the route is created with no destinations, and then updated with destinations as a separate step.

Add CFRoute and CFDomain CRDs

Why
With an initial design of a CFRoute Custom Resource and a minimal stub of CFDomain complete, we would like to bootstrap the cf-k8s-controllers codebase with the proposed spec for use by the api shim client (and subsequently the controllers themselves).

Acceptance Criteria
GIVEN that I am targeting a kubernetes cluster
AND that I have this repository checked out locally
WHEN I run "make install" in this repo
AND I run kubectl api-resources
THEN I see the CFRoute resource listed
AND I also see the CFDomain resource listed
WHEN I inspect each resource
THEN I see that it matches the spec in the cf-crd-explorations proposal
AND also I should see that there is an example CFRoute definition in the samples directory.

Notes
Don't treat further feedback from the community as a blocker, as we can still revise the spec after the initial implementation.

No unit tests should be necessary for this work.

For the CRDs, mark which fields are required, valid enum types, and any other API schema constraints that apply. (And note where constraints may depend on a separately tracked decision like the resource GUID)

Instead of making a pull request with the spec formalized as a yaml file in the cf-crds-exploration repo we would like to use the yaml embedded in the proposal doc directly.

Related Docs
[Exploration doc with CFRoute and CFDomain spec|https://docs.google.com/document/d/1WUiFwMgKkZrHHbqL1sMuHdqempzfIiYXpnq6qoazVKc]

[GUID RFC Doc|https://docs.google.com/document/d/1lUZ1kyZpExJNOHhXkFp0tqkjLJgEFqUFSyTDOoKCXuk/]

Set up GitHub Actions to run unit tests on PRs for CF CRDs

Description

GitHub Actions allow running arbitrary commands as validation checks before PRs can be merged.

Context

At the present time, we are optimizing for community engagement and simplicity. We should automate running unit tests as part of the PR process that all contributors will follow, so that the PR author can get feedback about merge-ability.

Acceptance Criteria

Given someone wants to introduce a change
When a PR is opened against the repository
Then unit tests are run and reported to the PR

AND...

Given someone wants to introduce a change directly to the main branch
When the commit is pushed
Then unit tests are run and reported in github (not sure where? research as part of this story)

[Feature]: API Client cannot create a duplicate route

Blockers/Dependencies

No response

Background

As a client of the API shim
I want to receive an error when I attempt to create a duplicate route based on its FQDN + context path
So that I am prevented from creating a route if it already exists

Acceptance Criteria

With context path

GIVEN I have an existing route (e.g. foo.example.com/bar )
WHEN I try to make another route with that same host, domain, and context path
THEN receive a 422 error

{
  "errors": [
    {
      "code": 10008,
      "detail": "Route already exists with host 'foo' and path '/bar' for domain 'atom-kitten.capi.land'.",
      "title": "CF-UnprocessableEntity"
    }
  ]
}

Without context path

GIVEN I have an existing route (e.g. foo.example.com )
WHEN I try to make another route with that same host and domain
THEN receive a 422 error

{
  "errors": [
    {
      "code": 10008,
      "detail": "Route already exists with host 'foo' for domain 'atom-kitten.capi.land'.",
      "title": "CF-UnprocessableEntity"
    }
  ]
}

Dev Notes

  • Start by implementing this the same way as we do for app names (with leases)
    • We recognize that the direct kubectl apply use case may not have sufficient guards against tampering, but the main API use case remains covered

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.