bitnami / kube-libsonnet Goto Github PK

kube-libsonnet currently creates DaemonSets, Deployments and StatefulSets with the apigroup apps/v1beta2. According to https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/ with Kubernetes 1.16 those API objects will no longer be served from anything but apps/v1.
From Kubernetes 1.18, Ingress will no longer be available in the extensions/v1beta1 group. Ingress should be update to use the networking.k8s.io/v1beta1 group.
Kube-libsonnet should be updated to reflect these changes.

Kubernetes v1.18 fully deprecates Ingress resource apiVersion extensions/v1beta1, need to use networking.k8s.io/v1beta1 which is available since v1.14.

We should fix it before beta, v1.18.0-alpha.1 has just been released, excerpt from https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.18.md#other-notable-changes

• kube-apiserver no longer serves the following deprecated APIs: (#85903, @liggitt)
  • [n/a] All resources under apps/v1beta1 and apps/v1beta2 - use apps/v1 instead
  • daemonsets, deployments, replicasets resources under extensions/v1beta1 - use apps/v1 instead [#23]
  • networkpolicies resources under extensions/v1beta1 - use networking.k8s.io/v1 instead [#34]
  • podsecuritypolicies resources under extensions/v1beta1 - use policy/v1beta1 instead [#23]

Steps to reproduce:

1. create pod with more than one container

Evaluation of this fails when put in tests directory:

local kube = import "../kube.libsonnet";
local simple_validate = (import "test-simple-validate.pass.jsonnet").items_;
simple_validate {
  pod+: {
    spec+: {
      containers_: {
        first: kube.Container("first") { image: "nginx" },
        second: kube.Container("second") { image: "nginx" },
      },
    },
  },
}

Cause

in kube.jibsonnet:

PodSpec: {
    // The 'first' container is used in various defaults in k8s.
    local container_names = std.objectFields(self.containers_),
    default_container:: if std.length(container_names) > 1 then "default" else container_names[0] 
    [...]
}

"default" as default_container should be used when std.length(container_names) < 1. In current code container_names[0] is used when container_names is empty.

The helper function in the container struct onyl supports args in the form
--arg=val
things like

-arg=1
-arg 1
--arg 1

are common usages of args, and not supported in kube-libsonnet

Please add a LICENSE for this code, so people know under what conditions they can integrate it.

Hi.
Great repo! Many thanks.

We noticed that using $() in env does not work 100% since this is order dependent, see link here.

Our solution atm is to prefix A_ on data variables.
Any tips on how to add/sort dependent variables?

Is this repo maintained @jjo @anguslees

Context

In K8s 1.18 a new field named pathType was introduced in the Ingress API. This field is mandatory on k8s > v1.19. Find more info below::

• https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/

Description

I have a solution running on a cluster that was recently moved to 1.19 managed with kubecfg and using this library. I tried today to update my solution, and I faced the error below while upgrading:

Error updating ingresses {{NAMESPASCE}}.{{INGRESS_NAME}}: Ingress.extensions "{{INGRESS_NAME}}" is invalid: spec.rules[0].http.paths[0].pathType: Required value: pathType must be specified

Am I missing something or do we need to update this library to include the new field?

Extra information

This is sth we solved in the Bitnami Helm catalog by implementing a macro that allows us to detect the k8s version and decide whether to include this parameter or not, see:

• https://github.com/bitnami/charts/blob/master/bitnami/common/templates/_ingress.tpl#L32
• https://github.com/bitnami/charts/blob/master/bitnami/nginx/templates/ingress.yaml#L30

Currently, Deployment's selector is populated based on deployment.spec.template.metadata.labels, which itself comes from the deployment.metadata.labels.

This works fine as long as the deployment.metadata.labels are not modified between two deployments. In the other cases, it will raise an immutable field error.

https://github.com/bitnami-labs/kube-libsonnet/blob/master/kube.libsonnet#L385-L387

Ultimately, pod labels can have for a wider range of use-cases which should not be tight to the Deployment selector.

An alternative approach would be to use a deterministic value to populate the the deployment.spec.template.metadata.labels and selector (eg: I think that ksonnet-lib uses the {app: metadata.name}).

Hi Bitnami Folks!

I'd like to gain some insight into your use of Jsonnet and/or Dhall, and if you could provide any knowledge/experience with either, such as how you debug problems in the code, problems with the language that need work, things that work really well, etc.

Thank you for your time!

Due to the same API deprecations/changes reported in #22 , the current ClusterRole defined here doesn't work anymore in recent Kubernetes versions (e.g 1.16) because pod kube-state-metrics pod doesn't get the needed authorizations to poll metrics.

I have an overlay library that sets defaults and adds various parameters to the base functions in this library such as:

local k = (import 'github.com/bitnami-labs/kube-libsonnet/kube.libsonnet');

k {
  Role(name, namespace):: super.Role(name) {
    metadata+: {
      namespace: namespace,
    },
  },
}

I found I need to wholesale replace ClusterRole otherwise it references Role which is in my overlay now a namespaced object.

ClusterRole(name): $._Object("rbac.authorization.k8s.io/v1", "ClusterRole", name),

The only deduplication that is gained by inheriting from Role is the apiVersion. I think it would be beneficial here to decouple Role from ClusterRole.

https://github.com/bitnami-labs/kube-libsonnet/blob/master/kube.libsonnet#L525
This selector is unnecessary, as kubernetes fills in the selector for you when you leave it out. By specifying it kubernetes complains that it isn't auto-generated but user generated.

https://github.com/bitnami-labs/kube-libsonnet/blob/master/kube.libsonnet#L448

Since it's not easy to remove fields from a jsonnet obj (google/jsonnet#312), and there's different behavior between:

replicas: null

and when the replicas field is missing.

In the latter case, the field isn't changed in the resulting object in Kubernetes. Thus, if you have a deployment replica count managed by an HPA (or similar), you want to exclude that field.

When a deployment is first deployed and the field doesn't exist, it it set to 1 in the final object in k8s.
If the deployment already exists and the field doesn't exist in the manifest being reapplied, the field remains unchanged.

replicas: null is essentially equivalent to replicas: 1.

So there's no way, if using this library to easily remove the replicas field in order to get the desired behavior of having that field managed externally.

As per https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22, we'll need to fix several library objects, notably:

• Ingress
  • non-backwards compatible fields
• CustomResourceDefinition
  • many changes here also, notably a versions map with required schema field

FYI above are already supported since v1.19

The PersistentVolumeClaim spec here is wrong for kubernetes 1.15 because it lacks the storageClassName field.

In order to allow the mapping with a statically provided PersistentVolume the storageClassName, if present, must be included in the spec as reported in the API reference.

I also created a PR

A couple of issues I'm finding:

• It references .spec.data instead of .spec.encryptedData.
• It assumes we're only passing in a single value (which it attempts to base64decode)

Something like this works for me:

# Create example secret and extra only the encrypted portion into `data.json`
kubectl create secret generic mysecret --dry-run --from-literal=foo=bar -o json >mysecret.json
kubeseal < mysecret.json | jq -r .spec.encryptedData > data.json

// Define SealedSecret (kube.libsonnet)
SealedSecret(name): $._Object('bitnami.com/v1alpha1', 'SealedSecret', name) {
},

// Create sealed-secret (calling code)
my_secret: kube.SealedSecret("mysecret") { 
    spec: {
      encryptedData: kubecfg.parseJson(importstr "./data.json"),
    },
  },

We've changed our certification management system in our clusters and the annotations have been changed. We should update them in the bitnami.libsonnet file.

It looks like the guestbook example is broken when using the kube-libsonnet at head. It looks like the breakage occurred in early April of last year. The issue appears to be that Service requires containers to have named ports, per this bit of logic:

name: service.target_pod.spec.containers[0].ports[0].name,

But the sample doesn't provide a name for the port.

trying the guestbook example:

jsonnet -version
Jsonnet commandline interpreter v0.11.2

[jsonnet_work]$ jsonnet guestbook.jsonnet 
RUNTIME ERROR: field does not exist: default_container
        lib/kube.libsonnet:257:100-122  thunk <b>
        lib/kube.libsonnet:257:95-122   function <anonymous>
        lib/kube.libsonnet:257:95-122   thunk <container_names_ordered>
        lib/kube.libsonnet:258:83-106   thunk <a>
        lib/kube.libsonnet:258:17-141   function <anonymous>
        lib/kube.libsonnet:258:17-141   object <anonymous>
        lib/kube.libsonnet:275:23-38
        lib/kube.libsonnet:275:12-39    thunk <object_assert>
        lib/kube.libsonnet:394:20-57    thunk <a>
        lib/kube.libsonnet:(392:22)-(396:10)    function <anonymous>
        lib/kube.libsonnet:(392:22)-(396:10)    thunk <pvcs>
        lib/kube.libsonnet:397:41-45
        lib/kube.libsonnet:397:30-46    thunk <a>
        lib/kube.libsonnet:397:30-51    function <anonymous>
        lib/kube.libsonnet:397:30-51    thunk <is_stateless>
        lib/kube.libsonnet:402:27-39    object <anonymous>
        lib/kube.libsonnet:(389:17)-(410:8)     object <anonymous>
        guestbook.jsonnet:(35:52)-(50:9)        object <anonymous>
        guestbook.jsonnet:(35:24)-(50:9)        object <anonymous>
        During manifestation