programming-kubernetes / cnat Goto Github PK

View Code? Open in Web Editor NEW

186.0 186.0 84.0 28.67 MB

Example Kubernetes controller: the cloud native at command

License: Apache License 2.0

Dockerfile 0.68% Shell 2.66% Go 95.58% Makefile 1.08%

kubernetes operator

cnat's People

Contributors

Stargazers

Watchers

Forkers

s-you sttts sthaha jianhuabi codesnk dshakes peishuli vamossagar12 nelvadas uday1bhanu softdev8 raulkite jmjava2020 sslavic clementblaise kishoregv azurecloudmonk chrisgreene jayryu benluteijn shashankv02 jayeshthamke aisonaku liya-xu nagarajgoud dmoore62 walk1ng akankshakumari393 freddyisaac avs6 sfdc-amenges shamhub just-do1 gnodli evanlixin deployment-io bishgupp outsideris zerodayz motionguided pierrelucboivin reonard myeung18 0xack13 nikawang jxlwqq sm43 jmmartin92 limoncho-san rexagod mandyl rnddevlgcp xinzhongyoumeng xiangyujiang-61 somtochiama cyber33 zhanchangbao njnuwjq vince15dk goops-top cszixin algobot76 partuoli allenliu88 yngwiewang yapcheekian xuanyingcool xiaobaowan1988 xiaomizhg forbearing tonychen15 fcl20052 elpsyr dimab07 tuxnotes mandreap ldd91 jrodonnell neisw c0deoo1 hamdikh harshitasao kangaechu xiaoliang2333

cnat's Issues

[Issue] [Chapter4.4.3] controller-runtime work with CR

I noticed this chapter reuse the go type and scheme from chapter 4.4.2. and I did the same as the book taught. like below

import (
	ctv1 "typedClient/apis/stable/v1"
	"typedClient/client/clientset/versioned/scheme"
)

it failed. the structure of my project is below, currently, we are located in the controllerRuntime folder, and each of these folders is a go module.

├── Readme.md
├── controllerRuntime
├── dynamicClient
└── typedClient

I tried to use the replace statement to fix this issue. it seems not to work.

require (
	typedClient v0.0.0
)

replace typedClient => ../typedClient

the error message like below, do you know the reason? thanks.

typedClient/client/clientset/versioned/scheme is not be found in GOROOT.

At last, I use go vendor, and copy typedClient under the folder vendor. it works. but I don't think this is the right way.

Running on GKE, need to use Upadate() instead of UpdateStatus() in cnat-client-go/controller.go.

Hi, in my set-up (Google Kubernetes Engine), as the CustomResourceSubresources feature gate is not enabled, I get the following error while running the cnat-client-go/controller.go:

E1012 22:10:45.503547 52194 controller.go:193] error syncing 'default/example-at': the server could not find the requested resource (put ats.cnat.programming-kubernetes.info example-at), requeuing

This could be resolved by replacing UpdateStatus(instance) with Update(instance) on L295

_, err = c.cnatClientset.CnatV1alpha1().Ats(instance.Namespace).UpdateStatus(instance)

as described in the sample-controller:

	// If the CustomResourceSubresources feature gate is not enabled,
	// we must use Update instead of UpdateStatus to update the Status block of the Foo resource.
	// UpdateStatus will not allow changes to the Spec of the resource,
	// which is ideal for ensuring nothing other than resource status has been updated.
	_, err := c.sampleclientset.SamplecontrollerV1alpha1().Foos(foo.Namespace).Update(fooCopy)

Does it make sense to add some logic to cnat-client-go/controller.go so that it can handle both set-ups (the CustomResourceSubresources feature gate is enabled / disabled)?

bug: odd number of arguments passed as key-value pairs

cnat/cnat-operator/pkg/controller/at/at_controller.go

Line 112 in 27f8ddb

reqLogger.Info("Schedule parsing done", "Result", "diff", fmt.Sprintf("%v", d))

Should the query back object be deepcopied before update it?

Please see the line here:

cnat/cnat-client-go/controller.go

Line 228 in 33d5f46

instance.Status.Phase = cnatv1alpha1.PhasePending

The object is query back using:

cnat/cnat-client-go/controller.go

Line 214 in 33d5f46

instance, err := c.atLister.Ats(namespace).Get(name)

So, should we call deep copy on this returned object and update it on the copied one?
When we should use the deep copy function that code-gen generated? I see official sample use the deep copy explicitly but there is no mentioned about this in your code.

Update-codegen.sh get a wrong package name

git clone https://github.com/kubernetes/sample-controller
modify the sample-controller to cnat-client-go
modify pkg/apis/samplecontroller to pkg/apis/samplecontroller
modify the hack/update-codegen.sh as following

set -o errexit
set -o nounset
set -o pipefail

SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
CODEGEN_PKG=${CODEGEN_PKG:-$(cd "${SCRIPT_ROOT}"; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../code-generator)}

# generate the code with:
# --output-base    because this script should also be able to run inside the vendor dir of
#                  k8s.io/kubernetes. The output-base is needed for the generators to output into the vendor dir
#                  instead of the $GOPATH directly. For normal projects this can be dropped.
"${CODEGEN_PKG}"/generate-groups.sh "deepcopy,client,informer,lister" \
  github.com/programming-kubernetes/cnat/cnat-client-go/pkg/generated github.com/programming-kubernetes/cnat/cnat-client-go/pkg/apis \
  cnat:v1alpha1 \
  --output-base "$(dirname "${BASH_SOURCE[0]}")/../../../../.." \
  --go-header-file "${SCRIPT_ROOT}"/hack/boilerplate.go.txt

exec the codegen script

./hack/update-codegen.sh

but i get a wrong package name, like this,
samplecontrollerv1alpha1 "k8s.io/sample-controller/pkg/apis/cnat/samplecontroller/v1alpha1
could anyone tell me how i can fix this?

"error": "ats.cnat.programming-kubernetes.info \"example-at\" not found"

I'm getting the following error

2020-09-19T13:16:17.164+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "at", "source": "kind source: /, Kind="}
2020-09-19T13:16:17.164+0530    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2020-09-19T13:16:17.365+0530    INFO    controller-runtime.controller   Starting Controller     {"controller": "at"}
2020-09-19T13:16:17.365+0530    INFO    controller-runtime.controller   Starting workers        {"controller": "at", "worker count": 1}
2020-09-19T13:16:17.366+0530    INFO    controllers.At  === Reconciling At      {"namespace": "cnat", "at": "example-at"}
2020-09-19T13:16:17.366+0530    INFO    controllers.At  Phase: PENDING  {"namespace": "cnat", "at": "example-at"}
2020-09-19T13:16:17.366+0530    INFO    controllers.At  Checking schedule       {"namespace": "cnat", "at": "example-at", "Target": "2019-04-12T10:12:00Z"}
2020-09-19T13:16:17.366+0530    INFO    controllers.At  It's time!      {"namespace": "cnat", "at": "example-at", "ready to execute": "echo YAY!"}
2020-09-19T13:16:17.366+0530    INFO    controllers.At  === Updating status     {"namespace": "cnat", "at": "example-at"}
2020-09-19T13:16:17.682+0530    ERROR   controller-runtime.controller   Reconciler error        {"controller": "at", "request": "cnat/example-at", "error": "ats.cnat.programming-kubernetes.info \"example-at\" not found"}
github.com/go-logr/zapr.(*zapLogger).Error
        /Users/ishankhare/godev/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /Users/ishankhare/godev/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /Users/ishankhare/godev/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
        /Users/ishankhare/godev/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
        /Users/ishankhare/godev/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /Users/ishankhare/godev/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
        /Users/ishankhare/godev/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88
^Cmake: *** [run] Error 1

I can find the said cr if i run kubectl get at example-at . But controller reports not found .

The specific line causing the error in my opinion is err = r.Status().Update(context.TODO(), instance)

Below is my whole code for at_controllers.go

package controllers

import (
	"context"

	"github.com/go-logr/logr"
	"k8s.io/apimachinery/pkg/api/errors"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/types"
	ctrl "sigs.k8s.io/controller-runtime"
	"sigs.k8s.io/controller-runtime/pkg/client"
	"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
	"sigs.k8s.io/controller-runtime/pkg/reconcile"

	cnatv1alpha1 "cnat-controller/api/v1alpha1"
	"cnat-controller/pkg/scheduler"
	"cnat-controller/pkg/spawn"

	corev1 "k8s.io/api/core/v1"
)

// AtReconciler reconciles a At object
type AtReconciler struct {
	client.Client
	Log    logr.Logger
	Scheme *runtime.Scheme
}

// +kubebuilder:rbac:groups=cnat.programming-kubernetes.info,resources=ats,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=cnat.programming-kubernetes.info,resources=ats/status,verbs=get;update;patch

func (r *AtReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
	_ = context.Background()
	reqLogger := r.Log.WithValues("namespace", req.Namespace, "at", req.Name)
	reqLogger.Info("=== Reconciling At")

	// your logic here

	// fetch the At instance
	instance := &cnatv1alpha1.At{}
	err := r.Get(context.TODO(), req.NamespacedName, instance)
	if err != nil {
		if errors.IsNotFound(err) {
			return reconcile.Result{}, nil
		}

		return reconcile.Result{}, err
	}

	// if no phase set, default to pending (the initial phase):
	if instance.Status.Phase == "" {
		instance.Status.Phase = cnatv1alpha1.PhasePending
	}

	// the state diagram PENDING -> RUNNING -> DONE
	switch instance.Status.Phase {
	case cnatv1alpha1.PhasePending:
		reqLogger.Info("Phase: PENDING")
		reqLogger.Info("Checking schedule", "Target", instance.Spec.Schedule)

		// check with a tolerance of 2 seconds
		d, err := scheduler.TimeUntilSchedule(instance.Spec.Schedule)
		if err != nil {
			reqLogger.Error(err, "Schedule parsing failure")
			return reconcile.Result{}, err
		}

		// reqLogger.Info("Schedule parsing done", "Result", "diff", fmt.Sprintf("%v", d))

		if d > 0 {
			// Not yet time to execute, wait until scheduled time
			return reconcile.Result{RequeueAfter: d}, nil
		}

		reqLogger.Info("It's time!", "ready to execute", instance.Spec.Command)
		instance.Status.Phase = cnatv1alpha1.PhaseRunning
	case cnatv1alpha1.PhaseRunning:
		reqLogger.Info("Phase: RUNNING")
		pod := spawn.NewPodForCR(instance)

		// set At instance as owner and controller
		err := controllerutil.SetControllerReference(instance, pod, r.Scheme)
		if err != nil {
			return reconcile.Result{}, err
		}

		query := &corev1.Pod{}
		nsName := types.NamespacedName{Name: pod.Name, Namespace: pod.Namespace}
		err = r.Get(context.TODO(), nsName, query)

		if err != nil && errors.IsNotFound(err) {
			// create the pod
			err = r.Create(context.TODO(), pod)
			if err != nil {
				return reconcile.Result{}, err
			}
			reqLogger.Info("Pod Launched", "name", pod.Name)
		} else if err != nil {
			return reconcile.Result{}, err
		} else if query.Status.Phase == corev1.PodFailed ||
			query.Status.Phase == corev1.PodSucceeded {
			reqLogger.Info("Container terminated", "reason",
				query.Status.Reason, "message", query.Status.Message)
			instance.Status.Phase = cnatv1alpha1.PhaseDone
		} else {
			// don't requeue, it will happen automatically when pod status changes
			return reconcile.Result{}, nil
		}
	case cnatv1alpha1.PhaseDone:
		reqLogger.Info("Phase: DONE")
		return reconcile.Result{}, nil
	default:
		reqLogger.Info("NOP")
		return reconcile.Result{}, nil
	}

	// update the At instance
	reqLogger.Info("=== Updating status")
	err = r.Status().Update(context.TODO(), instance)
	if err != nil {
		reqLogger.Info("Unable to update status")
		return reconcile.Result{}, err
	}

	return reconcile.Result{}, nil
}

func (r *AtReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&cnatv1alpha1.At{}).
		Complete(r)
}

[Issue] no kind "CronTab" is registered for version "stable.example.com/v1"

I made a copy from cnat to crontab, below is its definition of CRD

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must match the spec fields below, and be in the form: <plural>.<group>
  name: crontabs.stable.example.com
spec:
  # group name to use for REST API: /apis/<group>/<version>
  group: stable.example.com
  # list of versions supported by this CustomResourceDefinition
  versions:
    - name: v1
      # Each version can be enabled/disabled by Served flag.
      served: true
      # One and only one version must be marked as the storage version.
      storage: true
      additionalPrinterColumns:
      - name: schedule 
        type: string 
        jsonPath: .spec.cronSpec      
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                cronSpec:
                  type: string
                image:
                  type: string
                replicas:
                  type: integer
  # either Namespaced or Cluster
  scope: Namespaced
  names:
    # plural name to be used in the URL: /apis/<group>/<version>/<plural>
    plural: crontabs
    # singular name to be used as an alias on the CLI and for display
    singular: crontab
    # kind is normally the CamelCased singular type. Your resource manifests use this.
    kind: CronTab
    # shortNames allow shorter string to match your resource on the CLI
    shortNames:
    - ct

the code of types.go

package v1

import (
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

type CrontabSpecV1 struct {
	CronSpec string `json:"cronSpec"`
	Image    string `json:"image"`
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type Crontab struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec CrontabSpecV1 `json:"spec,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type CrontabList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`

	Items []Crontab `json:"items"`
}

and the code of register.go

// Define your schema name and the version
var SchemeGroupVersion = schema.GroupVersion{
	Group:   "stable.example.com",
	Version: "v1",
}

var (
	SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
	AddToScheme   = SchemeBuilder.AddToScheme
)

// Resource takes an unqualified resource and returns a Group qualified GroupResource
func Resource(resource string) schema.GroupResource {
	return SchemeGroupVersion.WithResource(resource).GroupResource()
}

func Kind(kind string) schema.GroupKind {
	return SchemeGroupVersion.WithKind(kind).GroupKind()
}

// Adds the list of known types to the given scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
	scheme.AddKnownTypes(SchemeGroupVersion,
		&CrontabList{},
		&Crontab{},
	)
	metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
	return nil
}

after the commandgenerate-groups.sh "deepcopy,client,informer,lister" be executed, following code raise an error

	client, e := versioned.NewForConfig(config)
	if e != nil {
		panic(e.Error())
	}
	crontab, err := client.StableV1().Crontabs("default").Get(context.TODO(), "cron-1", metav1.GetOptions{})

the error message was

no kind "CronTab" is registered for version "stable.example.com/v1" in scheme "pkg/crd/typedClient/client/clientset/versioned/scheme/register.go:15"

however the kind CronTab has already been registered in register.go, not sure why it happened. is there anyone who can help on that? thanks.

go.mod: unexpected module path "k8s.io/sample-controller"

What happened:

The Error about go module happened when I built cnat-controller by go build -o cnat-controller ..

What you expected to happen:

I can build cnat-controller successfully.

How to reproduce it (as minimally and precisely as possible):

$ go build -o cnat-controller .
go: finding github.com/programming-kubernetes/cnat/cnat-client-go/pkg/generated/listers/cnat/v1alpha1 latest
...
go: github.com/programming-kubernetes/cnat/[email protected]: parsing go.mod: unexpected module path "k8s.io/sample-controller"
go: error loading module requirements

Anything else we need to know?:

I can build successfully cnat-controller when I replaced go.mod module path from k8s.io/sample-controller to github.com/programming-kubernetes/cnat/cnat-client-go.

-module k8s.io/sample-controller
+module github.com/programming-kubernetes/cnat/cnat-client-go

Environment:

Mac OS X 10.14.5
go version go1.12.9 darwin/amd64

Thank you.