Giter VIP home page Giter VIP logo

pykorm's Introduction

pykorm - Python Kubernetes Object-relational mapping (ORM)

pykorm is a simple library that links your models to their kubernetes counterpart.

Each model and instance on your code is thus directly linked to your kubernetes cluster and modifications are thus reflected both ways.

Examples

Namespaced Custom Resource

Setup

First of all, you need to have Custom Resource Definitions on your cluster.
This README will use the following Namespaced resource. You can apply it on your cluster with kubectl.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: peaches.pykorm.infomaniak.com
spec:
  group: pykorm.infomaniak.com
  names:
    kind: Peach
    listKind: PeachList
    plural: peaches
    singular: peach
  scope: Namespaced
  versions:
  - name: v1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              variety:
                type: string
            required:
              - variety


    additionalPrinterColumns:
    - name: Variety
      type: string
      description: The variety of the peach
      jsonPath: .spec.variety

Class definition

In order to link a python class to a kubernetes CustomResourceDefinition, you need to inherit the class from pykorm's NamespacedModel or ClusterModel and annotate it with the kubernetes CRD information like so:

import pykorm

@pykorm.k8s_custom_object('pykorm.infomaniak.com', 'v1', 'peaches')
class Peach(pykorm.NamespacedModel):
    variety: str = pykorm.fields.Spec('variety')

Notice that a class inheriting from pykorm.NamespacedModel already has the name and namespace fields setup.

Create a CR

In order to create a kubernetes custom resource from python, you just have to instantiate the class and save it with Pykorm.save():

import pykorm
pk = pykorm.Pykorm()

cake_peach = Peach(namespace='default', name='cake-peach', variety='Frost')
pk.save(cake_peach)  # We save the resource

as you can see, the model is instantly ensured in kubernetes:

$ kubectl get peach -n default
NAME         VARIETY
cake-peach   Frost

List resources

Pykorm can also list resources from kubernetes

>>> all_peaches = Peach.query.all()
>>> for peach in all_peaches:
>>>  print(peach)
<Peach namespace=default, name=cake-peach, variety=Frost>

# Filter by namespace
>>> Peach.query.filter_by(namespace='default').filter_by(variety='Frost').all()

You can even filter resources by some criterion:

>>> Peach.query.filter_by(name='cake-peach').all()
[<Peach namespace=default, name=cake-peach, variety=Frost>]
>>> Peach.query.filter_by(namespace='kube-system').all()
[]

Delete resources

You can delete a resource with pykorm too:

pk.delete(peach)
$ kubectl get peach
No resources found in default namespace.

More examples

For more examples, don't hesitate to look into the examples/ directory

Is pykorm stable ?

pykorm is still very young and very naive. It's also missing quite a lot of features (relationships, etc.). It was originally created because a lot of boilerplate code was written each time a kubernetes custom object had to be interfaced with python code.

Work on pykorm is actually on the way. Don't hesitate to contribute to the project if you have the energy for it !

Equivalences

Python Kubernetes
Class CustomResourceDefinition
Instance CustomResource

pykorm's People

Contributors

anyisalin avatar dependabot[bot] avatar frankkkkk avatar gomes-infomaniak avatar hedinasr avatar pascal-dubois avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pykorm's Issues

Allow the modification of readonly fields

Read only fields were meant to be read only. However sometimes these fields must be set (in the case of an operator managing these resources for example).

We should add a mean to edit these fields with a context manager for example:

with pykorm.fields.UnlockReadOnly:
  obj.read_only = 42

Make pykorm faster on large number of objects

Pykorm is rather slow when dealing with a large number of objects. In my case, at ~850 objects, an .all() query takes around 20s !

>>> a = time.time() ; len(list(MyCRD.query.all())); print(time.time() - a)
851
21.512641668319702

in comparison, kubectl takes only 1s:

$ time kl get MyCRD.p6k.ch -A 
real	0m1.392s
user	0m0.536s
sys	0m0.182s

Add nested fields

AnyISalIn's MR #11 introduced nested fields. It wasn't merged in #13 because some tests didn't pass and the change would've been too big.

However the idea was really good and they should be inttroduced into pykorm

Provide pre-built models for all core k8s resource

I'm currently looking into using pykorm for a small project that I'm building with kopf. So far I've been using pykube but I'm not really happy with it's API.
pykorm looks very promising as it allows to properly map my CRD & core k8s resource to python objects.

One thing that are not clear to me or maybe might be missing are default models within pykorm for all the existing core resources.
Based on the example below i assume that i would need to build the models on my own?

@pykorm.pykorm.k8s_core(kind='Event')
class Event(pykorm.models.NamespacedModel):
action: str = pykorm.fields.DataField('action')
reason: str = pykorm.fields.DataField('reason')
type: str = pykorm.fields.DataField('type')
message: str = pykorm.fields.DataField('message')
@pykorm.pykorm.k8s_core(kind='Pod')
class Pod(pykorm.models.NamespacedModel):
_conditions: list = pykorm.fields.Status('conditions')
_containerStatuses: list = pykorm.fields.Status('containerStatuses', default=[])
_containers: list = pykorm.fields.Spec('containers', default=[])
_init_containers: list = pykorm.fields.Spec('initContainers', default=[])
_initContainerStatuses: list = pykorm.fields.Spec('initContainerStatuses', default=[])
phase: str = pykorm.fields.Status('phase')

If that is the case I'd propose to build these models & then create a PR to merge them into this lib

Support `status` subresource

In most cases the status field in the root is a subresource and can't be updated with the same api as spec or metadata. There are separate APIs like create_namespaced_pod_status, patch_namespaced_custom_object_status, etc.

I've worked around this by creating a mixin that I add to all model classes and provides a method that calls the relevant *_status variant of the APIs but it would be nice if this could be supported by pykorm directly. Either by calling both on save()/apply() if there is a Status field in the model or even a separate way (pk.status().save(obj), pk.save_status(obj), ...).

Add selection of kubeconfig to use

We should add the ability to choose a specific kubeconfig (when connecting to different cluster for example).

pk = pykorm.Pykorm(kubeconfig_file='foo/bar.yaml')

Dont send to k8s None fields

When creating aand saving a model with some empty (non mandatory) fields, it will fail because of the default value which will be None.

We should skip this attribute.

DictNestedField field wrongly inited

Models using DictNestedField fields are wrongly inited and can't be used for writing:

Example (from XXX):

class Score(pykorm.models.Nested):
    exterior: int = pykorm.fields.DataField('exterior')
    delicious: int = pykorm.fields.DataField('delicious', 10)


@pykorm.k8s_custom_object('pykorm.infomaniak.com', 'v1', 'peaches')
class Peach(pykorm.NamespacedModel):
    variety: str = pykorm.fields.Spec('variety', default='default-variety')
    price: str = pykorm.fields.Spec('price', default=1)
    colours: list = pykorm.fields.Spec('colours', default=[])
    score: Score = pykorm.fields.DictNestedField(Score, path=['spec', 'score'])

Results in the following error when writing to p.score.delicious:

>       p.score.exterior = 42
E       AttributeError: 'NoneType' object has no attribute 'exterior'

Field default attributes are not copied on object instantiation

CRDs fields can have default values:

@pykorm.k8s_custom_object('pykorm.infomaniak.com', 'v1', 'foo')
class Foo(ScoreMixin, pykorm.NamespacedModel):
    colours: list = pykorm.fields.Spec('colours', [])

However, the default value of the field is not copied on object instantiation. This results in bugs like these:

f1 = Foo(...)
f1.colours = ['foo']

f2 = Foo(...)
print(f2.colours)  # -> ['foo'] . Should return [] !

Document the code

The source code should be further documented in order to generate an autodoc

Add object relationships

We should add a Relationship field whose role is to reference another resource.

However, how to deal with references to other namespaces ? We could support:

  • ClusterResource → ClusterResource
  • NamespacedResource(ns n) → NamespacedResource(ns n)
  • NamespacedResource(ns n) → Cluster resource

filter_by: add namespace speedup

We can speedup the filter_by method on Namespaced objects when filtering by a specific namespace instead of crawling all the namespaces.

Add k8s_batchv1 et al

Pykorm supports CRDs, as well as core objects and apps/v1 objects. But many are missing (e.g. batch/v1 for example). We should add them too

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.