Giter VIP home page Giter VIP logo

pykube's Introduction

kelproject

image

image

Top-level repository for issues spanning multiple repositories.

Overall Kel documentation is available at <http://docs.kelproject.com>.

image

Kel is an open source Platform as a Service (PaaS) from Eldarion, Inc. that makes it easy to manage web application deployment and hosting through the entire lifecycle from development through testing to production. It adds components and tools on top of Kubernetes that help developers manage their application infrastructure. Kel builds on Eldarion's 7+ years experience running one of the leading Python and Django PaaSes.

For more information about Kel, see kelproject.com, follow us on Twitter @projectkel, and join our Slack team.

License

The code in this project is licensed under the Apache License, version 2.0.

Contributing

By making a contribution to this project, you are agreeing to the Developer Certificate of Origin v1.1.

Code of Conduct

In order to foster a kind, inclusive, and harassment-free community, the Kel Project follows the Contributor Covenant Code of Conduct.

Commercial Support

Commercial support for Kel is available through Eldarion, please contact [email protected].

pykube's People

Contributors

adambom avatar amnk avatar ashcrow avatar bretthoerner avatar brosner avatar cdrage avatar davidsiefert avatar frewsxcv avatar iliakur avatar jayme-github avatar jheiss avatar jonhosmer avatar kbrownlees avatar nebirhos avatar paralin avatar paultiplady avatar pcm32 avatar rajivm avatar reflection avatar shashank-te avatar spladug avatar sublimino avatar vgarcia-te avatar wbuchwalter avatar yuvipanda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pykube's Issues

ssl and token issue with from_service_account

Hi,

Found two issues with pykube.KubeConfig.from_service_account.

  1. Cannot turn SSL verification off.

There has to be a way to either specify the verify flag when calling the pykube api or set the verify to False in HttpClient.build_session() when no "certificate-authority" is found in self.config.cluster

  1. The token read in KubeConfig.from_service_account appends a newline at the end of the token making it an invalid header value.

This can be fixed with: token = fp.read().rstrip('\n')

Thanks,
Sudeep

List/Watch objects in all namespaces

I want to query objects in all namespaces (GET /api/v1/services for example). What is the correct/intended way to do so (as NamespacedAPIObject always sets DEFAULT_NAMESPACE on instantiation)?

I see at least the following two possible ways:

Manually overwrite namespace in ObjectManager instance:

serviceQuery = Service.objects(api)
serviceQuery.namespace = None # Query all namespaces
services = serviceQuery.all()

Create a "Non-namespaced" version of the object class:

class Service(APIObject):
    version = 'v1'
    endpoint = 'services'

    @property
    def namespace(self):
        if self.obj['metadata'].get('namespace'):
            return self.obj['metadata']['namespace']

services = Service.objects(api).all()

Accessing the API Server from a pod using service account causes an error

K8S Version: 1.2
Code:

api = HTTPClient(KubeConfig.from_service_account())
pods = Pod.objects(api).filter(namespace="gondor-system")
ready_pods = filter(operator.attrgetter("ready"), pods)

Error message:

SSLError: hostname '10.3.240.1' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local'

KeyError: 'client-certificate'

Hi,
I followed this https://pypi.python.org/pypi/pykube/ to query for all ready pods. I'm getting the KeyError: 'client-certificate' error.

>>> import operator
>>> import pykube
>>> api = pykube.HTTPClient(pykube.KubeConfig.from_file("/root/.kube/config"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/site-packages/pykube/http.py", line 33, in __init__
    self.session = self.build_session()
  File "/usr/local/lib/python3.5/site-packages/pykube/http.py", line 57, in build_session
    self.config.user["client-certificate"].filename(),
KeyError: 'client-certificate'

Im using Python 3.5

Can't use watch with request 2.4.3

I always get strange tracebacks like the following with requests==2.4.3 (debian jessie). Upgrading to jessie-backports version (2.8.1) fixed that for me.

2016-04-20 20:50:09,186 (urllib3.connectionpool._make_request) [DEBUG] "GET /api/v1/watch/nodes?resourceVersion=1152705 HTTP/1.1" 200 None
Traceback (most recent call last):
  File "/opt/path/watcher.py", line 111, in <module>
    main(args.output, args.context, args.targetPort)
  File "/opt/path/watcher.py", line 65, in main
    for event in wq.object_stream():
  File "/usr/local/lib/python2.7/dist-packages/pykube/query.py", line 135, in object_stream
    we = json.loads(line.decode("utf-8"))
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 3 - line 1 column 5 (char 2 - 4)

Release for 1.2 Kubernetes?

Now that we've got all the features from 1.2 implemented into pykube, is it possible to do a (even minor) release of pykube so we can pull the new version from pip?

Support for retrieving logs for pods

What would be needed to support log retrieval from pods like you do with kubectl logs mypod

I'm happy to start coding something with some directions.

Ability to use pykube from inside a container running in Kubernetes

If you get a shell in a container that is currently running inside a kubernetes cluster, and install kubectl, you can access the Kubernetes cluster from the inside. There is however no kubeconfig file used, yet still kubectl manages to contact the master node and create API objects. The only thing exposed on the environment I found was:

root@my-busybox-lc4bi:~# env | grep 'KUBERNETES'
KUBERNETES_PORT=tcp://10.100.0.1:443
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_HOST=10.100.0.1
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP_ADDR=10.100.0.1
KUBERNETES_PORT_443_TCP=tcp://10.100.0.1:443

so I presume that kubectl must be using something like this to contact the master, but I'm not sure. It would be great if pykube could be able to contact the Kubernetes master from inside a container running in the cluster in the same way as kubectl manages to do without a kubeconfig file. We need this functionality for our main use case with the Galaxy workflow environment. I would be happy to contribute to this with some guidance, as I don't see very clearly how to approach this.

Using pykube inside the cluster

Hello,

Is pykube prepared to be executed inside a pod and make calls to the Kubernetes API? I need to query it to know the state of a Job, and if it has not ended (no successful nor fail) I would like to wait until it happens.

Thanks so much!

Exec on Pod?

Hey,

Is there a way to run a command on a pod using exec? I can't see any way of doing it looking at the code.

Thanks!

Setting a namespace!

Am I doing something wrong? :( Been trying for a bit through the code (sorry, I'm still a Python n00b)

Just trying to set a namespace correctly.

from pykube.http import HTTPClient                                                                                                                                                                                                                             
from pykube.objects import Pod                                                                                                                                                                                                                                 


api = HTTPClient(KubeConfig.from_file("/home/wikus/.kube/config"))                                                                                                                                                                                             
api.namespace = "apples"                                                                                                                                                                                                                                       

tp = anymarkup.parse_file('hello-apache-pod.json')                                                                                                                                                                                                             
new = Pod(api, tp)                                                                                                                                                                                                                                             
new.create()

Allow ability to bypass cache

In some cases I'm polling the status of pods (due to terrible historical reasons I can not use WATCH). It would be awesome if pykube supported a parameter that'll force the query to actually happen and not hit the cache.

Thanks for the nice library!

Unable to update deployment

I tried to change image versions for containers of a deployment.
That breaks with errors like the following:

HTTPError: Deployment.extensions "deployment_name" is invalid: spec.template.spec.containers[1].name: Duplicate value: "container_name"

I think that jsonpatch creates a broken patch in that case and created jsonpatch/issue49 for that.

'get_by_name' does not work for Nodes

x = pykube.Node.objects(api)
x.get_by_name('ip-10-6-41-253.us-west-2.compute.internal')

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-37-199fc29ba186> in <module>()
----> 1 x.get_by_name('ip-10-6-41-253.us-west-2.compute.internal')

/usr/lib/python2.7/site-packages/pykube/query.pyc in get_by_name(self, name)
     55         if self.api_obj_class.version:
     56             kwargs["version"] = self.api_obj_class.version
---> 57         r = self.api.get(**kwargs)
     58         if not r.ok:
     59             if r.status_code == 404:

/usr/lib/python2.7/site-packages/pykube/http.pyc in get(self, *args, **kwargs)
    101            - `kwargs`: Keyword arguments
    102         """
--> 103         return self.session.get(*args, **self.get_kwargs(**kwargs))
    104 
    105     def options(self, *args, **kwargs):

/usr/lib/python2.7/site-packages/pykube/http.pyc in get_kwargs(self, **kwargs)
     68             url = url[1:]
     69         bits.append(url)
---> 70         kwargs["url"] = self.url + posixpath.join(*bits)
     71         return kwargs
     72 

/usr/lib64/python2.7/posixpath.pyc in join(a, *p)
     66     path = a
     67     for b in p:
---> 68         if b.startswith('/'):
     69             path = b
     70         elif path == '' or path.endswith('/'):

AttributeError: 'NoneType' object has no attribute 'startswith'

The problem is that namespace gets set to 'None' by the BaseQuery constructor, and then this 'None' value tries to get os.path.joined to the API:

ipdb> bits
['/api', 'v1', 'namespaces', None, 'nodes/ip-10-6-41-253.us-west-2.compute.internal']

pykube rpm / deb

Hey @brosner !

So I'd like to get this into debian / centos / fedora distros as a Python library so rpm / deb builds can use it.

Are there any plans in implementing it, if not, could we start to get the ball rolling? (I'll help!)

Syntax error

Looks like something checked in today broke this library.

from pykube import HTTPClient
File "/Users/chris/src/experimental/build/virtualenv/lib/python2.7/site-packages/pykube/init.py", line 8, in
from .objects import ( # noqa
File "/Users/chris/src/experimental/build/virtualenv/lib/python2.7/site-packages/pykube/objects.py", line 8, in
from .query import ObjectManager
File "/Users/chris/src/experimental/build/virtualenv/lib/python2.7/site-packages/pykube/query.py", line 115
def init(self, _args, resource_version=None, *_kwargs):
^
SyntaxError: invalid syntax

Documentation request - How To

Hello,

I ran across pykube as an integration for a project I'm working on as it seems to have the most recent contributions.
Have any initial documentation on how to use it by chance?

WatchQuery KeyError

The json objects returned by WatchQuery don't seem to match up with the keys expected in the codebase.

Example code:

for thing in pykube.Pod.objects(api).watch():
    print(thing)

Relevant tb:

  File "/.../lib/python3.5/site-packages/pykube/query.py", line 156, in object_stream
    yield WatchEvent(type=we["type"], object=self.api_obj_class(self.api, we["object"]))
KeyError: 'type'

RollingUpdater.update() method does not work

I want to upgrade my web app from v1 to v2, use RollingUpdater update() method, it does not work.
example:
`
def yaml_to_doc(filename):
with open(filename) as f:
doc = yaml.safe_load(f.read())
f.close()
return doc

api = pykube.HTTPClient(pykube.KubeConfig.from_file('/root/.kube/config'))
old_obj = yaml_to_doc('appname-controller-v1.yaml')
new_obj = yaml_to_doc('appname-controller-v2.yaml')
old_rc = pykube.ReplicationController(api, old_obj)
new_rc = pykube.ReplicationController(api, new_obj)
obj = rolling_updater.RollingUpdater(api, old_rc, new_rc)
obj.update()
`

while
kubectl rolling-update appname-v1 -f appname-controller-v2.yaml
can work

Improve API versioning

While implementing DaemonSet I came across the need to fix more at the HTTPClient level than I expected. I knew I needed to generate a completely new URL and HTTPClient has the ability to override all the parts, however, doing so per-request is difficult. This is largely do to the ORM bit.

The API objects should be the one telling the HTTPClient which version it should be working with. To do this correctly, I will need to break some of the API of HTTPClient.

This project is still early days so I am not too concerned about breaking stuff :-)

Unable to get a list of jobs in a namespace

I want to retrieve a list of jobs so I can see what jobs are current running. With kubectil, I can:

kubectl get jobs --namespace dev

However, when I try the following, it fails:

api = pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/Dave/.kube/config"))
jobs = pykube.Job.objects(api).filter(namespace="dev").get()

I get requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https:///apis/batch/v1/namespaces/dev/jobs

If I use:
kubectl get jobs --namespace=dev -v=8

Then I can see that kubectl is actually referencing /apis/extensions/v1beta1/namespaces/dev/jobs.

How can I get extensions/v1beta1 used instead of v1?

k8s 1.5 support

Small change happend in the k8s 1.5 release, now it returns None in case there is nothing to return and in 1.4 it was a empty array.

I did a fast ugly patch which fix this for me:

diff -ur pykube/pykube/query.py /home/vagrant/.local/lib/python2.7/site-packages/pykube/query.py
--- pykube/pykube/query.py      2016-12-13 12:03:11.924150049 +0000
+++ /home/vagrant/.local/lib/python2.7/site-packages/pykube/query.py    2016-12-13 12:03:45.035794481 +0000
@@ -105,17 +105,21 @@
         Execute the API request and return an iterator over the objects. This
         method does not use the query cache.
         """
-        for obj in self.execute().json()["items"]:
-            yield self.api_obj_class(self.api, obj)
+        if self.execute().json()["items"] is not None:
+            for obj in self.execute().json()["items"]:
+                yield self.api_obj_class(self.api, obj)

     @property
     def query_cache(self):
         if not hasattr(self, "_query_cache"):
             cache = {"objects": []}
             cache["response"] = self.execute().json()
-            for obj in cache["response"]["items"]:
-                cache["objects"].append(self.api_obj_class(self.api, obj))
-            self._query_cache = cache
+            if cache["response"]["items"] is not None:
+                for obj in cache["response"]["items"]:
+                    cache["objects"].append(self.api_obj_class(self.api, obj))
+                self._query_cache = cache
+            else:
+                self._query_cache = {"objects": {}}
         return self._query_cache

     def __len__(self):

Pls note - it's 0.13.0 diff.

Add new object types

We need "PersistentVolume", "PersistentVolumeClaim", and a few others, like "HorizontalScaler"

Deployment.ready may not accurately reflect deployment status

The check self.obj["status"]["updatedReplicas"] == self.replicas does not necessarily indicate the deployment is successful. It probably works ok if you are doing a create() but not if you are doing an update(). For instance if you have 2 replicas and update the deployment with a surge of 2 you will see an updatedReplicas of 2 but you will also see unavailableReplicas at 2. Following is an example of updating a deployment in this scenario. The True at the end of the line is the ready value.

2016-10-04 13:33:07,436 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 2, u'availableReplicas': 1, u'replicas': 3} 5 True
2016-10-04 13:33:09,535 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:11,632 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:13,730 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:15,824 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:17,922 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:20,017 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:22,115 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:24,212 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'availableReplicas': 2, u'replicas': 2} 5 True

I'm doing this right now but it is ugly. Would love for a cleaner check.

    start_time = datetime.datetime.now()
    while (datetime.datetime.now() - start_time).total_seconds() < timeout:
        try:
            deployment.reload()
            current_revision = deployment.annotations['deployment.kubernetes.io/revision']
            logging.info("%s %s", deployment.obj['status'], current_revision)

            if int(current_revision) < int(our_revision):
                logging.info("Waiting for our deployment to start")
                time.sleep(2)
                continue

            if 'availableReplicas' in deployment.obj['status'] and 'updatedReplicas' in deployment.obj['status'] and \
                deployment.obj['status']['updatedReplicas'] == deployment.obj['status']['availableReplicas'] and \
                deployment.obj['status']['replicas'] == deployment.obj['status']['availableReplicas']:

                #Final check, just in case
                if 'unavailableReplicas' in deployment.obj['status'] and \
                    deployment.obj['status']['unavailableReplicas'] != 0:
                    logging.info("Unavailable != 0")
                    time.sleep(2)
                    continue

                logging.info("Done waiting for deployment")
                if current_revision != our_revision:
                    logging.info("Looks like our deployment got bumped ours: %s current: %s", \
                                 our_revision, current_revision)
                return current_revision

        except Exception:  #pylint: disable=w0703
            logging.error("Error:", exc_info=True)

Error refreshing token: (invalid_scope) Bad Request

Just got the following error while creating an HTTPClient instance:

2016-10-21 09:03:10,344 (requests_oauthlib.oauth2_session.request) [DEBUG] Invoking 0 protected resource request hooks.
2016-10-21 09:03:10,344 (requests_oauthlib.oauth2_session.request) [DEBUG] Adding token {'access_token': 'secretToken', 'token_type': 'Bearer', 'expires_in': '3600', 'refresh_token': u'refreshToken'} to request.
2016-10-21 09:03:10,345 (requests_oauthlib.oauth2_session.request) [DEBUG] Requesting url https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=secretToken using method GET.
2016-10-21 09:03:10,345 (requests_oauthlib.oauth2_session.request) [DEBUG] Supplying headers {u'Authorization': u'Bearer secretToken'} and data None
2016-10-21 09:03:10,345 (requests_oauthlib.oauth2_session.request) [DEBUG] Passing through key word arguments {'allow_redirects': True}.
2016-10-21 09:03:10,351 (requests.packages.urllib3.connectionpool._new_conn) [INFO] Starting new HTTPS connection (1): www.googleapis.com
2016-10-21 09:03:10,602 (requests.packages.urllib3.connectionpool._make_request) [DEBUG] "GET /oauth2/v1/tokeninfo?access_token=secretToken HTTP/1.1" 400 None
2016-10-21 09:03:10,605 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Adding auto refresh key word arguments {}.
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Prepared refresh token request body grant_type=refresh_token&client_secret=clientSecret&client_id=clientId.apps.googleusercontent.com&refresh_token=refreshToken&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fplus.me
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.request) [DEBUG] Requesting url https://www.googleapis.com/oauth2/v4/token using method POST.
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.request) [DEBUG] Supplying headers {u'Content-Type': u'application/x-www-form-urlencoded;charset=UTF-8', u'Accept': u'application/json'} and data {u'client_secret': u'clientSecret', u'grant_type': u'refresh_token', u'refresh_token': u'refreshToken', u'client_id': u'clientId.apps.googleusercontent.com', u'scope': u'https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/plus.me'}
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.request) [DEBUG] Passing through key word arguments {'verify': True, 'json': None, 'proxies': None, 'auth': None, 'timeout': None}.
2016-10-21 09:03:10,818 (requests.packages.urllib3.connectionpool._make_request) [DEBUG] "POST /oauth2/v4/token HTTP/1.1" 400 None
2016-10-21 09:03:10,819 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Request to refresh token completed with status 400.
2016-10-21 09:03:10,819 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Response headers were {'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'Content-Encoding': 'gzip', 'Transfer-Encoding': 'chunked', 'Expires': 'Fri, 21 Oct 2016 07:03:10 GMT', 'Vary': 'Origin, X-Origin', 'Server': 'GSE', 'Cache-Control': 'private, max-age=0', 'Date': 'Fri, 21 Oct 2016 07:03:10 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Alt-Svc': 'quic=":443"; ma=2592000; v="36,35,34,33,32"', 'Content-Type': 'application/json; charset=UTF-8'} and content {
 "error": "invalid_scope",
 "error_description": "Bad Request",
 "error_uri": ""
}
2016-10-21 09:03:10,819 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Invoking 0 token response hooks.
Traceback (most recent call last):
  File ".pythonrc.py", line 29, in <module>
    api.session.verify = session_verify
  File "/home/jayme-github/code/pykube/pykube/http.py", line 41, in session
    self._session = build_session(self.config, self.gcloud_file)
  File "/home/jayme-github/code/pykube/pykube/session.py", line 23, in build_session
    s = _session_object("gcp", config, gcloud_file)
  File "/home/jayme-github/code/pykube/pykube/session.py", line 47, in _session_object
    return GCPSession(config, gcloud_file).create()
  File "/home/jayme-github/code/pykube/pykube/session.py", line 120, in create
    self._update_token()
  File "/home/jayme-github/code/pykube/pykube/session.py", line 131, in _update_token
    refresh_token=self.credentials.get('refresh_token'))
  File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/requests_oauthlib/oauth2_session.py", line 309, in refresh_token
    self.token = self._client.parse_request_body_response(r.text, scope=self.scope)
  File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 409, in parse_request_body_response
    self.token = parse_token_response(body, scope=scope)
  File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 376, in parse_token_response
    validate_token_parameters(params)
  File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 383, in validate_token_parameters
    raise_from_error(params.get('error'), params)
  File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 325, in raise_from_error
    raise cls(**kwargs)
oauthlib.oauth2.rfc6749.errors.InvalidScopeError: (invalid_scope) Bad Request

Everything is fine again after running one kubectl command (e.g. refreshing the token).

Setting current context explicitly still sends requests to the context set in kubeconfig file

Expected behaviour: When the current_context is set explictly using KubeConfig.set_current_context, and then further Kubernetes requests are made, Kubernetes requests are being made to new context that is set.

Actual behaviour: When the current_context is set explictly using KubeConfig.set_current_context, and then further Kubernetes requests are made, Kubernetes requests are being made to the context set in "kubeconfig" file.

Steps to reproduce (the current-context in my kube config file is test123):

config = pykube.KubeConfig.from_file("/home/Me/.kube/config")
api = pykube.HTTPClient(config)
config.set_current_context("test42")
pykube.ConfigMap.objects(api).get_by_name("my-config")

Executing the above tries to fetch the configmap from the "test123" context and not from the "test42" context.

GKE auth expires and pykube does not refresh it

This is another issue in the GKE authentication. I know the full end to end authentication is not yet done, and that GKE auth works if we already have the auth token in kube config (after this is merged #67)

But, having that token is not enough, kubectl is able to get a new token and update the 'expiry' field, but pykube, if the token is expired, can't authenticate.

I'd like to work on this but in the meantime, if someone knows how kubectl is dealing with this and share the info with me, would be helpful.

Is kubectl doing any extra request to GKE k8s api servers to fetch a new token? is kubectl using gcloud cli underneath?

Error in updating ReplicationController on Windows

I'm using pykube v0.13.0 on Windows, but I can't update ReplicationController.

>>> import pykube
>>> api = pykube.HTTPClient(pykube.KubeConfig.from_file("C:\\Users\\kazuyuki\\.kube\\config"))
>>> rc = pykube.ReplicationController.objects(api).get_by_name("myapp")
>>> rc.update()
Traceback (most recent call last):
  File "C:\Program Files\Python35\lib\site-packages\pykube\http.py", line 99, in raise_for_status
    resp.raise_for_status()
  File "C:\Program Files\Python35\lib\site-packages\requests\models.py", line 884, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://192.168.99.100:8443/api/v1/namespaces/default/replicationcontrollers%5Cmyapp

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python35\lib\site-packages\pykube\objects.py", line 90, in update
    self.api.raise_for_status(r)
  File "C:\Program Files\Python35\lib\site-packages\pykube\http.py", line 106, in raise_for_status
    raise HTTPError(payload["message"])
pykube.exceptions.HTTPError: the server could not find the requested resource
>>>

The url: https://192.168.99.100:8443/api/v1/namespaces/default/replicationcontrollers%5Cmyapp is invalid. This url is generated in objects.py,

54            kw["url"] = op.normpath(op.join(self.endpoint, self.name, operation))

It seems that this line can't work in Windows, where os.path uses \ (%5C in the url) instead of /.
I think this line should be

54            kw["url"] = "/".join([self.endpoint, self.name, operation])

Creating a deployment

This is most likely an issue with how I am using the module and not a bug itself.

I have a little test script that attempts to create a deployment similar to the rc creation under usage section in the README. Here is my test script.

import operator
import pykube

dep = {
  "apiVersion": "extensions/v1beta1",
  "kind": "Deployment",
  "metadata": {
    "name": "test1"
  },
  "spec": {
    "replicas": 3,
    "template": {
      "metadata": {
        "labels": {
          "app": "myApp"
        }
      },
      "spec": {
        "containers": [
          {
            "env": [
              {
                "name": "ROOT_URL",
                "value": "http://localhost:3000"
              },
              {
                "name": "MONGO_URL",
                "value": "mongodb://localhost:27017/app"
              },
              {
                "name": "PORT",
                "value": "3000"
              }
            ],
            "image": "myApp:latest",
            "name": "myApp",
            "ports": [
              {
                "containerPort": 3000
              }
            ]
          },
          {
            "image": "mongo",
            "name": "db",
            "ports": [
              {
                "containerPort": 27017
              }
            ]
          }
        ]
      }
    }
  }
}

pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/me/.kube/config"))
pykube.Deployment(dep).create()

running this with python3

python3 example.py

This gives the following error.

Traceback (most recent call last):
  File "example.py", line 114, in <module>
    pykube.Deployment(dep).create()
TypeError: __init__() missing 1 required positional argument: 'obj'

I used pip to install the module.

python3 -m pip install pykube

Python version.

Python 3.5.1

What am I doing wrong here? Thanks in advance!

Deleting deployment does not delete pods associated with it

We have the same issue with kubernetes/kubernetes#32985 .

It may be the bug of pykube and we have tested for serval times. If we delete the deployment, kubectl get deployment can not find that deployment, but kubectl get rs shows that the pods are not deleted.

If we use kubectl to delete the deployment, everything works well and the pods will switch to Terminating and deleted forever.

We have used pykube to implement the PaaS service and this is critical for us. Hope anyone can help to confirm and fix this ๐Ÿ˜ƒ

KeyError 'conditions'

I have noticed from time to time that I see this exception:

File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/rolling_updater.py", line 53, in update
    min_available, max_surge,
  File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/rolling_updater.py", line 83, in scale_down
    _, new_available = self.poll_for_ready_pods(old_rc, new_rc)
  File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/rolling_updater.py", line 126, in poll_for_ready_pods
    if pod.ready:
  File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/objects.py", line 158, in ready
    cs = self.obj["status"]["conditions"]
KeyError: 'conditions'

during a rolling update. Need to investigate why conditions is not present.

Cannot connect?

File "/home/drew/.local/lib/python2.7/site-packages/requests/adapters.py", line 477, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: hostname '52.38.116.180' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', 'kubernetes-master'

Can anyone help me with this error?

Thanks!
Drew

"tlsv1 alert protocol version" when running the example

This is the code I run in ipython with python3.5:

import operator
import pykube
api = pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/yuefengz/.kube/config"))
pods = pykube.Pod.objects(api).filter(namespace="default")
ready_pods = filter(operator.attrgetter("ready"), pods)

It throws exception:

---------------------------------------------------------------------------
SSLError                                  Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, **response_kw)
    594                                                   body=body, headers=headers,
--> 595                                                   chunked=chunked)
    596

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    351         try:
--> 352             self._validate_conn(conn)
    353         except (SocketTimeout, BaseSSLError) as e:

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    830         if not getattr(conn, 'sock', None):  # AppEngine might not have  `.sock`
--> 831             conn.connect()
    832

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connection.py in connect(self)
    288                                     server_hostname=hostname,
--> 289                                     ssl_version=resolved_ssl_version)
    290

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir)
    307     if HAS_SNI:  # Platform-specific: OpenSSL with enabled SNI
--> 308         return context.wrap_socket(sock, server_hostname=server_hostname)
    309

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname)
    376                          server_hostname=server_hostname,
--> 377                          _context=self)
    378

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in __init__(self, sock, keyfile, certfile, server_side, cert_reqs, ssl_version, ca_certs, do_handshake_on_connect, family, type, proto, fileno, suppress_ragged_eofs, npn_protocols, ciphers, server_hostname, _context)
    751                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 752                     self.do_handshake()
    753

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in do_handshake(self, block)
    987                 self.settimeout(None)
--> 988             self._sslobj.do_handshake()
    989         finally:

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in do_handshake(self)
    632         """Start the SSL/TLS handshake."""
--> 633         self._sslobj.do_handshake()
    634         if self.context.check_hostname:

SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)

During handling of the above exception, another exception occurred:

SSLError                                  Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    422                     retries=self.max_retries,
--> 423                     timeout=timeout
    424                 )

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, **response_kw)
    620             clean_exit = False
--> 621             raise SSLError(e)
    622

SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)

During handling of the above exception, another exception occurred:

SSLError                                  Traceback (most recent call last)
<ipython-input-4-93ae656b7489> in <module>()
      3 api = pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/yuefengz/.kube/config"))
      4 pods = pykube.Pod.objects(api).filter(namespace="default")
----> 5 ready_pods = filter(operator.attrgetter("ready"), pods)

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/query.py in __iter__(self)
    123
    124     def __iter__(self):
--> 125         return iter(self.query_cache["objects"])
    126
    127     @property

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/query.py in query_cache(self)
    113         if not hasattr(self, "_query_cache"):
    114             cache = {"objects": []}
--> 115             cache["response"] = self.execute().json()
    116             for obj in cache["response"]["items"]:
    117                 cache["objects"].append(self.api_obj_class(self.api, obj))

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/query.py in execute(self)
     97         if self.namespace is not None and self.namespace is not all_:
     98             kwargs["namespace"] = self.namespace
---> 99         r = self.api.get(**kwargs)
    100         r.raise_for_status()
    101         return r

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/http.py in get(self, *args, **kwargs)
    125            - `kwargs`: Keyword arguments
    126         """
--> 127         return self.session.get(*args, **self.get_kwargs(**kwargs))
    128
    129     def options(self, *args, **kwargs):

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py in get(self, url, **kwargs)
    486
    487         kwargs.setdefault('allow_redirects', True)
--> 488         return self.request('GET', url, **kwargs)
    489
    490     def options(self, url, **kwargs):

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    473         }
    474         send_kwargs.update(settings)
--> 475         resp = self.send(prep, **send_kwargs)
    476
    477         return resp

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py in send(self, request, **kwargs)
    594
    595         # Send the request
--> 596         r = adapter.send(request, **kwargs)
    597
    598         # Total elapsed time of the request (approximately)

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    495         except (_SSLError, _HTTPError) as e:
    496             if isinstance(e, _SSLError):
--> 497                 raise SSLError(e, request=request)
    498             elif isinstance(e, ReadTimeoutError):
    499                 raise ReadTimeout(e, request=request)

SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)

I was using python2.7 but switched to python3.5 because of issue 29.

GKE authentication

pykube is not capable of authenticating against GKE clusters. Recent clusters on GKE use Google OAuth to authenticate. The user configuration in ~/.kube/config looks like:

user:
  auth-provider:
    name: gcp

Once kubectl runs with that configuration it adds in the config key with an access-token and expiry.

In my initial research, I learned that oauth2client uses httplib which is a non-starter for integrating it with pykube. However, there's hope with googleapis/oauth2client#128

support for no-auth kube-apiserver access

I am accessing a kubernetes cluster with no auth required for convenience. The http client currently errors out when it cannot find the right credentials, when none are actually needed.

Traceback creating WatchEvent

It seems as if the response objects do not have a type :

wq = Node.objects(api).all().watch(now)
for e in wq.object_stream():
    print(e)
site-packages/pykube/query.py in object_stream(self)
    134             we = json.loads(line.decode("utf-8"))
--> 135             yield WatchEvent(type=we["type"], object=self.api_obj_class(self.api, we["object"]))
    136 
    137     def __iter__(self):

KeyError: 'type'

//EDIT:
My bad. Looks as watch() (at least for Nodes) must be created from a "namespace=all_" filter for the API-URL to be computed correctly:

wq = Node.objects(api).filter(namespace=all_).watch(now)
for e in wq.object_stream():
    print(e)

Maybe I just don't know how to use that correctly. ๐Ÿ˜„

Example code fails to run with error "ObjectManager instance has no attribute 'endpoint'"

>>> api = HTTPClient(KubeConfig.from_file("/Users/<username>/.kube/config"))
>>> pods = Pod.objects(api).filter(namespace="<a valid namespace>")

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/chris/src/experimental/build/krypton-virtualenv/lib/python2.7/site-packages/pykube/query.py", line 83, in __call__
    return Query(api, self.endpoint, self.api_obj_class, namespace=self.namespace)
AttributeError: ObjectManager instance has no attribute 'endpoint'

[Proposal] Generate API objects from swagger spec

After using this library for a while, I can see a few areas in which the flow for authoring specs could be improved. Here's a sketch of a design, to get a conversation going, and to see if you guys have done any work in this area.

The general idea I'd like to propose is to load object definitions from the Swagger spec files that the k8s API exposes, and use those object definitions to generate the set of classes that can be used to build specs.

A couple options:

  1. There are dynamic libraries like https://github.com/Yelp/bravado which load a Swagger spec into memory.
  2. Something like swagger-codegen could be used to statically compile a client, which would then give IDE autocompetion. (I favour this option).

Now you can write code like

my_pod = Pod(
  containers=[
    Container(name='first', image='alpine'),
  ],
  ..
)

With each of these objects having all fields defined by the swagger spec. Tab completion for your Kubernetes specs!

The other benefit of this approach is that you no longer need to add classes manually for each Kind, which implies less work to support a new major API version. There are some questions here though; would the static client definitions be checked in to the codebase (giving completions out of the box, without any additional build steps), or loaded by the library user so that they have the correct definitions for their cluster (this would be required to add dynamically added third-party-resources).

Either way it's simple to generate the client; you can run swagger-codegen generate -i http://kubernetes.io/kubernetes/swagger-spec/ -l python -o k8s-swagger to load the public swagger definition and generate a fresh client implementation. https://github.com/swagger-api/swagger-codegen#homebrew has more detailed instructions if you want to play further.

I'd be interested in your thoughts on this, before I go too much further in exploring the implementation.

Is there a way to skip ceritificate validation?

The error I am seeing is:

requests.exceptions.SSLError: hostname 'x.x.x.x' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local'

x.x.x.x is the server's ip

ssl.SSLError when connecting to 1.3 GKE cluster

I've just upgraded my GKE cluster to version 1.3, and I'm now getting an SSL error when trying to connect using pykube:

INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): <snip>
Traceback (most recent call last):
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 351, in _make_request
    self._validate_conn(conn)
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 814, in _validate_conn
    conn.connect()
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 289, in connect
    ssl_version=resolved_ssl_version)
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 308, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 365, in wrap_socket
    _context=self)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 583, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 810, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/adapters.py", line 403, in send
    timeout=timeout
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 604, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "deploy.py", line 97, in <module>
    main()
  File "deploy.py", line 90, in main
    deploy.create_namespace()
  File "deploy.py", line 29, in create_namespace
    pykube.Namespace(self.api, {'metadata': {'name': self.namespace}}).create()
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/pykube/objects.py", line 62, in create
    r = self.api.post(**self.api_kwargs(data=json.dumps(self.obj), collection=True))
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/pykube/http.py", line 153, in post
    return self.session.post(*args, **self.get_kwargs(**kwargs))
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/sessions.py", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/adapters.py", line 477, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600)

I can interact with the system using kubectl, so doesn't look like a cluster problem at first pass.

Have you tested pykube with 1.3 yet? Is there known to be work required to support the new version?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.