kelproject / pykube Goto Github PK
View Code? Open in Web Editor NEWPython client library for Kubernetes
License: Apache License 2.0
Python client library for Kubernetes
License: Apache License 2.0
Hey,
Is there a way to run a command on a pod using exec? I can't see any way of doing it looking at the code.
Thanks!
We need "PersistentVolume", "PersistentVolumeClaim", and a few others, like "HorizontalScaler"
I want to query objects in all namespaces (GET /api/v1/services
for example). What is the correct/intended way to do so (as NamespacedAPIObject
always sets DEFAULT_NAMESPACE
on instantiation)?
I see at least the following two possible ways:
Manually overwrite namespace in ObjectManager
instance:
serviceQuery = Service.objects(api)
serviceQuery.namespace = None # Query all namespaces
services = serviceQuery.all()
Create a "Non-namespaced" version of the object class:
class Service(APIObject):
version = 'v1'
endpoint = 'services'
@property
def namespace(self):
if self.obj['metadata'].get('namespace'):
return self.obj['metadata']['namespace']
services = Service.objects(api).all()
I tried to change image versions for containers of a deployment.
That breaks with errors like the following:
HTTPError: Deployment.extensions "deployment_name" is invalid: spec.template.spec.containers[1].name: Duplicate value: "container_name"
I think that jsonpatch creates a broken patch in that case and created jsonpatch/issue49 for that.
It seems as if the response objects do not have a type
:
wq = Node.objects(api).all().watch(now)
for e in wq.object_stream():
print(e)
site-packages/pykube/query.py in object_stream(self)
134 we = json.loads(line.decode("utf-8"))
--> 135 yield WatchEvent(type=we["type"], object=self.api_obj_class(self.api, we["object"]))
136
137 def __iter__(self):
KeyError: 'type'
//EDIT:
My bad. Looks as watch()
(at least for Nodes) must be created from a "namespace=all_" filter for the API-URL to be computed correctly:
wq = Node.objects(api).filter(namespace=all_).watch(now)
for e in wq.object_stream():
print(e)
Maybe I just don't know how to use that correctly. ๐
K8S Version: 1.2
Code:
api = HTTPClient(KubeConfig.from_service_account())
pods = Pod.objects(api).filter(namespace="gondor-system")
ready_pods = filter(operator.attrgetter("ready"), pods)
Error message:
SSLError: hostname '10.3.240.1' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local'
This is the code I run in ipython with python3.5:
import operator
import pykube
api = pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/yuefengz/.kube/config"))
pods = pykube.Pod.objects(api).filter(namespace="default")
ready_pods = filter(operator.attrgetter("ready"), pods)
It throws exception:
---------------------------------------------------------------------------
SSLError Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, **response_kw)
594 body=body, headers=headers,
--> 595 chunked=chunked)
596
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
351 try:
--> 352 self._validate_conn(conn)
353 except (SocketTimeout, BaseSSLError) as e:
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in _validate_conn(self, conn)
830 if not getattr(conn, 'sock', None): # AppEngine might not have `.sock`
--> 831 conn.connect()
832
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connection.py in connect(self)
288 server_hostname=hostname,
--> 289 ssl_version=resolved_ssl_version)
290
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir)
307 if HAS_SNI: # Platform-specific: OpenSSL with enabled SNI
--> 308 return context.wrap_socket(sock, server_hostname=server_hostname)
309
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname)
376 server_hostname=server_hostname,
--> 377 _context=self)
378
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in __init__(self, sock, keyfile, certfile, server_side, cert_reqs, ssl_version, ca_certs, do_handshake_on_connect, family, type, proto, fileno, suppress_ragged_eofs, npn_protocols, ciphers, server_hostname, _context)
751 raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 752 self.do_handshake()
753
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in do_handshake(self, block)
987 self.settimeout(None)
--> 988 self._sslobj.do_handshake()
989 finally:
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py in do_handshake(self)
632 """Start the SSL/TLS handshake."""
--> 633 self._sslobj.do_handshake()
634 if self.context.check_hostname:
SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)
During handling of the above exception, another exception occurred:
SSLError Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
422 retries=self.max_retries,
--> 423 timeout=timeout
424 )
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, **response_kw)
620 clean_exit = False
--> 621 raise SSLError(e)
622
SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)
During handling of the above exception, another exception occurred:
SSLError Traceback (most recent call last)
<ipython-input-4-93ae656b7489> in <module>()
3 api = pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/yuefengz/.kube/config"))
4 pods = pykube.Pod.objects(api).filter(namespace="default")
----> 5 ready_pods = filter(operator.attrgetter("ready"), pods)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/query.py in __iter__(self)
123
124 def __iter__(self):
--> 125 return iter(self.query_cache["objects"])
126
127 @property
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/query.py in query_cache(self)
113 if not hasattr(self, "_query_cache"):
114 cache = {"objects": []}
--> 115 cache["response"] = self.execute().json()
116 for obj in cache["response"]["items"]:
117 cache["objects"].append(self.api_obj_class(self.api, obj))
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/query.py in execute(self)
97 if self.namespace is not None and self.namespace is not all_:
98 kwargs["namespace"] = self.namespace
---> 99 r = self.api.get(**kwargs)
100 r.raise_for_status()
101 return r
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pykube/http.py in get(self, *args, **kwargs)
125 - `kwargs`: Keyword arguments
126 """
--> 127 return self.session.get(*args, **self.get_kwargs(**kwargs))
128
129 def options(self, *args, **kwargs):
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py in get(self, url, **kwargs)
486
487 kwargs.setdefault('allow_redirects', True)
--> 488 return self.request('GET', url, **kwargs)
489
490 def options(self, url, **kwargs):
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
473 }
474 send_kwargs.update(settings)
--> 475 resp = self.send(prep, **send_kwargs)
476
477 return resp
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py in send(self, request, **kwargs)
594
595 # Send the request
--> 596 r = adapter.send(request, **kwargs)
597
598 # Total elapsed time of the request (approximately)
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
495 except (_SSLError, _HTTPError) as e:
496 if isinstance(e, _SSLError):
--> 497 raise SSLError(e, request=request)
498 elif isinstance(e, ReadTimeoutError):
499 raise ReadTimeout(e, request=request)
SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)
I was using python2.7 but switched to python3.5 because of issue 29.
Small change happend in the k8s 1.5 release, now it returns None in case there is nothing to return and in 1.4 it was a empty array.
I did a fast ugly patch which fix this for me:
diff -ur pykube/pykube/query.py /home/vagrant/.local/lib/python2.7/site-packages/pykube/query.py
--- pykube/pykube/query.py 2016-12-13 12:03:11.924150049 +0000
+++ /home/vagrant/.local/lib/python2.7/site-packages/pykube/query.py 2016-12-13 12:03:45.035794481 +0000
@@ -105,17 +105,21 @@
Execute the API request and return an iterator over the objects. This
method does not use the query cache.
"""
- for obj in self.execute().json()["items"]:
- yield self.api_obj_class(self.api, obj)
+ if self.execute().json()["items"] is not None:
+ for obj in self.execute().json()["items"]:
+ yield self.api_obj_class(self.api, obj)
@property
def query_cache(self):
if not hasattr(self, "_query_cache"):
cache = {"objects": []}
cache["response"] = self.execute().json()
- for obj in cache["response"]["items"]:
- cache["objects"].append(self.api_obj_class(self.api, obj))
- self._query_cache = cache
+ if cache["response"]["items"] is not None:
+ for obj in cache["response"]["items"]:
+ cache["objects"].append(self.api_obj_class(self.api, obj))
+ self._query_cache = cache
+ else:
+ self._query_cache = {"objects": {}}
return self._query_cache
def __len__(self):
Pls note - it's 0.13.0 diff.
I've just upgraded my GKE cluster to version 1.3, and I'm now getting an SSL error when trying to connect using pykube:
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): <snip>
Traceback (most recent call last):
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
chunked=chunked)
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 351, in _make_request
self._validate_conn(conn)
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 814, in _validate_conn
conn.connect()
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 289, in connect
ssl_version=resolved_ssl_version)
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 308, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 365, in wrap_socket
_context=self)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 583, in __init__
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 810, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/adapters.py", line 403, in send
timeout=timeout
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 604, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "deploy.py", line 97, in <module>
main()
File "deploy.py", line 90, in main
deploy.create_namespace()
File "deploy.py", line 29, in create_namespace
pykube.Namespace(self.api, {'metadata': {'name': self.namespace}}).create()
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/pykube/objects.py", line 62, in create
r = self.api.post(**self.api_kwargs(data=json.dumps(self.obj), collection=True))
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/pykube/http.py", line 153, in post
return self.session.post(*args, **self.get_kwargs(**kwargs))
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/sessions.py", line 518, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/sessions.py", line 585, in send
r = adapter.send(request, **kwargs)
File "/Users/paul/.virtualenvs/deploy/lib/python3.4/site-packages/requests/adapters.py", line 477, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600)
I can interact with the system using kubectl
, so doesn't look like a cluster problem at first pass.
Have you tested pykube with 1.3 yet? Is there known to be work required to support the new version?
x = pykube.Node.objects(api)
x.get_by_name('ip-10-6-41-253.us-west-2.compute.internal')
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-37-199fc29ba186> in <module>()
----> 1 x.get_by_name('ip-10-6-41-253.us-west-2.compute.internal')
/usr/lib/python2.7/site-packages/pykube/query.pyc in get_by_name(self, name)
55 if self.api_obj_class.version:
56 kwargs["version"] = self.api_obj_class.version
---> 57 r = self.api.get(**kwargs)
58 if not r.ok:
59 if r.status_code == 404:
/usr/lib/python2.7/site-packages/pykube/http.pyc in get(self, *args, **kwargs)
101 - `kwargs`: Keyword arguments
102 """
--> 103 return self.session.get(*args, **self.get_kwargs(**kwargs))
104
105 def options(self, *args, **kwargs):
/usr/lib/python2.7/site-packages/pykube/http.pyc in get_kwargs(self, **kwargs)
68 url = url[1:]
69 bits.append(url)
---> 70 kwargs["url"] = self.url + posixpath.join(*bits)
71 return kwargs
72
/usr/lib64/python2.7/posixpath.pyc in join(a, *p)
66 path = a
67 for b in p:
---> 68 if b.startswith('/'):
69 path = b
70 elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'startswith'
The problem is that namespace
gets set to 'None' by the BaseQuery
constructor, and then this 'None' value tries to get os.path.join
ed to the API:
ipdb> bits
['/api', 'v1', 'namespaces', None, 'nodes/ip-10-6-41-253.us-west-2.compute.internal']
pykube is not capable of authenticating against GKE clusters. Recent clusters on GKE use Google OAuth to authenticate. The user configuration in ~/.kube/config
looks like:
user:
auth-provider:
name: gcp
Once kubectl
runs with that configuration it adds in the config
key with an access-token
and expiry
.
In my initial research, I learned that oauth2client uses httplib which is a non-starter for integrating it with pykube. However, there's hope with googleapis/oauth2client#128
If you get a shell in a container that is currently running inside a kubernetes cluster, and install kubectl, you can access the Kubernetes cluster from the inside. There is however no kubeconfig file used, yet still kubectl manages to contact the master node and create API objects. The only thing exposed on the environment I found was:
root@my-busybox-lc4bi:~# env | grep 'KUBERNETES'
KUBERNETES_PORT=tcp://10.100.0.1:443
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_HOST=10.100.0.1
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP_ADDR=10.100.0.1
KUBERNETES_PORT_443_TCP=tcp://10.100.0.1:443
so I presume that kubectl must be using something like this to contact the master, but I'm not sure. It would be great if pykube could be able to contact the Kubernetes master from inside a container running in the cluster in the same way as kubectl manages to do without a kubeconfig file. We need this functionality for our main use case with the Galaxy workflow environment. I would be happy to contribute to this with some guidance, as I don't see very clearly how to approach this.
I am accessing a kubernetes cluster with no auth required for convenience. The http client currently errors out when it cannot find the right credentials, when none are actually needed.
Now that we've got all the features from 1.2 implemented into pykube
, is it possible to do a (even minor) release of pykube so we can pull the new version from pip?
hi,
is there anyway to set the gracePeriodSeconds when deleting pods?
Looking at the code it seems not?
Am I doing something wrong? :( Been trying for a bit through the code (sorry, I'm still a Python n00b)
Just trying to set a namespace correctly.
from pykube.http import HTTPClient
from pykube.objects import Pod
api = HTTPClient(KubeConfig.from_file("/home/wikus/.kube/config"))
api.namespace = "apples"
tp = anymarkup.parse_file('hello-apache-pod.json')
new = Pod(api, tp)
new.create()
After using this library for a while, I can see a few areas in which the flow for authoring specs could be improved. Here's a sketch of a design, to get a conversation going, and to see if you guys have done any work in this area.
The general idea I'd like to propose is to load object definitions from the Swagger spec files that the k8s API exposes, and use those object definitions to generate the set of classes that can be used to build specs.
A couple options:
Now you can write code like
my_pod = Pod(
containers=[
Container(name='first', image='alpine'),
],
..
)
With each of these objects having all fields defined by the swagger spec. Tab completion for your Kubernetes specs!
The other benefit of this approach is that you no longer need to add classes manually for each Kind, which implies less work to support a new major API version. There are some questions here though; would the static client definitions be checked in to the codebase (giving completions out of the box, without any additional build steps), or loaded by the library user so that they have the correct definitions for their cluster (this would be required to add dynamically added third-party-resources).
Either way it's simple to generate the client; you can run swagger-codegen generate -i http://kubernetes.io/kubernetes/swagger-spec/ -l python -o k8s-swagger
to load the public swagger definition and generate a fresh client implementation. https://github.com/swagger-api/swagger-codegen#homebrew has more detailed instructions if you want to play further.
I'd be interested in your thoughts on this, before I go too much further in exploring the implementation.
Expected behaviour: When the current_context
is set explictly using KubeConfig.set_current_context
, and then further Kubernetes requests are made, Kubernetes requests are being made to new context that is set.
Actual behaviour: When the current_context
is set explictly using KubeConfig.set_current_context
, and then further Kubernetes requests are made, Kubernetes requests are being made to the context set in "kubeconfig" file.
Steps to reproduce (the current-context in my kube config file is test123):
config = pykube.KubeConfig.from_file("/home/Me/.kube/config")
api = pykube.HTTPClient(config)
config.set_current_context("test42")
pykube.ConfigMap.objects(api).get_by_name("my-config")
Executing the above tries to fetch the configmap from the "test123" context and not from the "test42" context.
I have noticed from time to time that I see this exception:
File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/rolling_updater.py", line 53, in update
min_available, max_surge,
File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/rolling_updater.py", line 83, in scale_down
_, new_available = self.poll_for_ready_pods(old_rc, new_rc)
File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/rolling_updater.py", line 126, in poll_for_ready_pods
if pod.ready:
File "/Users/brian/Development/Envs/tmp-98df323c389668d/lib/python3.5/site-packages/pykube/objects.py", line 158, in ready
cs = self.obj["status"]["conditions"]
KeyError: 'conditions'
during a rolling update. Need to investigate why conditions
is not present.
The error I am seeing is:
requests.exceptions.SSLError: hostname 'x.x.x.x' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local'
x.x.x.x is the server's ip
Hello,
I ran across pykube as an integration for a project I'm working on as it seems to have the most recent contributions.
Have any initial documentation on how to use it by chance?
Looks like something checked in today broke this library.
from pykube import HTTPClient
File "/Users/chris/src/experimental/build/virtualenv/lib/python2.7/site-packages/pykube/init.py", line 8, in
from .objects import ( # noqa
File "/Users/chris/src/experimental/build/virtualenv/lib/python2.7/site-packages/pykube/objects.py", line 8, in
from .query import ObjectManager
File "/Users/chris/src/experimental/build/virtualenv/lib/python2.7/site-packages/pykube/query.py", line 115
def init(self, _args, resource_version=None, *_kwargs):
^
SyntaxError: invalid syntax
File "/home/drew/.local/lib/python2.7/site-packages/requests/adapters.py", line 477, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: hostname '52.38.116.180' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', 'kubernetes-master'
Can anyone help me with this error?
Thanks!
Drew
I want to retrieve a list of jobs so I can see what jobs are current running. With kubectil, I can:
kubectl get jobs --namespace dev
However, when I try the following, it fails:
api = pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/Dave/.kube/config"))
jobs = pykube.Job.objects(api).filter(namespace="dev").get()
I get requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https:///apis/batch/v1/namespaces/dev/jobs
If I use:
kubectl get jobs --namespace=dev -v=8
Then I can see that kubectl is actually referencing /apis/extensions/v1beta1/namespaces/dev/jobs.
How can I get extensions/v1beta1 used instead of v1?
In order to automatically delete (for example) pods created by a deployment, deleteOptions.orphanDependents
must be set to false
. I see no way of doing that in pykube at the moment?
See also http://kubernetes.io/docs/user-guide/garbage-collection/
I'm using pykube v0.13.0 on Windows, but I can't update ReplicationController.
>>> import pykube
>>> api = pykube.HTTPClient(pykube.KubeConfig.from_file("C:\\Users\\kazuyuki\\.kube\\config"))
>>> rc = pykube.ReplicationController.objects(api).get_by_name("myapp")
>>> rc.update()
Traceback (most recent call last):
File "C:\Program Files\Python35\lib\site-packages\pykube\http.py", line 99, in raise_for_status
resp.raise_for_status()
File "C:\Program Files\Python35\lib\site-packages\requests\models.py", line 884, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://192.168.99.100:8443/api/v1/namespaces/default/replicationcontrollers%5Cmyapp
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files\Python35\lib\site-packages\pykube\objects.py", line 90, in update
self.api.raise_for_status(r)
File "C:\Program Files\Python35\lib\site-packages\pykube\http.py", line 106, in raise_for_status
raise HTTPError(payload["message"])
pykube.exceptions.HTTPError: the server could not find the requested resource
>>>
The url: https://192.168.99.100:8443/api/v1/namespaces/default/replicationcontrollers%5Cmyapp
is invalid. This url is generated in objects.py
,
54 kw["url"] = op.normpath(op.join(self.endpoint, self.name, operation))
It seems that this line can't work in Windows, where os.path
uses \
(%5C
in the url) instead of /
.
I think this line should be
54 kw["url"] = "/".join([self.endpoint, self.name, operation])
The json objects returned by WatchQuery don't seem to match up with the keys expected in the codebase.
Example code:
for thing in pykube.Pod.objects(api).watch():
print(thing)
Relevant tb:
File "/.../lib/python3.5/site-packages/pykube/query.py", line 156, in object_stream
yield WatchEvent(type=we["type"], object=self.api_obj_class(self.api, we["object"]))
KeyError: 'type'
Hello,
Is pykube
prepared to be executed inside a pod and make calls to the Kubernetes API? I need to query it to know the state of a Job, and if it has not ended (no successful nor fail) I would like to wait until it happens.
Thanks so much!
While implementing DaemonSet
I came across the need to fix more at the HTTPClient
level than I expected. I knew I needed to generate a completely new URL and HTTPClient
has the ability to override all the parts, however, doing so per-request is difficult. This is largely do to the ORM bit.
The API objects should be the one telling the HTTPClient
which version it should be working with. To do this correctly, I will need to break some of the API of HTTPClient
.
This project is still early days so I am not too concerned about breaking stuff :-)
The check self.obj["status"]["updatedReplicas"] == self.replicas
does not necessarily indicate the deployment is successful. It probably works ok if you are doing a create()
but not if you are doing an update()
. For instance if you have 2 replicas and update the deployment with a surge of 2 you will see an updatedReplicas
of 2 but you will also see unavailableReplicas
at 2. Following is an example of updating a deployment in this scenario. The True
at the end of the line is the ready
value.
2016-10-04 13:33:07,436 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 2, u'availableReplicas': 1, u'replicas': 3} 5 True
2016-10-04 13:33:09,535 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:11,632 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:13,730 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:15,824 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:17,922 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:20,017 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:22,115 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'unavailableReplicas': 1, u'availableReplicas': 1, u'replicas': 2} 5 True
2016-10-04 13:33:24,212 - root - INFO - {u'observedGeneration': 15, u'updatedReplicas': 2, u'availableReplicas': 2, u'replicas': 2} 5 True
I'm doing this right now but it is ugly. Would love for a cleaner check.
start_time = datetime.datetime.now()
while (datetime.datetime.now() - start_time).total_seconds() < timeout:
try:
deployment.reload()
current_revision = deployment.annotations['deployment.kubernetes.io/revision']
logging.info("%s %s", deployment.obj['status'], current_revision)
if int(current_revision) < int(our_revision):
logging.info("Waiting for our deployment to start")
time.sleep(2)
continue
if 'availableReplicas' in deployment.obj['status'] and 'updatedReplicas' in deployment.obj['status'] and \
deployment.obj['status']['updatedReplicas'] == deployment.obj['status']['availableReplicas'] and \
deployment.obj['status']['replicas'] == deployment.obj['status']['availableReplicas']:
#Final check, just in case
if 'unavailableReplicas' in deployment.obj['status'] and \
deployment.obj['status']['unavailableReplicas'] != 0:
logging.info("Unavailable != 0")
time.sleep(2)
continue
logging.info("Done waiting for deployment")
if current_revision != our_revision:
logging.info("Looks like our deployment got bumped ours: %s current: %s", \
our_revision, current_revision)
return current_revision
except Exception: #pylint: disable=w0703
logging.error("Error:", exc_info=True)
I want to upgrade my web app from v1 to v2, use RollingUpdater update() method, it does not work.
example:
`
def yaml_to_doc(filename):
with open(filename) as f:
doc = yaml.safe_load(f.read())
f.close()
return doc
api = pykube.HTTPClient(pykube.KubeConfig.from_file('/root/.kube/config'))
old_obj = yaml_to_doc('appname-controller-v1.yaml')
new_obj = yaml_to_doc('appname-controller-v2.yaml')
old_rc = pykube.ReplicationController(api, old_obj)
new_rc = pykube.ReplicationController(api, new_obj)
obj = rolling_updater.RollingUpdater(api, old_rc, new_rc)
obj.update()
`
while
kubectl rolling-update appname-v1 -f appname-controller-v2.yaml
can work
What would be needed to support log retrieval from pods like you do with kubectl logs mypod
I'm happy to start coding something with some directions.
watch = pykube.Job.objects(
api,
namespace="gondor-system")
.filter(field_selector={"metadata.name":"my-job"})
.watch()
In some cases I'm polling the status of pods (due to terrible historical reasons I can not use WATCH). It would be awesome if pykube supported a parameter that'll force the query to actually happen and not hit the cache.
Thanks for the nice library!
The certificate problem is currently causing us issues
Hi,
I followed this https://pypi.python.org/pypi/pykube/ to query for all ready pods. I'm getting the KeyError: 'client-certificate' error.
>>> import operator
>>> import pykube
>>> api = pykube.HTTPClient(pykube.KubeConfig.from_file("/root/.kube/config"))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/site-packages/pykube/http.py", line 33, in __init__
self.session = self.build_session()
File "/usr/local/lib/python3.5/site-packages/pykube/http.py", line 57, in build_session
self.config.user["client-certificate"].filename(),
KeyError: 'client-certificate'
Im using Python 3.5
Hi,
Is there a preferred method to access key/values in the dict object for API class object?
Now we are using PetSet
to deploy our cluster applications. We accept request in Django
and it would be better to access Kubernetes
with Python SDK like pykube
.
After reading https://github.com/kelproject/pykube/blob/99e193a81f59044e1d6f3a3b96e6e33853059749/pykube/__init__.py , I thinks pykube
doesn't support PetSet
yet. Is it possible to support PetSet
?
We're also glad to test it.
It would make sense to have available the status of a Job object since it is something which is run and it is expected to be completed at some point in time.
https://github.com/kelproject/pykube/blob/master/pykube/objects.py#L182-L195
Hey @brosner !
So I'd like to get this into debian / centos / fedora distros as a Python library so rpm / deb builds can use it.
Are there any plans in implementing it, if not, could we start to get the ball rolling? (I'll help!)
Just got the following error while creating an HTTPClient
instance:
2016-10-21 09:03:10,344 (requests_oauthlib.oauth2_session.request) [DEBUG] Invoking 0 protected resource request hooks.
2016-10-21 09:03:10,344 (requests_oauthlib.oauth2_session.request) [DEBUG] Adding token {'access_token': 'secretToken', 'token_type': 'Bearer', 'expires_in': '3600', 'refresh_token': u'refreshToken'} to request.
2016-10-21 09:03:10,345 (requests_oauthlib.oauth2_session.request) [DEBUG] Requesting url https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=secretToken using method GET.
2016-10-21 09:03:10,345 (requests_oauthlib.oauth2_session.request) [DEBUG] Supplying headers {u'Authorization': u'Bearer secretToken'} and data None
2016-10-21 09:03:10,345 (requests_oauthlib.oauth2_session.request) [DEBUG] Passing through key word arguments {'allow_redirects': True}.
2016-10-21 09:03:10,351 (requests.packages.urllib3.connectionpool._new_conn) [INFO] Starting new HTTPS connection (1): www.googleapis.com
2016-10-21 09:03:10,602 (requests.packages.urllib3.connectionpool._make_request) [DEBUG] "GET /oauth2/v1/tokeninfo?access_token=secretToken HTTP/1.1" 400 None
2016-10-21 09:03:10,605 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Adding auto refresh key word arguments {}.
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Prepared refresh token request body grant_type=refresh_token&client_secret=clientSecret&client_id=clientId.apps.googleusercontent.com&refresh_token=refreshToken&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fplus.me
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.request) [DEBUG] Requesting url https://www.googleapis.com/oauth2/v4/token using method POST.
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.request) [DEBUG] Supplying headers {u'Content-Type': u'application/x-www-form-urlencoded;charset=UTF-8', u'Accept': u'application/json'} and data {u'client_secret': u'clientSecret', u'grant_type': u'refresh_token', u'refresh_token': u'refreshToken', u'client_id': u'clientId.apps.googleusercontent.com', u'scope': u'https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/plus.me'}
2016-10-21 09:03:10,606 (requests_oauthlib.oauth2_session.request) [DEBUG] Passing through key word arguments {'verify': True, 'json': None, 'proxies': None, 'auth': None, 'timeout': None}.
2016-10-21 09:03:10,818 (requests.packages.urllib3.connectionpool._make_request) [DEBUG] "POST /oauth2/v4/token HTTP/1.1" 400 None
2016-10-21 09:03:10,819 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Request to refresh token completed with status 400.
2016-10-21 09:03:10,819 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Response headers were {'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'Content-Encoding': 'gzip', 'Transfer-Encoding': 'chunked', 'Expires': 'Fri, 21 Oct 2016 07:03:10 GMT', 'Vary': 'Origin, X-Origin', 'Server': 'GSE', 'Cache-Control': 'private, max-age=0', 'Date': 'Fri, 21 Oct 2016 07:03:10 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Alt-Svc': 'quic=":443"; ma=2592000; v="36,35,34,33,32"', 'Content-Type': 'application/json; charset=UTF-8'} and content {
"error": "invalid_scope",
"error_description": "Bad Request",
"error_uri": ""
}
2016-10-21 09:03:10,819 (requests_oauthlib.oauth2_session.refresh_token) [DEBUG] Invoking 0 token response hooks.
Traceback (most recent call last):
File ".pythonrc.py", line 29, in <module>
api.session.verify = session_verify
File "/home/jayme-github/code/pykube/pykube/http.py", line 41, in session
self._session = build_session(self.config, self.gcloud_file)
File "/home/jayme-github/code/pykube/pykube/session.py", line 23, in build_session
s = _session_object("gcp", config, gcloud_file)
File "/home/jayme-github/code/pykube/pykube/session.py", line 47, in _session_object
return GCPSession(config, gcloud_file).create()
File "/home/jayme-github/code/pykube/pykube/session.py", line 120, in create
self._update_token()
File "/home/jayme-github/code/pykube/pykube/session.py", line 131, in _update_token
refresh_token=self.credentials.get('refresh_token'))
File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/requests_oauthlib/oauth2_session.py", line 309, in refresh_token
self.token = self._client.parse_request_body_response(r.text, scope=self.scope)
File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 409, in parse_request_body_response
self.token = parse_token_response(body, scope=scope)
File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 376, in parse_token_response
validate_token_parameters(params)
File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 383, in validate_token_parameters
raise_from_error(params.get('error'), params)
File "/home/jayme-github/.virtualenvs/pykube/local/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 325, in raise_from_error
raise cls(**kwargs)
oauthlib.oauth2.rfc6749.errors.InvalidScopeError: (invalid_scope) Bad Request
Everything is fine again after running one kubectl
command (e.g. refreshing the token).
See:
Edit: nvm, deployment is included
I always get strange tracebacks like the following with requests==2.4.3 (debian jessie). Upgrading to jessie-backports version (2.8.1) fixed that for me.
2016-04-20 20:50:09,186 (urllib3.connectionpool._make_request) [DEBUG] "GET /api/v1/watch/nodes?resourceVersion=1152705 HTTP/1.1" 200 None
Traceback (most recent call last):
File "/opt/path/watcher.py", line 111, in <module>
main(args.output, args.context, args.targetPort)
File "/opt/path/watcher.py", line 65, in main
for event in wq.object_stream():
File "/usr/local/lib/python2.7/dist-packages/pykube/query.py", line 135, in object_stream
we = json.loads(line.decode("utf-8"))
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 369, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 3 - line 1 column 5 (char 2 - 4)
[Errno 13] Permission denied: '/var/run/secrets/kubernetes.io/serviceaccount/token'
Hi,
Found two issues with pykube.KubeConfig.from_service_account.
There has to be a way to either specify the verify flag when calling the pykube api or set the verify to False in HttpClient.build_session() when no "certificate-authority" is found in self.config.cluster
This can be fixed with: token = fp.read().rstrip('\n')
Thanks,
Sudeep
We have the same issue with kubernetes/kubernetes#32985 .
It may be the bug of pykube
and we have tested for serval times. If we delete
the deployment, kubectl get deployment
can not find that deployment, but kubectl get rs
shows that the pods are not deleted.
If we use kubectl to delete the deployment, everything works well and the pods will switch to Terminating
and deleted forever.
We have used pykube
to implement the PaaS service and this is critical for us. Hope anyone can help to confirm and fix this ๐
>>> api = HTTPClient(KubeConfig.from_file("/Users/<username>/.kube/config"))
>>> pods = Pod.objects(api).filter(namespace="<a valid namespace>")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/chris/src/experimental/build/krypton-virtualenv/lib/python2.7/site-packages/pykube/query.py", line 83, in __call__
return Query(api, self.endpoint, self.api_obj_class, namespace=self.namespace)
AttributeError: ObjectManager instance has no attribute 'endpoint'
This is most likely an issue with how I am using the module and not a bug itself.
I have a little test script that attempts to create a deployment similar to the rc creation under usage section in the README. Here is my test script.
import operator
import pykube
dep = {
"apiVersion": "extensions/v1beta1",
"kind": "Deployment",
"metadata": {
"name": "test1"
},
"spec": {
"replicas": 3,
"template": {
"metadata": {
"labels": {
"app": "myApp"
}
},
"spec": {
"containers": [
{
"env": [
{
"name": "ROOT_URL",
"value": "http://localhost:3000"
},
{
"name": "MONGO_URL",
"value": "mongodb://localhost:27017/app"
},
{
"name": "PORT",
"value": "3000"
}
],
"image": "myApp:latest",
"name": "myApp",
"ports": [
{
"containerPort": 3000
}
]
},
{
"image": "mongo",
"name": "db",
"ports": [
{
"containerPort": 27017
}
]
}
]
}
}
}
}
pykube.HTTPClient(pykube.KubeConfig.from_file("/Users/me/.kube/config"))
pykube.Deployment(dep).create()
running this with python3
python3 example.py
This gives the following error.
Traceback (most recent call last):
File "example.py", line 114, in <module>
pykube.Deployment(dep).create()
TypeError: __init__() missing 1 required positional argument: 'obj'
I used pip to install the module.
python3 -m pip install pykube
Python version.
Python 3.5.1
What am I doing wrong here? Thanks in advance!
as title, I'd like to know whether there is some api to do the action just like:
kubectl scale --replicas=1 demo
by kubectl
thanks a lot.
This is another issue in the GKE authentication. I know the full end to end authentication is not yet done, and that GKE auth works if we already have the auth token in kube config (after this is merged #67)
But, having that token is not enough, kubectl is able to get a new token and update the 'expiry' field, but pykube, if the token is expired, can't authenticate.
I'd like to work on this but in the meantime, if someone knows how kubectl is dealing with this and share the info with me, would be helpful.
Is kubectl doing any extra request to GKE k8s api servers to fetch a new token? is kubectl using gcloud cli underneath?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.