leastauthority / kubetop Goto Github PK

View Code? Open in Web Editor NEW

179.0 7.0 14.0 67 KB

A top(1)-like tool for Kubernetes.

License: MIT License

Python 98.19% Dockerfile 1.81%

kubernetes kubernetes-monitoring kubernetes-operator python twisted top sysadmin

kubetop's Introduction

kubetop

What is this?

kubetop is a top(1)-like tool for Kubernetes.

NOTE kubetop does not work against recent versions of Kubernetes anymore due to Kubernetes API changes. This is not expected to change in the near future. Patches welcome.

Usage Sample

$ kubetop

Output Sample

kubetop - 13:02:57
Node 0 CPU%   9.80 MEM% 57.97 (   2 GiB/   4 GiB)  POD%  7.27 (  8/110) Ready
Node 1 CPU%  21.20 MEM% 59.36 (   2 GiB/   4 GiB)  POD%  3.64 (  4/110) Ready
Node 2 CPU%  99.90 MEM% 58.11 (   2 GiB/   4 GiB)  POD%  7.27 (  8/110) Ready
Pods:       20 total        0 running        0 terminating        0 pending
                 POD               (CONTAINER)        %CPU         MEM   %MEM
s4-infrastructure-3073578190-2k2vw                    75.5  782.05 MiB  20.76
                      (subscription-converger)        72.7  459.11 MiB
                                 (grid-router)         2.7   98.07 MiB
                                         (web)         0.1   67.61 MiB
                        (subscription-manager)         0.0   91.62 MiB
                       (foolscap-log-gatherer)         0.0   21.98 MiB
                                       (flapp)         0.0   21.46 MiB
                              (wormhole-relay)         0.0   22.19 MiB

Installing

Pip / Pipsi

Python 2.7 is required to run kubetop.

To install the latest version of kubetop using pip or pipsi:

$ pipsi install kubetop

Docker

A Docker image containing a kubetop installation is also available. You can run it like this:

$ docker run -it --rm --volume ~/.kube/:/root/.kube/:ro exarkun/kubetop

Testing

kubetop uses pyunit-style tests. After installing the development dependencies, you can run the test suite with trial:

$ pip install kubetop[dev]
$ trial kubetop

Version

kubetop uses the CalVer versioning convention. The first three segments of a kubetop version tell you the year (two digit), month, and day that version was released. The fourth segment of a kubetop version is a bugfix release counter. It is present if a new release is made that diffs from a previous release only by including one or more bug fixes. For each bug fix release, the fourth segment is incremented.

License

txkube is open source software released under the MIT License. See the LICENSE file for more details.

kubetop's People

Contributors

Stargazers

Watchers

Forkers

ionelmc tyrannasaurusbanks wallrj marccardinal optionalg testimx62 theobrigitte stretchcloud basic-system darksidesfear kokizzu alfer71 icodein awesome-finops

kubetop's Issues

Display the correct memory %

It's some totally wild made up number right now.

Do an initial release

So it can be announced. So folks can try using it.

Client-Cert Error

  File "/usr/local/bin/kubetop", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/kubetop/_twistmain.py", line 79, in __call__
    ] + argv[1:])
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/twist/_twist.py", line 124, in main
    options=options.subOptions,
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/twist/_twist.py", line 62, in service
    service = plugin.makeService(options)
  File "/usr/local/lib/python2.7/dist-packages/kubetop/_twistmain.py", line 87, in _make_service
    return self.make_service(self, options)
  File "/usr/local/lib/python2.7/dist-packages/kubetop/_script.py", line 63, in makeService
    s = make_source(reactor, CONFIG, options["context"])
  File "/usr/local/lib/python2.7/dist-packages/kubetop/_topdata.py", line 32, in make_source
    kubernetes = network_kubernetes_from_context(reactor, context_name, config_path)
  File "/usr/local/lib/python2.7/dist-packages/txkube/_network.py", line 86, in network_kubernetes_from_context
    client_chain = parse(user[u"client-certificate"].bytes())
KeyError: u'client-certificate'

I already have a setup for gcloud & kube. Please explain how to generate this certificate & how to configure otherwise?

pkg_resources.ContextualVersionConflict on fresh install

Arch linux

myuser /srv/http/stuff $ pip install --user kubetop
Collecting kubetop
  Downloading kubetop-17.4.17.1.tar.gz
Collecting bitmath (from kubetop)
  Downloading bitmath-1.3.1.2.tar.gz (166kB)
    100% |████████████████████████████████| 174kB 621kB/s 
Collecting attr (from kubetop)
  Downloading attr-0.2.0.tar.gz
Requirement already satisfied: pyyaml in /usr/lib/python3.6/site-packages (from kubetop)
Collecting twisted[tls] (from kubetop)
  Downloading Twisted-17.1.0.tar.bz2 (3.0MB)
    100% |████████████████████████████████| 3.0MB 395kB/s 
Collecting treq (from kubetop)
  Downloading treq-17.3.1-py2.py3-none-any.whl (41kB)
    100% |████████████████████████████████| 51kB 1.8MB/s 
Collecting txkube (from kubetop)
  Downloading txkube-0.1.0.tar.gz (124kB)
    100% |████████████████████████████████| 133kB 1.2MB/s 
Collecting zope.interface>=4.0.2 (from twisted[tls]->kubetop)
  Downloading zope.interface-4.4.0-cp36-cp36m-manylinux1_x86_64.whl (174kB)
    100% |████████████████████████████████| 184kB 1.4MB/s 
Collecting constantly>=15.1 (from twisted[tls]->kubetop)
  Downloading constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from twisted[tls]->kubetop)
  Downloading incremental-16.10.1-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from twisted[tls]->kubetop)
  Downloading Automat-0.5.0-py2.py3-none-any.whl
Collecting pyopenssl>=16.0.0 (from twisted[tls]->kubetop)
  Downloading pyOpenSSL-17.0.0-py2.py3-none-any.whl (51kB)
    100% |████████████████████████████████| 61kB 2.0MB/s 
Collecting service_identity (from twisted[tls]->kubetop)
  Downloading service_identity-16.0.0-py2.py3-none-any.whl
Collecting idna>=0.6 (from twisted[tls]->kubetop)
  Downloading idna-2.5-py2.py3-none-any.whl (55kB)
    100% |████████████████████████████████| 61kB 2.2MB/s 
Requirement already satisfied: requests>=2.1.0 in /home/myuser/.local/lib/python3.6/site-packages (from treq->kubetop)
Collecting attrs (from treq->kubetop)
  Using cached attrs-16.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/lib/python3.6/site-packages (from treq->kubetop)
Collecting pyrsistent (from txkube->kubetop)
  Downloading pyrsistent-0.12.1.tar.gz (94kB)
    100% |████████████████████████████████| 102kB 1.7MB/s 
Collecting pem (from txkube->kubetop)
  Downloading pem-16.1.0-py2.py3-none-any.whl
Collecting eliot (from txkube->kubetop)
  Downloading eliot-1.0.0-py2.py3-none-any.whl (90kB)
    100% |████████████████████████████████| 92kB 1.2MB/s 
Requirement already satisfied: python-dateutil in /home/myuser/.local/lib/python3.6/site-packages (from txkube->kubetop)
Collecting pykube (from txkube->kubetop)
  Downloading pykube-0.15.0-py2.py3-none-any.whl
Collecting klein (from txkube->kubetop)
  Downloading klein-17.2.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (from zope.interface>=4.0.2->twisted[tls]->kubetop)
Collecting cryptography>=1.7 (from pyopenssl>=16.0.0->twisted[tls]->kubetop)
  Downloading cryptography-1.8.1.tar.gz (423kB)
    100% |████████████████████████████████| 430kB 1.6MB/s 
Requirement already satisfied: pyasn1-modules in /home/myuser/.local/lib/python3.6/site-packages (from service_identity->twisted[tls]->kubetop)
Requirement already satisfied: pyasn1 in /home/myuser/.local/lib/python3.6/site-packages (from service_identity->twisted[tls]->kubetop)
Collecting tzlocal (from pykube->txkube->kubetop)
  Downloading tzlocal-1.4.tar.gz
Collecting requests-oauthlib (from pykube->txkube->kubetop)
  Downloading requests_oauthlib-0.8.0-py2.py3-none-any.whl
Requirement already satisfied: oauth2client in /home/myuser/.local/lib/python3.6/site-packages (from pykube->txkube->kubetop)
Collecting werkzeug (from klein->txkube->kubetop)
  Downloading Werkzeug-0.12.1-py2.py3-none-any.whl (312kB)
    100% |████████████████████████████████| 317kB 1.8MB/s 
Requirement already satisfied: packaging>=16.8 in /usr/lib/python3.6/site-packages (from setuptools->zope.interface>=4.0.2->twisted[tls]->kubetop)
Requirement already satisfied: appdirs>=1.4.0 in /usr/lib/python3.6/site-packages (from setuptools->zope.interface>=4.0.2->twisted[tls]->kubetop)
Collecting asn1crypto>=0.21.0 (from cryptography>=1.7->pyopenssl>=16.0.0->twisted[tls]->kubetop)
  Downloading asn1crypto-0.22.0-py2.py3-none-any.whl (97kB)
    100% |████████████████████████████████| 102kB 3.0MB/s 
Collecting cffi>=1.4.1 (from cryptography>=1.7->pyopenssl>=16.0.0->twisted[tls]->kubetop)
  Downloading cffi-1.10.0-cp36-cp36m-manylinux1_x86_64.whl (406kB)
    100% |████████████████████████████████| 409kB 1.2MB/s 
Collecting pytz (from tzlocal->pykube->txkube->kubetop)
  Downloading pytz-2017.2-py2.py3-none-any.whl (484kB)
    100% |████████████████████████████████| 491kB 1.2MB/s 
Collecting oauthlib>=0.6.2 (from requests-oauthlib->pykube->txkube->kubetop)
  Downloading oauthlib-2.0.2.tar.gz (125kB)
    100% |████████████████████████████████| 133kB 2.2MB/s 
Requirement already satisfied: httplib2>=0.9.1 in /home/myuser/.local/lib/python3.6/site-packages (from oauth2client->pykube->txkube->kubetop)
Requirement already satisfied: rsa>=3.1.4 in /home/myuser/.local/lib/python3.6/site-packages (from oauth2client->pykube->txkube->kubetop)
Requirement already satisfied: pyparsing in /usr/lib/python3.6/site-packages (from packaging>=16.8->setuptools->zope.interface>=4.0.2->twisted[tls]->kubetop)
Collecting pycparser (from cffi>=1.4.1->cryptography>=1.7->pyopenssl>=16.0.0->twisted[tls]->kubetop)
  Downloading pycparser-2.17.tar.gz (231kB)
    100% |████████████████████████████████| 235kB 1.8MB/s 
Installing collected packages: bitmath, attr, zope.interface, constantly, incremental, attrs, Automat, idna, asn1crypto, pycparser, cffi, cryptography, pyopenssl, service-identity, twisted, treq, pyrsistent, pem, eliot, pytz, tzlocal, oauthlib, requests-oauthlib, pykube, werkzeug, klein, txkube, kubetop
  Running setup.py install for bitmath ... done
  Running setup.py install for attr ... done
  Running setup.py install for pycparser ... done
  Running setup.py install for cryptography ... done
  Running setup.py install for twisted ... done
  Running setup.py install for pyrsistent ... done
  Running setup.py install for tzlocal ... done
  Running setup.py install for oauthlib ... done
  Running setup.py install for txkube ... done
  Running setup.py install for kubetop ... done
Successfully installed Automat-0.5.0 asn1crypto-0.22.0 attr-0.2.0 attrs-16.3.0 bitmath-1.3.1.2 cffi-1.10.0 constantly-15.1.0 cryptography-1.8.1 eliot-1.0.0 idna-2.5 incremental-16.10.1 klein-17.2.0 kubetop-17.4.17.1 oauthlib-2.0.2 pem-16.1.0 pycparser-2.17 pykube-0.15.0 pyopenssl-17.0.0 pyrsistent-0.12.1 pytz-2017.2 requests-oauthlib-0.8.0 service-identity-16.0.0 treq-17.3.1 twisted-17.1.0 txkube-0.1.0 tzlocal-1.4 werkzeug-0.12.1 zope.interface-4.4.0

myuser /srv/http/stuff $ kubetop
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 654, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 968, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 859, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (Twisted 17.1.0 (/home/myuser/.local/lib/python3.6/site-packages), Requirement.parse('twisted[tls]!=17.1.0'), {'txkube'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/myuser/.local/bin/kubetop", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3036, in <module>
    @_call_aside
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3020, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3049, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 656, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 669, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 859, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (Twisted 17.1.0 (/home/myuser/.local/lib/python3.6/site-packages), Requirement.parse('twisted[tls]!=17.1.0'), {'txkube'})

Colorize everything

For easier reading.

Format memory usage more nicely

It would be good to have the decimal point aligned in all the values. This makes the numbers much easier to read.

Travis should not allow test failures on Python 3

When we fix all the kubetop tests on Python 3, and also release a version of
txkube which works on Python 3, then travis should not allow test failures on Python 3.

Clean up output in response to C-c

C-c is the only way to exit kubetop right now. It can leave unsightly blemishes on the terminal:

^CTraceback (most recent call last):
Failure: twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.web._newclient.ResponseFailed'>: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>, <twisted.python.failure.Failure twisted.web.http._DataLoss: Chunked decoder in 'BODY' state, still expecting more data to get to 'FINISHED' state.>]
]]
]]

Distribute as a docker container?

I think this could be great to distribute as a docker container.

Run with

docker run -it -v $HOME/.kube:/root/.kube kubetop

I was going to push my own container, but couldn't figure out the exact requirements to make it run.

Set up CI

kubetop command-line fails on Python 3

I tried to run the kubetop command-line on Python 3 and ran into this:

Traceback (most recent call last):
  File "/Users/crodrigues/venv-3.6/bin/kubetop", line 11, in <module>
    load_entry_point('kubetop', 'console_scripts', 'kubetop')()
  File "/Users/crodrigues/kubetop/src/kubetop/_twistmain.py", line 79, in __call__
    ] + argv[1:])
  File "/Users/crodrigues/twisted10/src/twisted/application/twist/_twist.py", line 119, in main
    options = cls.options(argv)
  File "/Users/crodrigues/twisted10/src/twisted/application/twist/_twist.py", line 40, in options
    options.parseOptions(argv[1:])
  File "/Users/crodrigues/twisted10/src/twisted/application/twist/_options.py", line 168, in parseOptions
    Options.parseOptions(self, options=options)
  File "/Users/crodrigues/twisted10/src/twisted/python/usage.py", line 235, in parseOptions
    self.shortOpt, self.longOpt)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/getopt.py", line 88, in getopt
    while args and args[0].startswith('-') and args[0] != '-':
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

Add the pod phase/count line

From the readme:

Pods:     123 total      7 Running     3 Terminating     1 Pending

Render minutes into the timestamp

Currently rendering the month 🤦‍♂️

Render memory percentage without scientific notation

Oops:

s4-infrastructure-2487244764-zfp4h                     108  566.50 MiB1.5e+01

render pod/container top data one time and then exit

Just get some rendering code in place without worrying about periodic updates and such.

Exceptions running both docker and local version

$ docker run -it --rm --volume ~/.kube/:/root/.kube/:ro exarkun/kubetop
Unable to find image 'exarkun/kubetop:latest' locally
latest: Pulling from exarkun/kubetop
79650cf9cc01: Pull complete
581a2604819e: Pull complete
a7826377ee0a: Pull complete
406f1af34710: Pull complete
6322e2dfca88: Pull complete
f4278c7f5467: Pull complete
43da5e7d9205: Pull complete
94e609f3078b: Pull complete
Digest: sha256:fc6a1f382cb6775862a109b02760a40a6117f0ea9f8cd43e9f143e9e990c37d7
Status: Downloaded newer image for exarkun/kubetop:latest
Traceback (most recent call last):
Failure: twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'schedulerName' are not among the specified fields for v1.PodSpec
/usr/local/lib/python2.7/site-packages/twisted/web/_newclient.py:1185:_bodyDataFinished_CONNECTED
/usr/local/lib/python2.7/site-packages/twisted/web/client.py:2115:connectionLost
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:457:callback
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:565:_startRunCallbacks
--- ---
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:651:_runCallbacks
/usr/local/lib/python2.7/site-packages/eliot/twisted.py:82:callbackWithContext
/usr/local/lib/python2.7/site-packages/eliot/_action.py:389:run
/usr/local/lib/python2.7/site-packages/txkube/_network.py:271:
/usr/local/lib/python2.7/site-packages/pyrsistent/_helpers.py:80:inner_f
/usr/local/lib/python2.7/site-packages/txkube/_model.py:379:iobject_from_raw
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/txkube/_model.py:256:new
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:52:new
/usr/local/lib/python2.7/site-packages/pyrsistent/_checked_types.py:200:_checked_type_create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:52:new
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:255:optional
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:67:new

Add timeouts to network operations

Sometimes requests for Kubernetes data can get hung up. Right now they don't ever time out so that's the end of updates. Add some timeouts so that things move on and we might eventually try again. Bonus points for rendering some information about request errors into the display somewhere so it's clear stale data is being displayed.

Render CPU usage correctly

CPU information is rendered inconsistently between pods and nodes and neither one seems to be a percentage.

Crashes immediately with Kubernetes 1.6

I ran kubetop against my local minikube installation and got this error: 'schedulerName' are not among the specified fields for v1.PodSpec.

I tried kubectl get po immediately after, to confirm k8s was running in some mostly-working way, and it was indeed working.

I don't know if Kubernetes 1.6 is the main factor, but subject lines are hard.

Full output

$ kubetop
Traceback (most recent call last):
Failure: twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'schedulerName' are not among the specified fields for v1.PodSpec
/Users/jml/Library/Python/2.7/lib/python/site-packages/twisted/web/_newclient.py:1210:_bodyDataFinished_CONNECTED
/Users/jml/Library/Python/2.7/lib/python/site-packages/twisted/web/client.py:2144:connectionLost
/Users/jml/Library/Python/2.7/lib/python/site-packages/twisted/internet/defer.py:459:callback
/Users/jml/Library/Python/2.7/lib/python/site-packages/twisted/internet/defer.py:567:_startRunCallbacks
--- <exception caught here> ---
/Users/jml/Library/Python/2.7/lib/python/site-packages/twisted/internet/defer.py:653:_runCallbacks
/Users/jml/Library/Python/2.7/lib/python/site-packages/eliot/twisted.py:82:callbackWithContext
/Users/jml/Library/Python/2.7/lib/python/site-packages/eliot/_action.py:389:run
/Users/jml/Library/Python/2.7/lib/python/site-packages/txkube/_network.py:271:<lambda>
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_helpers.py:80:inner_f
/Users/jml/Library/Python/2.7/lib/python/site-packages/txkube/_model.py:379:iobject_from_raw
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_pclass.py:113:create
/Users/jml/Library/Python/2.7/lib/python/site-packages/txkube/_model.py:256:__new__
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_pclass.py:52:__new__
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_checked_types.py:200:_checked_type_create
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_pclass.py:113:create
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_pclass.py:52:__new__
/Users/jml/Library/Python/2.7/lib/python/site-packages/txkube/_swagger.py:255:optional
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_pclass.py:113:create
/Users/jml/Library/Python/2.7/lib/python/site-packages/pyrsistent/_pclass.py:67:__new__
]]
]]

Versions

$ kubetop --version
Twisted version: 17.1.0

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.0", GitCommit:"fff5156092b56e6bd60fff75aad4dc9de6b6ef37", GitTreeState:"clean", BuildDate:"2017-03-28T16:36:33Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.0", GitCommit:"fff5156092b56e6bd60fff75aad4dc9de6b6ef37", GitTreeState:"dirty", BuildDate:"2017-04-07T20:46:46Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"linux/amd64"}

$ minikube version
minikube version: v0.18.0

macOS 10.12.4

filter() is being used in a way in the partition() function which does not work on Python 3

When trying to run some of the tests in Python 3, I ran into this:

[ERROR]
Traceback (most recent call last):
  File "/Users/crodrigues/kubetop/src/kubetop/test/test_textrenderer.py", line 226, in test_render_several
    in _render_pods(pods, pod_usage, nodes).splitlines()
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 295, in _render_pods
    in sorted(pod_usage, key=_pod_stats, reverse=True)
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 312, in _pod_stats
    ), 0,
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 378, in parse_cpu
    return parse_k8s_resource(s, default_scale=1000)
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 391, in parse_k8s_resource
    return int(amount) * scale
builtins.TypeError: int() argument must be a string, a bytes-like object or a number, not 'filter'

SyntaxError after install

I installed kubetop using the following command sudo pip install kubetop, which installed kubetop version 17.4.17.1 and its dependencies.

When I try to run kubetop it crashes and show the following error:

$ kubetop
Traceback (most recent call last):
  File "/usr/bin/kubetop", line 11, in <module>
    load_entry_point('kubetop==17.4.17.1', 'console_scripts', 'kubetop')()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 487, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2728, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2346, in load
    return self.resolve()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2352, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.7/site-packages/kubetop/_script.py", line 27, in <module>
    from ._textrenderer import Sink, kubetop
  File "/usr/lib/python3.7/site-packages/kubetop/_textrenderer.py", line 114
    def _render_pod_top(reactor, (node_info, pod_info)):
                                 ^
SyntaxError: invalid syntax

I am running archlinux and Python 3.7.1.

Is kubetop supported on Windows 10?

OS: Windows 10 Enterprise
Following is the error, I get after installing in Cygwin. Prior to trying on Cygwin, I had installed kubetop by first installing python using python-3.7.1-amd64.exe and had encountered similar error.

==========
$ kubetop
Unhandled Error
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/twisted/application/twist/_options.py", line 28, in
class TwistOptions(Options):
File "/usr/lib/python3.6/site-packages/twisted/application/twist/_options.py", line 75, in TwistOptions
'"{}"'.format(rt.shortName) for rt in getReactorTypes()
File "/usr/lib/python3.6/site-packages/twisted/application/twist/_options.py", line 75, in
'"{}"'.format(rt.shortName) for rt in getReactorTypes()
File "/usr/lib/python3.6/site-packages/twisted/plugin.py", line 213, in getPlugins
allDropins = getCache(package)
--- ---
File "/usr/lib/python3.6/site-packages/twisted/plugin.py", line 171, in getCache
provider = pluginModule.load()
File "/usr/lib/python3.6/site-packages/twisted/python/modules.py", line 392, in load
return self.pathEntry.pythonPath.moduleLoader(self.name)
File "/usr/lib/python3.6/site-packages/twisted/python/reflect.py", line 308, in namedAny
topLevelPackage = importAndCheckStack(trialname)
File "/usr/lib/python3.6/site-packages/twisted/python/reflect.py", line 247, in importAndCheckStack
return import(importName)
File "/usr/lib/python3.6/site-packages/twisted/plugins/twisted_core.py", line 6, in
from twisted.internet.endpoints import (
File "/usr/lib/python3.6/site-packages/twisted/internet/endpoints.py", line 30, in
from twisted.internet.address import (
File "/usr/lib/python3.6/site-packages/twisted/internet/address.py", line 23, in
class IPv4Address(object):
File "/usr/lib/python3.6/site-packages/twisted/internet/address.py", line 37, in IPv4Address
type = attr.ib(validator=attr.validators.in(["TCP", "UDP"]))
builtins.AttributeError: module 'attr.validators' has no attribute 'in'

Traceback (most recent call last):
File "/usr/bin/kubetop", line 7, in
from kubetop._script import main
File "/usr/lib/python3.6/site-packages/kubetop/_script.py", line 26, in
from ._runmany import run_many_service
File "/usr/lib/python3.6/site-packages/kubetop/_runmany.py", line 17, in
from twisted.internet.task import deferLater
File "/usr/lib/python3.6/site-packages/twisted/internet/task.py", line 25, in
from twisted.internet import base, defer
File "/usr/lib/python3.6/site-packages/twisted/internet/base.py", line 27, in
from twisted.internet.resolver import (
File "/usr/lib/python3.6/site-packages/twisted/internet/resolver.py", line 25, in
from twisted.internet.address import IPv4Address, IPv6Address
File "/usr/lib/python3.6/site-packages/twisted/internet/address.py", line 23, in
class IPv4Address(object):
File "/usr/lib/python3.6/site-packages/twisted/internet/address.py", line 37, in IPv4Address
type = attr.ib(validator=attr.validators.in(["TCP", "UDP"]))
AttributeError: module 'attr.validators' has no attribute 'in'

Retrieve and display node resource information

The README suggests there is a top(1)-CPU-like display for resource usage on nodes.

Implement that.

unicode builtin is referenced in code, but it is gone on Python 3

Trying to run the tests in Python 3, I ran into this:

Traceback (most recent call last):
  File "/Users/crodrigues/kubetop/src/kubetop/test/test_textrenderer.py", line 226, in test_render_several
    in _render_pods(pods, pod_usage, nodes).splitlines()
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 295, in _render_pods
    in sorted(pod_usage, key=_pod_stats, reverse=True)
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 312, in _pod_stats
    ), 0,
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 378, in parse_cpu
    return parse_k8s_resource(s, default_scale=1000)
  File "/Users/crodrigues/kubetop/src/kubetop/_textrenderer.py", line 386, in parse_k8s_resource
    amount, suffix = partition(s, unicode.isdigit)
builtins.NameError: name 'unicode' is not defined

Gather data from all namespaces

Currently only the default namespace get scraped.

python setup.py develop fails on Python 3

I tried to run python setup.py develop on Python 3 and got this:

Traceback (most recent call last):
  File "setup.py", line 8, in <module>
    exec(f.read(), _metadata)
  File "<string>", line 3, in <module>
TypeError: unsupported operand type(s) for +: 'map' and 'list'

kubetop crashes with deprecation warning when used against kubernetes 1.10.3

I created a new Python 3.6 virtualenv, and tried to install kubetop from sources.

I ran against a kubernetes 1.10.3 cluster, and
right away, kubetop crashed:

Traceback (most recent call last):
--- <exception caught here> ---
  File "/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/kubetop-17.4.17.1.dev0-py3.6.egg/kubetop/_runmany.py", line 33, in _iterate
    yield f()
twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback: <class 'txkube._swagger.NotClassLike'>:
('io.k8s.kubernetes.pkg.api.v1.Namespace', pmap({'$ref': '#/definitions/io.k8s.api.core.v1.Namespace', 'description': 'Deprecated. Please use io.k8s.api.core.v1.Namespace instead.'}))
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/kubetop-17.4.17.1.dev0-py3.6.egg/kubetop/_topdata.py:48:_pods
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/kubetop-17.4.17.1.dev0-py3.6.egg/kubetop/_topdata.py:83:_pod_usage_from_client
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/Twisted-18.7.0-py3.6-linux-x86_64.egg/twisted/internet/defer.py:322:addCallback
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/Twisted-18.7.0-py3.6-linux-x86_64.egg/twisted/internet/defer.py:311:addCallbacks
--- <exception caught here> ---
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/Twisted-18.7.0-py3.6-linux-x86_64.egg/twisted/internet/defer.py:654:_runCallbacks
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/kubetop-17.4.17.1.dev0-py3.6.egg/kubetop/_topdata.py:83:<lambda>
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/txkube-0.3.0-py3.6.egg/txkube/_swagger.py:957:__getattr__
/home/ubuntu/kubetop-venv/lib/python3.6/site-packages/txkube-0.3.0-py3.6.egg/txkube/_swagger.py:179:pclass_for_definition
]]
]]

incorrect dependency on attr instead of attrs

glyph reported this incorrect dependency - http://twistedmatrix.com/pipermail/twisted-python/2017-April/031372.html

setup.py names the wrong package. kubetop apparently works regardless because there is a transitive dependency on the right package via one (or more) of the other direct dependencies.

Show metrics for cluster (all nodes aggregated)

Thanks for implementing kubetop, I've been personally meaning to write it myself for a while.

What I had in mind separately was to aggregate sum of all nodes in the cluster and show CPU/memory metrics of an entire cluster. Currently I would have to sum all the numbers and do the math myself to find out overall cluster utilization.

I got this idea from Google Container Engine as in GKE we list the cluster as total resources (e.g. Total cores: 24 vCPUs, Total memory 84.00 GB) and it would be really cool to see how much of the cluster I am utilizing currently.

Tuple parameter packing in _topdata.py does not work on Python 3

I ran into this while trying to run kubetop on Python 3:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/crodrigues/kubetop/src/kubetop/_topdata.py", line 51
    lambda (usage, info): {
           ^
SyntaxError: invalid syntax

exceptions.AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'

Getting this error with Python 2.7.5 or red hat linux 7.3

Traceback (most recent call last):
  File "/usr/bin/kubetop", line 7, in <module>
    from kubetop._script import main
  File "/usr/lib/python2.7/site-packages/kubetop/_script.py", line 25, in <module>
    from ._twistmain import TwistMain
  File "/usr/lib/python2.7/site-packages/kubetop/_twistmain.py", line 43, in <module>
    from twisted.application.twist._twist import Twist
  File "/usr/lib64/python2.7/site-packages/twisted/application/twist/_twist.py", line 14, in <module>
    from ..runner._runner import Runner, RunnerOptions
  File "/usr/lib64/python2.7/site-packages/twisted/application/runner/_runner.py", line 20, in <module>
    from twisted.internet import default as defaultReactor
  File "/usr/lib64/python2.7/site-packages/twisted/internet/default.py", line 56, in <module>
    install = _getInstallFunction(platform)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/default.py", line 44, in _getInstallFunction
    from twisted.internet.epollreactor import install
  File "/usr/lib64/python2.7/site-packages/twisted/internet/epollreactor.py", line 24, in <module>
    from twisted.internet import posixbase
  File "/usr/lib64/python2.7/site-packages/twisted/internet/posixbase.py", line 18, in <module>
    from twisted.internet import error, udp, tcp
  File "/usr/lib64/python2.7/site-packages/twisted/internet/tcp.py", line 28, in <module>
    from twisted.internet._newtls import (
  File "/usr/lib64/python2.7/site-packages/twisted/internet/_newtls.py", line 21, in <module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "/usr/lib64/python2.7/site-packages/twisted/protocols/tls.py", line 63, in <module>
    from twisted.internet._sslverify import _setAcceptableProtocols
  File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 38, in <module>
    TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'

I tried in python 3.6.0 but got a different error (I'll open a separate ticket)

Need a travis build for Python 3

We need a travis build running on Python 3, so we can make sure that the unit tests pass as we port kubetop to Python 3.

Doesn't work in Python 3

Traceback (most recent call last):
  File "/home/gjc/.virtualenvs/py3/bin/kubetop", line 7, in <module>
    from kubetop._script import main
  File "/home/gjc/.virtualenvs/py3/lib/python3.6/site-packages/kubetop/_script.py", line 27, in <module>
    from ._textrenderer import Sink, kubetop
  File "/home/gjc/.virtualenvs/py3/lib/python3.6/site-packages/kubetop/_textrenderer.py", line 114
    def _render_pod_top(reactor, (node_info, pod_info)):
                                 ^
SyntaxError: invalid syntax

port kubetop to Python 3

pip3 install kubetop on 16.04 ubuntu leads to:

$ kubetop 
Traceback (most recent call last):
  File "/usr/local/bin/kubetop", line 7, in <module>
    from kubetop._script import main
  File "/usr/local/lib/python3.5/dist-packages/kubetop/_script.py", line 27, in <module>
    from ._textrenderer import Sink, kubetop
  File "/usr/local/lib/python3.5/dist-packages/kubetop/_textrenderer.py", line 114
    def _render_pod_top(reactor, (node_info, pod_info)):
                                 ^
SyntaxError: invalid syntax

exceptions.AttributeError: 'schedulerName, tolerations' are not among the specified fields for v1.PodSpec

I got this from my Google Container Engine 1.6.4 cluster:

$ kubetop
Traceback (most recent call last):
Failure: twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'schedulerName, tolerations' are not among the specified fields for v1.PodSpec
/usr/local/lib/python2.7/site-packages/twisted/web/_newclient.py:1220:_bodyDataFinished_CONNECTED
/usr/local/lib/python2.7/site-packages/twisted/web/client.py:2152:connectionLost
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:459:callback
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:567:_startRunCallbacks
--- <exception caught here> ---
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:653:_runCallbacks
/usr/local/lib/python2.7/site-packages/eliot/twisted.py:82:callbackWithContext
/usr/local/lib/python2.7/site-packages/eliot/_action.py:389:run
/usr/local/lib/python2.7/site-packages/txkube/_network.py:311:<lambda>
/usr/local/lib/python2.7/site-packages/pyrsistent/_helpers.py:80:inner_f
/usr/local/lib/python2.7/site-packages/txkube/_model.py:94:iobject_from_raw
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:756:discard_constant_fields
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:754:ctor
/usr/local/lib/python2.7/site-packages/txkube/_model.py:260:__new__
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:52:__new__
/usr/local/lib/python2.7/site-packages/pyrsistent/_checked_types.py:221:_checked_type_create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:756:discard_constant_fields
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:754:ctor
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:52:__new__
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:347:optional
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:756:discard_constant_fields
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:754:ctor
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:67:__new__
]]
]]

(kubetop 17.4.17.1)

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.6", GitCommit:"7fa1c1756d8bc963f1a389f4a6937dc71f08ada2", GitTreeState:"clean", BuildDate:"2017-06-16T18:34:20Z", GoVersion:"go1.7.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:33:17Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}

Don't render past the bottom of the terminal

Only so much fits on a screen. Chop off everything afterwards instead of rendering it and scrolling the top info away.

Support initContainerStatuses

When I run kubetop I get an error 'initContainerStatuses' are not among the specified fields for v1.PodStatus

Here is the full output

➜  ~ kubetop
Traceback (most recent call last):
Failure: twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'initContainerStatuses' are not among the specified fields for v1.PodStatus
/usr/local/lib/python2.7/site-packages/twisted/web/_newclient.py:1210:_bodyDataFinished_CONNECTED
/usr/local/lib/python2.7/site-packages/twisted/web/client.py:2144:connectionLost
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:459:callback
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:567:_startRunCallbacks
--- <exception caught here> ---
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:653:_runCallbacks
/usr/local/lib/python2.7/site-packages/eliot/twisted.py:82:callbackWithContext
/usr/local/lib/python2.7/site-packages/eliot/_action.py:389:run
/usr/local/lib/python2.7/site-packages/txkube/_network.py:271:<lambda>
/usr/local/lib/python2.7/site-packages/pyrsistent/_helpers.py:80:inner_f
/usr/local/lib/python2.7/site-packages/txkube/_model.py:379:iobject_from_raw
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/txkube/_model.py:256:__new__
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:52:__new__
/usr/local/lib/python2.7/site-packages/pyrsistent/_checked_types.py:200:_checked_type_create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:52:__new__
/usr/local/lib/python2.7/site-packages/txkube/_swagger.py:255:optional
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:113:create
/usr/local/lib/python2.7/site-packages/pyrsistent/_pclass.py:67:__new__
]]
]]```

python script in image gets permission denied

Seems the python script is not run as root and thus with the command from Readme we get

Traceback (most recent call last):
  File "/usr/local/bin/kubetop", line 11, in <module>
    load_entry_point('kubetop==17.4.17.1.dev0', 'console_scripts', 'kubetop')()
  File "/usr/local/lib/python2.7/site-packages/kubetop/_twistmain.py", line 89, in __call__
    ] + argv[1:])
  File "/usr/local/lib/python2.7/site-packages/twisted/application/twist/_twist.py", line 119, in main
    options = cls.options(argv)
  File "/usr/local/lib/python2.7/site-packages/twisted/application/twist/_twist.py", line 40, in options
    options.parseOptions(argv[1:])
  File "/usr/local/lib/python2.7/site-packages/twisted/application/twist/_options.py", line 168, in parseOptions
    Options.parseOptions(self, options=options)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/usage.py", line 267, in parseOptions
    self.subOptions.parseOptions(rest)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/usage.py", line 277, in parseOptions
    self.postOptions()
  File "/usr/local/lib/python2.7/site-packages/kubetop/_script.py", line 49, in postOptions
    self['context'] = current_context(FilePath(expanduser(self['config'])))
  File "/usr/local/lib/python2.7/site-packages/kubetop/_script.py", line 34, in current_context
    with config_path.open() as cfg:
  File "/usr/local/lib/python2.7/site-packages/twisted/python/filepath.py", line 1012, in open
    return open(self.path, mode + 'b')
IOError: [Errno 13] Permission denied: '/root/.kube/config'

 $ kubetop --version
Twisted version: 17.1.0

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:52:34Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

Running MacOS Sierra
Python 2.7

When running kubetop, I receive this error:

 $ kubetop
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 459, in callback
    self._startRunCallbacks(result)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1357, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python2.7/site-packages/kubetop/_runmany.py", line 33, in _iterate
    yield f()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/kubetop/_textrenderer.py", line 91, in _render_kubetop
    sink.write(_render_pod_top(reactor, data))
  File "/usr/local/lib/python2.7/site-packages/kubetop/_textrenderer.py", line 124, in _render_pod_top
    _render_nodes(nodes, node_usage, pods),
  File "/usr/local/lib/python2.7/site-packages/kubetop/_textrenderer.py", line 182, in _render_nodes
    in enumerate(nodes)
  File "/usr/local/lib/python2.7/site-packages/kubetop/_textrenderer.py", line 181, in <genexpr>
    for i, node
exceptions.KeyError: u'ip-X-X-X-X.ec2.internal'