weaveworks / kubediff Goto Github PK
View Code? Open in Web Editor NEWKubediff: a tool for Kubernetes to show differences between running state and version controlled configuration.
License: Apache License 2.0
Kubediff: a tool for Kubernetes to show differences between running state and version controlled configuration.
License: Apache License 2.0
kubernetes yaml generation tool always generate:
creationTimestamp: null
see:
https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md
so, compare creationTimestamp disturbed me.
Publish to pypi
When python3 is the default python
, typing make
crashes on the pytest step with Python syntax errors caused by running a python3 interpreter against the python2-only source code.
If kubediff requires python 2, kubediff's makefile should attempt to specifically find a python2 interpreter rather than using the system default.
kubediff on ๎ master via ๐ v3.8.10
โฏ make
touch .ensure-pip
touch .ensure-virtualenv
virtualenv .env
created virtual environment CPython3.8.10.final.0-64 in 133ms
creator CPython3Posix(dest=/home/daniel/dev/kubediff/.env, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/home/daniel/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
.env/bin/pip install -e .
Obtaining file:///home/daniel/dev/kubediff
Collecting PyYAML
Using cached PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
Collecting attrs
Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Processing /home/daniel/.cache/pip/wheels/8e/70/28/3d6ccd6e315f65f245da085482a2e1c7d14b90b30f239e2cf4/future-0.18.2-py3-none-any.whl
Installing collected packages: PyYAML, attrs, future, kubedifflib
Running setup.py develop for kubedifflib
Successfully installed PyYAML-5.4.1 attrs-21.2.0 future-0.18.2 kubedifflib
.env/bin/pip install -r requirements.txt
Requirement already satisfied: attrs in ./.env/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (21.2.0)
Requirement already satisfied: pyyaml in ./.env/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (5.4.1)
Collecting tabulate
Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)
Requirement already satisfied: future in ./.env/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (0.18.2)
Installing collected packages: tabulate
Successfully installed tabulate-0.8.9
.env/bin/pip install -r dev-requirements.txt
Collecting flake8
Using cached flake8-3.9.2-py2.py3-none-any.whl (73 kB)
Requirement already satisfied: future in ./.env/lib/python3.8/site-packages (from -r dev-requirements.txt (line 2)) (0.18.2)
Collecting hypothesis
Downloading hypothesis-6.20.1-py3-none-any.whl (371 kB)
|โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ| 371 kB 3.0 MB/s
Collecting pytest
Using cached pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting tox
Using cached tox-3.24.3-py2.py3-none-any.whl (85 kB)
Collecting mccabe<0.7.0,>=0.6.0
Using cached mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting pycodestyle<2.8.0,>=2.7.0
Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)
Collecting pyflakes<2.4.0,>=2.3.0
Using cached pyflakes-2.3.1-py2.py3-none-any.whl (68 kB)
Requirement already satisfied: attrs>=19.2.0 in ./.env/lib/python3.8/site-packages (from hypothesis->-r dev-requirements.txt (line 3)) (21.2.0)
Collecting sortedcontainers<3.0.0,>=2.1.0
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting packaging
Using cached packaging-21.0-py3-none-any.whl (40 kB)
Collecting pluggy<2.0,>=0.12
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting py>=1.8.2
Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting toml
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.7.2-py2.py3-none-any.whl (5.3 MB)
Collecting six>=1.14.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting filelock>=3.0.0
Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting distlib<1,>=0.3.1
Using cached distlib-0.3.2-py2.py3-none-any.whl (338 kB)
Collecting backports.entry-points-selectable>=1.0.4
Using cached backports.entry_points_selectable-1.1.0-py2.py3-none-any.whl (6.2 kB)
Collecting platformdirs<3,>=2
Using cached platformdirs-2.3.0-py3-none-any.whl (13 kB)
Installing collected packages: mccabe, pycodestyle, pyflakes, flake8, sortedcontainers, hypothesis, pyparsing, packaging, pluggy, iniconfig, py, toml, pytest, distlib, backports.entry-points-selectable, filelock, platformdirs, six, virtualenv, tox
Successfully installed backports.entry-points-selectable-1.1.0 distlib-0.3.2 filelock-3.0.12 flake8-3.9.2 hypothesis-6.20.1 iniconfig-1.1.1 mccabe-0.6.1 packaging-21.0 platformdirs-2.3.0 pluggy-1.0.0 py-1.10.0 pycodestyle-2.7.0 pyflakes-2.3.1 pyparsing-2.4.7 pytest-6.2.5 six-1.16.0 sortedcontainers-2.4.0 toml-0.10.2 tox-3.24.3 virtualenv-20.7.2
touch .env/.deps-uptodate
.env/bin/py.test --junitxml="junit.xml"
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/daniel/dev/kubediff
plugins: hypothesis-6.20.1
collected 13 items
kubedifflib/tests/test_diff.py ............. [100%]
===================================================================================== warnings summary ======================================================================================
.env/lib/python3.8/site-packages/past/builtins/misc.py:45
/home/daniel/dev/kubediff/.env/lib/python3.8/site-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import reload
-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------------------------------------------------------------ generated xml file: /home/daniel/dev/kubediff/junit.xml ------------------------------------------------------------------
=============================================================================== 13 passed, 1 warning in 4.44s ===============================================================================
.env/bin/flake8 kubediff
CGO_ENABLED=0 GOOS=linux go build ./vendor/github.com/tomwilkie/prom-run
vendor/github.com/tomwilkie/prom-run/main.go:16:2: cannot find package "github.com/prometheus/client_golang/prometheus" in any of:
/usr/lib/go-1.13/src/github.com/prometheus/client_golang/prometheus (from $GOROOT)
/home/daniel/go/src/github.com/prometheus/client_golang/prometheus (from $GOPATH)
vendor/github.com/tomwilkie/prom-run/main.go:17:2: cannot find package "github.com/sirupsen/logrus" in any of:
/usr/lib/go-1.13/src/github.com/sirupsen/logrus (from $GOROOT)
/home/daniel/go/src/github.com/sirupsen/logrus (from $GOPATH)
make: *** [Makefile:49: prom-run] Fehler 1
kubediff on ๎ master [?] via ๐ v3.8.10 took 14s
โฏ
If a yaml file contains multiple definitions the code errors.
Example:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: readonly-binding
subjects:
- kind: Group
name: readonly
roleRef:
kind: ClusterRole
name: readonly-role
apiVersion: rbac.authorization.k8s.io/v1alpha1
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: system-admin-binding
subjects:
- kind: ServiceAccount
name: default
namespace: system
roleRef:
kind: ClusterRole
name: admin-role
apiVersion: rbac.authorization.k8s.io/v1alpha1
Traceback (most recent call last):
File "kubediff/kubediff", line 40, in <module>
main()
File "kubediff/kubediff", line 34, in main
failed = check_files(args, printer, options.kubeconfig)
File "kubediff/kubedifflib/_diff.py", line 137, in check_files
differences += check_file(printer, path, kubeconfig=kubeconfig)
File "kubediff/kubedifflib/_diff.py", line 80, in check_file
expected = yaml.load(stream)
File "/usr/local/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "/usr/local/lib/python2.7/site-packages/yaml/composer.py", line 43, in get_single_node
event.start_mark)
yaml.composer.ComposerError: expected a single document in the stream
in "workspace/charts/x/manifests/x.yaml", line 1, column 1
but found another document
in "workspace/charts/x/manifests/x.yaml", line 10, column 1
Update alpine image
I might be missing something obvious, but is there way pass stdin to kubediff
rather than reading from a file?
I generate the yaml to stdout and would then like to diff that against the current cluster.
I'm using python so actually building it in might be another option if anyone has an example?
E.g. if we have a Deployment
foo with apiVersion: apps/v1
, kubectl get
will return it as extensions/v1beta1
It appears that you are supposed to explicitly request the version, like this: kubectl get deployment.v1.apps/foo
I didn't find this clearly documented anywhere, it's sort of inferred from the kubectl docs telling you what not to do.
Using this deployment.yaml
:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
namespace: test
spec:
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
name: http
protocol: TCP
If I change a field in the containers
part, e.g. the containerPort
, kubediff
returns:
## test/nginx (Deployment)
.spec.template.spec.containers: 'element [0]' missing
Hi, I maintain https://krew.dev, a plugin manager for kubectl.
We'd be happy to have this as kubectl diff
subcommand installed as a plugin.
If you're interested, I'm happy to walk you through the process.
I received the follow error while I am validating RoleList. It seems that RoleList doesn't have to have a name.
File "kubediff/kubedifflib/_kube.py", line 58, in from_dict
name = data["metadata"]["name"]
KeyError: 'metadata'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleList
items:
- apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: prometheus-k8s
namespace: default
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- pods
verbs:
- get
- list
- watch
When doing local development, I often don't have the entire cluster deployed, just the things I'm interested in.
I'd like to run kubediff
and see only the differences between what I've got running and what's in the files.
Bring up a kind cluster, apply manifests, do cluster-local change, confirm drift using kubediff
.
setup.py lists the license as MIT, but there's no other indication of the license in the project except that some vendored code is listed as the LGPL. Could you add a clearer indication of what license this is available under?
## monitoring/prometheus-config (ConfigMap)
Traceback (most recent call last):
File "/kubediff", line 40, in <module>
main()
File "/kubediff", line 34, in main
failed = check_files(args, printer, options.kubeconfig)
File "/usr/lib/python2.7/site-packages/kubedifflib/_diff.py", line 191, in check_files
differences += check_file(printer, path, kubeconfig=kubeconfig)
File "/usr/lib/python2.7/site-packages/kubedifflib/_diff.py", line 122, in check_file
printer.diff(path, difference)
File "/usr/lib/python2.7/site-packages/kubedifflib/_diff.py", line 158, in diff
self._write(difference.to_text())
File "/usr/lib/python2.7/site-packages/kubedifflib/_diff.py", line 145, in _write
self._stream.write(msg % args)
TypeError: not enough arguments for format string
Since the recent updates we're getting these on every run:
YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
#31 rightly made kubediff ignore list ordering, but it also made it harder than ever to understand what's wrong with a list.
e.g. consider a pod with one container. If the container definition varies, then we will now be told that the two lists of containers differ, that element 0 can't be found. We won't get any information about how the containers differ.
Some ways I can think to fix this:
Maybe i have missed something in the documentation. But after installing kubediff in my k8s cluster,
i get the following error message for all may yaml files:
## /kubediff (Deployment.v1beta1.extensions)
The connection to the server localhost:8080 was refused - did you specify the right host or port?
I am not sure what kubediff expects on localhost:8080 and were i have to configure this setting.
Any hints for me?
Hi,
I would like to use kubediff as part of CICD pipeline, just to print a summary of changes to logs. I would love to be able to skip printing the full diff for two reasons: readability and security (clear text secrets are part of the diff). In addition it would be even more awesome if kubediff could allow me to quantify the change so I can trigger warnings or stop the pipeline on bigger changes etc. Imagine output similar to git diff --stat
like so:
git diff --stat 45d5db1 foo.yaml | 199 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------------------------------------------------------------------------- 1 file changed, 65 insertions(+), 134 deletions(-)
Not sure which version, but we should update the kubectl binary in the image
It's a bit of a faff to run this locally.
When I try to diff a daemonset, I get
$ kubediff ./logstash-es.yaml
## default/logstash-elasticsearch (DaemonSet)
Error from server: daemonsets.extensions "logstash-elasticsearch" not found
But kubectl finds it:
$ kubectl get daemonset logstash-elasticsearch
NAME DESIRED CURRENT NODE-SELECTOR AGE
logstash-elasticsearch 5 5 <none> 23h
Details:
kubectl version
Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.5", GitCommit:"5a0a696437ad35c133c0c8493f7e9d22b0f9b81b", GitTreeState:"clean", BuildDate:"2016-10-29T01:38:40Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.5", GitCommit:"5a0a696437ad35c133c0c8493f7e9d22b0f9b81b", GitTreeState:"clean", BuildDate:"2016-10-29T01:32:42Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
Often, a differing configmap will only differ by a small change. It would be helpful if kubediff's output could present the differences as a line-by-line diff output.
The contents of SealedSecrets
are by definition not sensitive, so they should not be redacted when printing diffs.
Fixed by #78.
I use kustomize
[https://github.com/kubernetes-sigs/kustomize], which takes many YAML manifest files (optionally overriding certain fields via other files), and then generates a single YAML file output (with multiple documents separated by ---
).
If I point kubediff at this mega-YAML it prints out the differences at the file-level, which is unhelpful since I get all my errors collected under this file.
e.g.:
{
"dev-all.yaml": [
".spec.selector: 'name' missing",
".metadata.labels: 'name' missing",
".spec.selector: 'name' missing",
".metadata.labels: 'name' missing",
".spec.template.spec.containers: 'element [0]' missing",
".spec.template.spec.containers: 'element [0]' missing"
]
}
I would like to know the resource kind/name from the offending resources when a diff is printed (e.g. deployment/foobar, deployment/barbar)
CircleCI have deprecated their 1.0 config.
Perhaps migrate to CircleCI 2.0.
https://github.com/weaveworks/kubediff/blob/master/.circleci/config.yml#L35-L36
It'd be good to check a python3
variant manually before pushing the image live.
Not sure if this is necessary or will make things easier, but it might help to move to go modules.
I'd like to use kubediff in my environment where the proxy and firewall are working so that I cannot pull docker or something outdoor to build kubediff.
Some workaround surely exists, but is there one binary of kubediff for me to diff between my file and k8s cluster object?
kubediff should ignore the differences on missing defaults in the .yaml files. Poking through the source code you're getting the object (kubectl get
), whilst instead should do something similar to kubectl diff
using the version I cloned from github today ( 2020-02-26 ) and performing a kubediff without the -j
flag the output tells me what object is not matched up.
## /source-vars-7hfkggft52 (ConfigMap.v1.)
.metadata.annotations.scm-sha: 'cd59a385c4dadcef8a8e60aaa14fd53fbe58dd6d' != '2d109564509611cdea1a0412953e0a6b829f632a'
But when I add the -j
flag, I'm shown there us a mis-match but the object ( source-vars-7hfkggft52 ) isn't mentioned.
{
"infrastructure.yaml": [
".metadata.annotations.scm-sha: 'cd59a385c4dadcef8a8e60aaa14fd53fbe58dd6d' != '2d109564509611cdea1a0412953e0a6b829f632a'"
]
}
Move from Circle CI to GitHub workflow for publishing - maybe we can remove some of the ./tools
bits this way.
Python 2 is now end-of-life.
It would be useful to be able to check kubediff version from the cli (for example to check the installation in an automation process).
Usage could be:
kubediff -h
Usage: kubediff [options] <dir/file>...
Options:
-h, --help show this help message and exit
-v, --version show version number and exit
We wanted to specify the namespace to check against when using kubediff as we didnt wan't to run kubediff in the same namespace as the rest of our services.
In .circleci
When migrating to Python3 and using the new argparse
module, there is a print that must be removed.
def main():
options = ParseArgs()
-> print(options.args)
I had to grep around to find this one ... I'd been naming my files .yml, which kubediff didn't deal with so well. Some kind of output or a note about it in the help would be nice.
On the other hand, thank you for this tool!
Reported in PR #78
error below for e.g. SealedSecret.v1alpha1.bitnami.com
Traceback (most recent call last):
File "/Users/po/src/kubediff/kubediff", line 48, in <module>
main()
File "/Users/po/src/kubediff/kubediff", line 42, in main
failed = check_files(args, printer, config)
File "/Users/po/src/kubediff/kubedifflib/_diff.py", line 246, in check_files
differences += check_file(printer, path, config)
File "/Users/po/src/kubediff/kubedifflib/_diff.py", line 174, in check_file
printer.diff(path, difference)
File "/Users/po/src/kubediff/kubedifflib/_diff.py", line 214, in diff
self._write('%s', difference.to_text(self._current.kind))
File "/Users/po/src/kubediff/kubedifflib/_diff.py", line 35, in to_text
message = self.message % ((len(self.args[0]) * '*'), (len(self.args[1]) * '*'))
TypeError: object of type 'NoneType' has no len()
You get the following:
Traceback (most recent call last):
File "/kubediff", line 6, in <module>
from kubedifflib import (
ImportError: No module named kubedifflib
As kubedifflib isn't installed in the image.
This was introduced in #5 AFAICT
I've got the following configmap in my cluster:
kc get cm test-mat -o yaml
apiVersion: v1
data:
CONTINENT: EU
ENV: PREPROD
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"CONTINENT":"EU","ENV":"PREPROD"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"test-mat","namespace":"preprod"}}
creationTimestamp: 2019-05-17T14:01:25Z
name: test-mat
namespace: preprod
[...]
And the corresponding manifest, but with the PREPROD
field removed:
$ cat test.yml
kind: ConfigMap
metadata:
name: test-mat
data:
CONTINENT: EU
Wen I run Kubediff, it doesn't detect the change between the resource and the manifest:
$ ~/kubediff/kubediff test.yml
$
I expected the missing field in my local config to be detected as a difference.
Comparing version control configuration to live Kubernetes cluster configuration is helpful, but an option to go ahead and synchronize the effective cluster configuration to match version control would be even better!
Note that some resource types require deleting and then creating a fresh instance, rather than modifying existing instances. This includes Kubernetes Jobs, as well as many custom resources.
We are currently seeing this kubediff on prod:
## scope/pipe (Deployment)
.spec.template.spec.containers: 'element [0]' missing
## monitoring/alertmanager (Deployment)
.spec.template.spec.containers: 'element [0]' missing
AFAICT, this is because for scope/pipe, the YAML file says:
resources:
requests:
cpu: 0.5
But the k8s object says:
resources:
requests:
cpu: 500m
Despite being freshly applied.
Since this difference isn't meaningful, it shouldn't be reported.
Before we switch to python3 (#103), we should add CI which tests against old and current python releases, just so we're safe and don't break anyone.
Ie if there is a Deployment that doesn't have a corresponding yaml file checked in, we should alert.
message:
error: batch/, Kind=CronJob matches multiple kinds [batch/v1beta1, Kind=CronJob batch/v2alpha1, Kind=CronJob]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.