Comments (4)
I think this is because of our import for the otel semconv that uses 1.12, which is pretty old https://github.com/kubernetes-sigs/descheduler/blob/748495a022b7445e3bcce0be20effc4890b8a9fc/pkg/tracing/tracing.go#L30C11-L30C51
I opened #1429 to bump this to 1.24 which should help. What's annoying is that we will have to keep these in sync, so I'd like to see if there's a way we can drop the direct semconv import. I think it's being used by some of the other otel imports too, and the conflict is probably coming from one of those being at a higher version.
Also not sure why this didn't come up in tests. @morremeyer is this a fatal error or is the descheduler still running with that error? Our otel implementation should be silent if it's not enabled so an error in this case is bad
from descheduler.
@morremeyer thanks, to help debug why your pods aren't being evicted you can check the docs which list all the reasons a pod would be skipped by default. You can also set --v=4
to increase the logging verbosity to print a message why each pod is skipped.
I'll update my PR because I don't think we should be causing fatal errors if tracing fails to start up
from descheduler.
@damemi it is a fatal error, it exits directly after the last log line. Once I disable it, it works fine.
I also have the problem that if set the helm values to
deschedulerPolicy:
tracing:
collectorEndpoint: monitoring-alloy.monitoring.svc.cluster.local:4317
sampleRate: 1.0
It does not even recognize the config. I did not mention this in the initial comment since it also looks like the policy (see initial comment) does not do anything - it's running but no pods are evicted, even ones that look like they match the policy to me. I didn't comment about this because so far I think this is a user error and I'm still looking into why this is happening.
from descheduler.
I agree, if tracing fails to start up, that should be a warning or error log message and then execution continues.
Thanks for looking into it this quick, I appreciate it a lot!
from descheduler.
Related Issues (20)
- The descheduler supports limiting the total number of pods evicted per rescheduling cycle HOT 3
- Invalid Helm template in v0.30.1 HOT 7
- "Pod fits on node" that has lower utilization than current node HOT 1
- There is no plugin for podAffinity violations HOT 4
- Allow pods to self-identify as being unable to deschedule
- Publish helm chart also as an OCI package (e.g. on GHCR registry)
- Prometheus Scraping with Secure Port HOT 3
- Migrate all e2es to run descheduler as an image HOT 13
- Possibility of selecting nodes that descheduler analysis with Helm HOT 2
- Support node selector for each profile
- Bump Kubernetes dependencies to v1.31.0
- 1.31: Update CI in test-infra
- Promote v0.31.0 docker images
- Create v0.31.0 tag on master
- Helm chart version update to v0.31.0
- 1.31: Update version references in docs and readme
- Cut release-1.31 branch
- Publish v0.31.0 GitHub Release HOT 1
- Email sig mailing list
- Replace namespace include/exclude filter with label selector HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from descheduler.