Comments (11)
@vrutkovs i believe this is transient error. The operator is reporting available==true, the problem here is that we just not cleaning up the failing status.
from cluster-openshift-apiserver-operator.
Right, this happens on ~10% of installs and doesn't seem to be reporting a real issue.
These conditions are used in playbooks to wait for operator to complete setup though
from cluster-openshift-apiserver-operator.
This now happens fairly often and blocks ingress operator install: operator gets stuck on 'Generation: expected: 3, actual: 2'
(numbers may vary, but this is most frequent combination) and ingress is not deployed
from cluster-openshift-apiserver-operator.
This causes CVO to stumble and blocks other operators - e.g. machine approver, so some CI runs don't get CSR approved and return empty logs during artifacts stage
from cluster-openshift-apiserver-operator.
@abhinavdahiya how long the CVO wait to settle? Also, any idea why the CI isn't pulling pod logs?
from cluster-openshift-apiserver-operator.
how long the CVO wait to settle?
it waits as long it takes. CVO keeps retrying always...
Also, any idea why the CI isn't pulling pod logs?
hmm, do you have any particular one?
from cluster-openshift-apiserver-operator.
hmm, do you have any particular one?
the one linked from the description. https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_openshift-ansible/10695/pull-ci-openshift-openshift-ansible-master-e2e-gcp/1078
from cluster-openshift-apiserver-operator.
hmm, do you have any particular one?
the one linked from the description. https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_openshift-ansible/10695/pull-ci-openshift-openshift-ansible-master-e2e-gcp/1078
This is ansible ci job, i have no clue how it s setup. 😇
from cluster-openshift-apiserver-operator.
Also, any idea why the CI isn't pulling pod logs?
Investigated this with @eparis in Slack.
CVO starts operators.
apiserver gets stuck on Generation: expected 3, actual: 5
or similar
CVO is paused and never calls machine-approver operator
node csrs are stuck in Pending and any oc log
call returns tls error
. As a result we can get events, pods list and nodes - but not the logs for pods
from cluster-openshift-apiserver-operator.
$ oc get clusteroperator -o yaml openshift-cluster-kube-apiserver-operator
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
creationTimestamp: 2018-12-12T15:18:13Z
generation: 1
name: openshift-cluster-kube-apiserver-operator
resourceVersion: "97258"
selfLink: /apis/config.openshift.io/v1/clusteroperators/openshift-cluster-kube-apiserver-operator
uid: 2451d21c-fe21-11e8-a512-021a424c72ca
spec: {}
status:
conditions:
- lastTransitionTime: 2018-12-12T19:01:09Z
status: "False"
type: Failing
- lastTransitionTime: 2018-12-12T19:01:09Z
message: 3 of 3 nodes are at revision 1
status: "True"
type: Available
- lastTransitionTime: 2018-12-12T19:01:09Z
message: 0 of 3 nodes are at revision 1
reason: AllNodesAtLatestRevision
status: "False"
type: Progressing
extension: null
version: ""
Somehow both 0/3 and 3/3 nodes were at revision 1 at the same time. Hiesenburg!!!
from cluster-openshift-apiserver-operator.
I waited at least 4 hours for it to settle (and this is how it settled)
from cluster-openshift-apiserver-operator.
Related Issues (14)
- operator does not upgrade apiserver daemonset HOT 2
- Future Release Branches Frozen For Merging | branch:release-4.5 branch:release-4.4
- Future Release Branches Frozen For Merging | branch:release-4.18 branch:release-4.19 HOT 2
- update imageregister defaultRoute will make openshift api restart. HOT 7
- Will openshift-apiserver-operator keep openshift-apiserver unchanged HOT 6
- How to configure custom Audit Policy? HOT 2
- error synchronizing: clusterrole.rbac.authorization.k8s.io "aggregated-apiserver-clusterrole" not found HOT 9
- Return error on RunOperator context done HOT 5
- Access to a privileged container allows for breakout to the underlying host HOT 5
- Future Release Branches Frozen For Merging | branch:release-4.16 branch:release-4.17
- Apiserver startup takes 50+ seconds HOT 6
- Move TestingEventRecorder to a subpackage under spot where the in memory one is HOT 1
- Need invariant checker for all e2e tests HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cluster-openshift-apiserver-operator.