Comments (3)
Had some good insight from onedr0p in this thread, and did some experimentation trying an approach with and without kustomization.yaml
s everywhere:
inside ./kube-system
,
external-secrets
├── 1password
│ ├── 1password.yaml
│ ├── clustersecretstore.yaml
│ └── kustomization.yaml
├── chart
│ ├── external-secrets.yaml
│ └── kustomization.yaml
├── ks.yaml
└── kustomization.yaml
... works as intended/expected where the ks.yaml
is a flux kustomization with proper dependency settings.
When trying the same thing without the (IMO redundant kustomization.yamls
),
external-secrets
├── 1password
│ ├── 1password.yaml
│ └── clustersecretstore.yaml
├── chart
│ └── external-secrets.yam
└── ks.yaml
The flux apply ordering is not applied properly. This suggests that having kustomization.yaml
files is a hard requirement.
from k8s-gitops.
Over in the flux slack, Kingdon B provided some additional insight (thread).
He noted that, in my case here, there is still the top-level flux kustomization referencing the repo root and as such there are two different flux kustomization's applying both on the external-secrets
directory. This is why things break without the presence of the kustomize.yaml
files, and this makes logical sense.
Given this information, there are likely three approaches to leveraging the flux ordering (dependsOn
) features:
- Ensure that there are
kustomize.yaml
files everywhere - or at least everywhere in trees where you need to control the ordering. This probably really means everywhere though. It appears that most of the k8s-at-home folks have either submitted that this is the way forward or simply copied from each-other this pattern - Better control/organization of top-level directories. If the 'root' of flux is going to point to the repo root (as is currently the case in this repo). Perhaps following the example from onedr0p and others where there is some sort of 'infrastructure' tree and 'applications' tree. Things that make CRDs would go in 'infrastructure' (e.g. cert manager, external secrets, prometheus CRDs, rook CRDs?, velero, etc) and everything else would go in 'applications'
- Maybe some sort of top-level flux kustomization that specifically calls-out certain components that need be installed first, while still maintaining a flat directory structure - not sure if this is even possible
Something about the need to reflect the actual directory/file structure in separate (kustomization.yaml
) really rubs me the wrong way and I would love to find a way around this problem without 'junking up' the repo with this cruft.
Will do deeper experiments with the new test cluster.
from k8s-gitops.
Another option that would avoid the necessity to let flux try to handle dependencies all-together, and resulting ugly hacks (IMO), would be to include required CRDs as part of the initial bootstrap process. This would be an extra one-time activity at cluster creation vs long-lasting poor structure.
Initially, CRDs that would be necessary to be in-place before flux installation would be:
- Prometheus: Lots of different servicemonitors and other prometheus-style things are invoked all over the cluster
- external-secrets: Anything using secrets needs this and are everywhere
- volsync: similar to secrets, there are a lot of references spread throughout the cluster
- cert-manager: This is relatively self-contained and I wonder if it's possible to handle bootstrapping within the cert-manager directory tree
Implementation would look like an additional set of steps to apply CRDs at cluster setup time, with extra reference (probably a single kustomization file) with reference to the version of externalized CRDs that could be revved automatically via mend.
from k8s-gitops.
Related Issues (20)
- yeet k8s-at-home charts HOT 1
- implement 1Password connect HOT 1
- leverage 1Password for secrets
- experiment with talos or kairos HOT 1
- Enable rook/ceph object storage
- Migrate loki to ceph object storage
- Consider leveraging vector instead of promtail
- Migrate thanos to ceph object storage
- Explore draino HOT 3
- rook/ceph OSD repaving incident on 2023-02-22 HOT 1
- Renovate (Mend) Dashboard
- Incorrect ingress in echo-server
- experiment with NixOS HOT 6
- Duplicate key in kube-prometheus-stack chart values HOT 1
- upgrade to k8s 1.20 HOT 2
- cert-manager crds race condition HOT 5
- Migrate from minio to minio-operator HOT 1
- Migrate from zwave2mqtt to zwavejs2mqtt HOT 1
- explore sops HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from k8s-gitops.