Details

https://github.com/cilium/cilium

Details

Ensure that the --anonymous-auth argument is set to false

Describe the solution you'd like:
Audit:

ps -ef | grep kube-apiserver

Verify that the --anonymous-auth argument is set to false.

Anything else you would like to add:

Remediation:
Edit the API server pod specification file /etc/kubernetes/manifests/kube- apiserver.yaml on the master node and set the below parameter.

--anonymous-auth=false

Details

https://istio.io/latest/docs/ops/best-practices/security/#configure-third-party-service-account-tokens

2020-07-23T03:55:07.799195Z	info	Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details.

Details

Currently, git-crypt is being used within the repository and works nicely, however, there is no visibility into the secret structure of the resource so users who fork the repo don't know what key pair values are required in these secret objects without having to inspect each sealed secret respectively.

SOPs provides encryption similar to git-crypt whilst allowing the structure to be viewable and providing a DIFF in the pull request process.

Details

Describe the solution you'd like:

Ensure that the --basic-auth-file argument is not set

Anything else you would like to add:

Audit:
Run the following command on the master node:

ps -ef | grep kube-apiserver

Verify that the --basic-auth-file argument does not exist.

Remediation:

Follow the documentation and configure alternate mechanisms for authentication. Then, edit the API server pod specification file /etc/kubernetes/manifests/kube- apiserver.yaml on the master node and remove the --basic-auth-file= parameter.

Details

Using vagrant and VirtualBox so that anyone should be able to spin up a cluster

Details

oauth2 proxy is currently setup with github and would be good to also setup with keycloak internal to the cluster.

https://github.com/keycloak/keycloak

https://github.com/codecentric/helm-charts

The image is currently not supported for arm64

Details

Implement istio to allow ingress to the cluster and eventually phase out using nginx ingress. Some caveats may be oauth2 and keycloak integration not readily available in istio natively and may require further investigation - might run istio on a subdomain as a proof of concept before migrating completely across.

Details

Create a terraform module so that users will be able to spin up clusters and get involved in the project.

Details

In istio 1.8.x add-on installation via istioctl will no longer be embedded and available as they're deprecated in 1.7.x and removed in 1.8.x

These services will need to be deployed via their respective deployment methods as standalone workloads

https://istio.io/latest/blog/2020/addon-rework/#timeline

Details

Either provide an SSH key which is encrypted in the repo and applied via Ansible or similar approach with Git over HTTPS

https://docs.fluxcd.io/en/1.18.0/guides/provide-own-ssh-key.html
https://docs.fluxcd.io/en/1.18.0/guides/use-git-https.html

Details

Due to not having a wildcard CNAME currently, all CNAMEs are manually created in the cloudflare console, ideally though whenever either an ingress or equivalent object is created the CNAMEs should be updated to include the new object automatically and likewise once an object is removed it should no longer persist as a CNAME in cloudflare.

EDIT** Wildcard CNAME records were available previously within cloudflares initial offering, however, it has since become an enterprise function which cost is unjustifiable for a homelab.

Details

A workflow to update the git sub-module should be setup to automatically promote new policy releases into the repository. Currently, this is a manual process to keep it in parity with the upstream repository.

Details

Store loki logs remotely in GCP

https://github.com/grafana/loki/blob/master/docs/sources/configuration/examples.md#google-cloud-storage

Details

Velero operator has been configured in the cluster, backing up resources & snapshots. Create playbook to that tests the recovery scenario.

Details

Setup Dashboard for Fluxv2 (Weave Flux is not in use/outdated).

Details

Moving to Fluxv2 will require a massive uplift.

Currently, there is still a dependency on upstream providing arm64 images, and a means to deploy using those images as GHCR doesn't support multi-arch images and therefore images are appended with -arch EG: -arm64 also the default installation using gotk sets amd64 node affinity.

fluxcd/flux2#194

Details

Describe the solution you'd like:

Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)

Anything else you would like to add:

Audit:
Run the below command (based on the file location on your system) on the master node. For example:

stat -c %a /etc/kubernetes/manifests/kube-apiserver.yaml

Verify that the permissions are 644 or more restrictive.

Remediation:
Run the below command (based on the file location on your system) on the master node. For example:

chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml

Details

Adopt chaos engineering principles by leveraging litmus. This would be a fantastic way to test and validate the clusters' resilience and recovery with configurable levels of disruption.

Currently pending arm64 support: litmuschaos/litmus#1550

Details

Configure prometheus operator to run using Thanos

https://github.com/helm/charts/tree/master/stable/prometheus-operator

Details

Migrate master branch to the main branch; GHA action workflows and repo.yaml will need references updated.

Details

Describe the solution you'd like:

When the upstream helm charts or maintainers provide a new official release the helm-operator should be able to be templated via CI and create a PR with the proposed upgrade within the repository.

Anything else you would like to add:

It could even be possible that the particular PR is using Flagger to test the upgrade before complete rollout

Additional Information:
N/A

Details

I would like to have my fully hydrated k8s resource manifests checked directly into source code rather than having helm-operator to the template and apply the manifests in-cluster. It also would make version controlling potential breaking changes and making resource changes more clear to observe during the PR process rather than introducing breaking changes directly in-cluster.

https://github.com/microsoft/fabrikate

Details

Implement an auto-release mechanism that can publish releases with changes that have been deployed into the cluster, this would also be handy to version pin the cluster to a given SHA and allowing newly created functionality continue to be merged into master/main branch.

One such tool that could help facilitate this functionality could be https://github.com/marketplace/actions/release-drafter

Details

Network policy currently allows all traffic, these need to be updated to allow isolated network traffic between namespaces and workloads respectively within the internal cluster network traffic.

Where feasible and not overlapping with istio authorization policies, global network policies provided via either Calico or Cilium should be enforced to provide L7 network controls.

Details

2 etcd

• 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate
• 2.2 Ensure that the --client-cert-auth argument is set to true
• 2.3 Ensure that the --auto-tls argument is not set to true
• 2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate
• 2.5 Ensure that the --peer-client-cert-auth argument is set to true
• 2.6 Ensure that the --peer-auto-tls argument is not set to true
• 2.7 Ensure that a unique Certificate Authority is used for etcd

Details

https://github.com/google/gvisor

Note: Currently supports ARM64

5.4 Secrets Management

• 5.4.1 Prefer using secrets as files over secrets as environment variables
• 5.4.2 Consider external secret storage

5.5 Extensible Admission Control

• 5.5.1 Configure Image Provenance using ImagePolicyWebhook admission controller

Details

Describe the solution you'd like:

Enable the ability to canary out certain functionality and changes using a tool such as Flagger

https://github.com/weaveworks/flagger

Anything else you would like to add:

N/A

Additional Information:

N/A

Details

The following are resources within GCP that are required dependencies within this project:

• Velero:
  • GCS
  • GSA
  • Custom Role
• Thanos:
  • GCS
  • GSA
  • Custom Role
• SOPS
  • KMS
  • GSA
  • Custom Role

This would also include things such as enabling required APIs and can further expand into other cloud equivalent services.

Details

Implement a service monitor for cert-manager and configure the Grafana dashboard. This may be possible within the chart itself but needs further investigation

Dashboard examples:

• https://grafana.com/grafana/dashboards/11001
• https://grafana.com/grafana/dashboards/12170

Details

We should be able to run authorization filter for L7 (Envoy) proxies within the mesh to check if the incoming request is authorized or not.

More info for decision making https://blog.openpolicyagent.org/envoy-external-authorization-with-opa-578213ed567c

Details

TBC

Details

Rather than users required to use basic auth to login to the grafana instance use github auth to allow authenticated logins.

helm/charts#22473

Currently pending the above issue being implemented otherwise may need to consider another approach.

https://github.com/raspbernetes/k8s-gitops/blob/ef405eb40bab75e3d826c6f2a8a69de3de4fb7a2/namespaces/observability/prometheus-operator/prometheus-operator.yaml#L115

Details

What steps did you take and what happened:

Attempted to deploy helm-operator and its required CRD's however Fluxcd did not apply the resources?

What did you expect to happen:

Fluxcd should be able to deploy and manage the helm-operator within the cluster.

Anything else you would like to add:

Note: Miscellaneous information that will assist in solving the issue.

Additional Information:

Note: Anything to give further context to the bug report.

Details

https://github.com/wehkamp/docker-prometheus-cloudflare-exporter

Details

The existing helm chart is being deprecated and has moved to the new chart location.

Old: https://github.com/helm/charts/tree/master/stable/prometheus-operator
New: https://github.com/prometheus-community/helm-charts

Details

Services such as Prometheus don't have native login oauth2 integration, this can be provided using envoy filters similar to the kiali envoy filter to enforce traffic is authenticated prior to hitting the downstream service via the virtual service resource.

Details

Configure alert manager with some sample/basic rule sets.

Details

Currently, istio has no active workloads deployed into the mesh, this issue would require that most services be integrated and tested whilst running in the mesh using mTLS and no direct external internet facing traffic other than via the egress gateway.

Namespaces to add to the mesh:

• actions-runner-system
• backups
• flux-system
• gatekeeper-system
• home-system
• istio-operator - Should not be added to the mesh
• istio-system - Should not be added to the mesh
• kube-system - Should not be added to the mesh
• litmus
• network
• observability
• openebs
• security

Details

The sealed secret operator pod starts will generate a new private key if one does not exist within the cluster. Currently, the private key that is used to decrypt all existing secrets is stored in git-crypt and manually applied post-setup of Flux in the cluster using the install/sealed-secret.sh script.

Ideally, Flux being integrated with git-crypt means Flux will have the capability to check out the private key within the cluster and apply it without requiring manual intervention after the operator has already initialized it.

https://github.com/bitnami-labs/sealed-secrets#managing-existing-secrets

Details

Deploying Gatekeeper into the cluster with the CIS benchmark policies written in the raspbernetes/k8s-security-policies repository.

Details

Describe the solution you'd like:

Sealed secrets must be encrypted with the appropriate public key and information to be decrypted in the cluster. These can cause issues when not valid or have been misconfigured, testing should be done to ensure these are not invalidated due to the before mentioned reasons.

To accomplish this there is a kubeseal --validate command that will --dry-run a decryption against the running sealed secret controller to ensure it is able to decrypt appropriately, if the resources is not valid it will prompt the following error output error: unable to decrypt sealed secret

Details

No user currently has admin permissions to Grafana. Users logged in via oauth2 should be made admins.

Details

Ensure that the --anonymous-auth argument is set to false

Describe the solution you'd like:
Audit:

ps -ef | grep kube-apiserver

Verify that the --anonymous-auth argument is set to false.

Anything else you would like to add:

Remediation:
Edit the API server pod specification file /etc/kubernetes/manifests/kube- apiserver.yaml on the master node and set the below parameter.

--anonymous-auth=false

Details

v3.16.x Calico eBPF support went GA.

Might need to investigate BGP routing in place of using metallb #75

https://www.projectcalico.org/whats-new-in-calico-3-16/

Details

Update the submodule with the correct repository link, the repository was renamed and Github will continue to redirect the link until a new repository with the same name collides with it in which it will no longer work.

This should be updated so there is a consistent behavior.

Details

Revisit the list of currently installed dashboard and remove the ones that are outdated or not in use e.g. Weave Flux

New Chart location: https://github.com/bitnami/charts/tree/master/bitnami/metallb

https://github.com/raspbernetes/k8s-gitops/blob/ef405eb40bab75e3d826c6f2a8a69de3de4fb7a2/namespaces/network/metallb/metallb.yaml#L20