Append a new section named terraform-docs into each experiment's README.md.

The content will include select output from

terraform-docs markdown .

See https://github.com/terraform-docs/terraform-docs.

Should make it more clear what are the required and optional inputs and what are the outputs (if any) from each experiment.

Some charts have dependencies on MySQL or Postgres.

Consider delegating to pre-configured modules in Terraform docs or registry, like:

• AWS

  • https://www.terraform.io/docs/providers/aws/r/rds_cluster_instance.html
  • https://registry.terraform.io/modules/terraform-aws-modules/rds-aurora/aws/2.19.0

• Azure

  • https://www.terraform.io/docs/providers/azurerm/r/mysql_database.html
  • https://www.terraform.io/docs/providers/azurerm/r/postgresql_database.html

• Google

  • https://registry.terraform.io/modules/GoogleCloudPlatform/sql-db/google/3.2.0
  • https://www.terraform.io/docs/providers/google/r/sql_database_instance.html

There's a number of great articles and tutorials. How could we codify the steps and drive and installation through the same interface of bash + Terraform delegating to appropriate tools?

For reference:

• https://tanzu.vmware.com/content/practitioners/a-closer-look-at-vmware-tanzu-kubernetes-grid-multitenant-setup?src=so_5a314d05e49f5&cid=70134000001SkJn
• https://theithollow.com/2020/04/06/deploying-tanzu-kubernetes-grid-management-clusters-vsphere/
• https://www.virtuallyghetto.com/2020/03/sneak-peak-at-deploying-tanzu-kubernetes-grid-plus-on-vsphere-vmware-cloud-on-aws.html

So there's some solid work to make it easier to install the CSB, here: https://github.com/warroyo/csb-k8s.

Considering that CSB is going to be useful as the managed-services back-end for cf marketplace in cf4k8s or tas4k8s, we might opt for installing CSB with cf push using the manifest.yml in https://github.com/pivotal/cloud-service-broker.

Using a combo of Terraform and bash scripting we might:

• download the prebuilt artifacts for the cloud-service-broker and *-brokerpak(s) from releases (as described for example, here)
• opt for use of cloud native buildpacks, so use a modified form of the manifest.yml without the explicit reference to the legacy go-buildpack.
• provide a template config.tpl that includes all variables place-holdered configuration - use Terraform to produce a rendered form in a file named config.yml
• execute cf cups -p config.yml
• cf push the app, bind the user-provided-service instance and register the broker (as described, here)

On cf api, seeing

❯ cf api api.cf.lab.ironleg.me
Setting api endpoint to api.cf.lab.ironleg.me...
SSL Certificate Error x509: certificate is valid for *.cf-system.svc.cluster.local, not api.cf.lab.ironleg.me

then once authenticated and on cf push attempt seeing this in pod container logs for log-cache

[METRICS] 2020/05/20 20:37:01 Metrics endpoint is listening on [::]:6063
[GATEWAY] 2020/05/20 20:37:01 listening on [::]:8081...
2020/05/20 20:58:12.049713 http: TLS handshake error from 127.0.0.1:57338: remote error: tls: bad certificate
2020/05/20 20:58:12.339371 http: TLS handshake error from 127.0.0.1:57344: remote error: tls: bad certificate

More than likely need to adjust the tls_cert_request in modules/cf4k8s/tls.tf

@see https://www.terraform.io/docs/providers/tls/r/cert_request.html

See https://docs.eduk8s.io/en/latest/getting-started/installing-eduk8s.html.

We could use k14s kapp to deploy 1-n workshop definitions then deploy a portal for 1-n participants.

Later on, and more aspirationally, we could turn this repo into a workshop itself.

The installation process has been updated and there's a new cli. Update the experiments scripts and the scripts in the bom directory.

Gitlab has a helm chart, here: https://docs.gitlab.com/charts/. Seems like a rather straight-forward affair. Make sure we follow recommendations around resource utilization. More ambitiously integrate Postgres, Minio, Prometheus and Grafana.

Currently unable to issue valid certificates for Harbor.

Harbor can be deployed successfully, but user will be challenged by the web browser.

Checking to see if certificate is ready, shows us that there's definitely something amiss.

❯ kubectl get certificates -A -o wide
NAMESPACE   NAME          READY   SECRET              ISSUER             STATUS                                                               AGE
harbor      harbor-cert   False   harbor-tls-secret   letsencrypt-prod   Waiting for CertificateRequest "harbor-cert-682570108" to complete   19m

See https://github.com/keel-hq/keel#quick-start

There are a couple operators:

Confluent
https://docs.confluent.io/current/installation/operator/co-deployment.html

Strimzi
https://strimzi.io/

See https://docs.pivotal.io/rabbitmq-kubernetes/0-7/installing-with-helm.html

So there's this chart: https://github.com/concourse/concourse-chart. Handle secrets, ingress, persistence and credentials management.

Have a look at Vault, here: https://www.vaultproject.io/docs/platform/k8s/helm/run. Setup Vault in HA mode.
Auto-unsealing requires a dependency on a KMS.

• GCP - https://learn.hashicorp.com/vault/day-one/autounseal-gcp-kms
• Azure - https://learn.hashicorp.com/vault/day-one/autounseal-azure-keyvault
• AWS - https://learn.hashicorp.com/vault/day-one/ops-autounseal-aws-kms

Maybe start with Helm chart for OPA, https://github.com/helm/charts/tree/master/stable/opa.

We've a number of tools for the job.

• JenkinsX
• Concourse
• ArgoCD
• Tekton

Pick from above and author pipelines that work with a Git repository, build and unit test an artifact, assemble and publish a container image into an artifact repository, then deploy and/or promote the image as a container into one or more runtime environments.

Contemplate targeting runtime environments that:

a) are Kubernetes native
b) provide an abstraction with cf4k8s or tas4k8s

Could we tie in Tanzu Build Service with (a)?

What other value-adds come into play? Flagger?

Should allow for some basic templating and configuration.

Add two cloud provider and one local option (as alternatives to Harbor).

JFrog Container Registry

• https://github.com/jfrog/charts/tree/master/stable/artifactory-jcr

ACR

• https://www.terraform.io/docs/providers/azurerm/r/container_registry.html

GCR

• https://www.terraform.io/docs/providers/google/r/container_registry.html

Build out module and experiments support for Container Services Manager (KSM).

Note: there's a nice article about getting it installed expediently, here: http://www.clue2solve.io/tanzu/2020/07/14/install-ksm-and-configure-the-cf-marketplace.html.

See https://learn.hashicorp.com/vault/kubernetes/sidecar. Overall this will result in a more secure footprint and allow for previously deployed applications to be patched.

There's some interesting work here because if we want to demonstrate an HA vault setup, we have some additional effort beyond a "dev mode" setup.

Take a look at these additional articles and videos for inspiration:

• https://blog.doit-intl.com/vault-high-availability-on-gke-68ef4fd7ca33
• https://medium.com/@amimahloof/automating-vault-secrets-consumption-in-kubernetes-via-mutation-webhook-eea1864d40b5
• https://www.hashicorp.com/resources/terraform-your-deployment-of-vault-on-kubernetes/

This work will support #10.

Add a README.md into each module.

The content should explain the purpose for and how to use the module. Ideally, we would explain how to consume the module without having to clone the entire Git repository, rather just the module itself.

We should also add a terraform-docs section to make it clear what are the providers, inputs, and outputs. (See https://github.com/terraform-docs/terraform-docs).

Optional features should be able to be activated

Example: config-optional/disaster-recovery.yml for backup and restore

Some amazing effort and execution by @honnuanand has resulted in a nice Terraform demo of TKG used to bootstrap management and workload clusters on AWS, here: https://gitlab.eng.vmware.com/raoan/tkg-playground (private repository).

Would like to replicate and refine that work here while automating retrieval of CLI (if possible).

Harbor and possibly other destroy scripts do not delete the TXT entries.

Expected Outcome:
TXT entry txtharbor.domain. and txtnotary.domain. entries do not exist after running destroy scripts.

Actual Outcome:
These entries persist after running destroy scripts.

Calico and Antrea are solid options.

Adapt kubectl apply installation by perhaps using kapp Terraform provider to install. Use in concert with bash script to scale or delete pods that may have been created prior to installation.

Calico

• AKS
• EKS
• GKE

Antrea

• EKS
• GKE