This would reinforce to users that this repo is not the entirety of "the Selkies project/platform", and it would make writing/reading/talking about this repo less ambiguous.

The TURN components have been externalized to the selkies-gstreamer repo and will be removed from the core repo.

This includes removing:

• coturn images
• coturn infra deployments
• coturn manifests

This is the time to do this. Selkies has diverged from a GCP-centric affiliate project to a much wider audience.
Other Kubernetes providers including Vast.ai, RunPod, and CoreWeave are watching and incorporating our projects.

The GCE proxy is used by the vdi-vm example to create a localhost proxy that uses the GCE VM service account credentials to transparently authenticate to the pod broker.

This should be moved to a top-level selkies-project repo so it can be maintained separately and should remove most/all of the dependabot warnings.

TODO:

• Copy images/gce-proxy to new repo in selkies-project
• Remove cloud builder
• Remove any other references in this repo.
• Create GH workflow to automate image building.
• Update selkies-examples/vdi-vm if necessary.

Brand new GCP project, after I run:

$ ACCOUNT=$(gcloud config get-value account) && \
  gcloud builds submit \
    --project=${PROJECT_ID?} \
    --substitutions=_USER=${ACCOUNT?},_REGION=${REGION?}

I get the following error:

Step #10 - "deploy-cluster-manifests-region": Step #1 - "deploy-manifests": error: unable to recognize "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
Step #10 - "deploy-cluster-manifests-region": Step #1 - "deploy-manifests": Error: failed to apply deployment: failed to apply CustomResourceDefinition configuration file with name "istiocontrolplanes.install.istio.io" to cluster: failed to apply config from string: command to apply kubernetes config from string to cluster failed: exit status 1

I think the error is at https://github.com/selkies-project/selkies/blob/master/setup/manifests/deploy.sh:

log_cyan "Installing CRDs"
gke-deploy apply --project ${PROJECT_ID} --cluster ${CLUSTER_NAME} --location ${CLUSTER_LOCATION} --filename /opt/istio-operator/deploy/crds/istio_v1alpha2_istiocontrolplane_crd.yaml
gke-deploy apply --project ${PROJECT_ID} --cluster ${CLUSTER_NAME} --location ${CLUSTER_LOCATION} --filename base/pod-broker/crd.yaml

But I'm no expert. Will continue debugging this.

I'm planning to delete every branch other than master, since they're all either behind (pull-secrets, dev), abandoned (app-launcher-pwa-1, dependabot/npm_and_yarn/images/gce-proxy/src/http-proxy-1.18.1), or replaced by a tag (v1.0.0). Do we know of any production/live environments that depend on those branches?

Really, this applies to any registry, but I only care about ours 😉

In particular, here, it would be great to set a unique user agent so we can differentiate this traffic from any other random golang binary.

If you end up using my library (as suggested in #2), you can accomplish this with google.WithUserAgent and remote.WithUserAgent.

main is becoming the new default (already has on GitHub), and now seems like a good time, since we're already de-forking and revisiting our branching workflow.

I've already pushed a main branch at the master commit, so if we want to be cautious with this, we could

1. switch the "default branch" on GitHub (for cloning & PRs) to main
2. manually fast-forward master to meet main as PRs merge
3. slowly update production references from master to main
4. delete master when it feels safe

without needing to update all production references from master to main right away.

Remove dependency on fixed istio releases in favor of Managed ASM release channel.

Some things to keep in mind:

• support for EnvoyFilters to preserve OPA addon support.

This will reduce the manual dependencies and deployment complexity.

You're calling list tags in order to determine the digest of an image here.

It is much cheaper for the registry to serve a simpler HEAD /v2/<repo>/manifests/<tag> request instead of producing the entire tags list response.

In some situations (e.g. if you need to resolve 100 tags in a single repository), it might be cheaper to do this by listing tags, but you don't appear to be deduping these findImageTags calls by their repository. You could cache the response by repo and re-use that for subsequent checks, but I would guess that the HEAD calls will average out to be cheaper in almost any circumstance.

I maintain a library that does most of the things you're doing, e.g. google.List, google.NewEnvAuthenticator, and remote.Head should cover most of it.

This repo has an outdated reference to the App Launcher web interface in images/web and needs to be replaced with the submodule: https://github.com/selkies-project/selkies-app-launcher-web

You are polling in a loop trying to discover new tags every 10s, which is less than ideal.

If possible, it would be great if you could use pubsub instead of polling.

At least, it would be great to set this polling interval to something less aggressive (is 5 minutes too slow?) and configurable, with some minimum. Anything more frequent than ~60 seconds becomes ~abusive at scale.

After the web and coturn cleanup is done (blocked by #24 and #25) build a workflow to build all of the images and push them to GHCR with versions.

Follow the same build philosophy as the selkies-gstreamer repo:
https://github.com/selkies-project/selkies-gstreamer/tree/master/.github/workflows

Preserve the cloudbuild image building to allow folks to build and self-host images.

Strategy:

• master and dev branches are built when pushed to, images are tagged with the branch name respectively.
• Pushing a tag builds the images, creates a release and tags the images with latest

Instead of creating the GCLB, Managed Certs and AutoNEG controller for the ingress tier, create the GCLB using the new Gateway API.

This will reduce the manual dependencies and deployment complexity.

Depends on #34

Convert all image references in base yamls and kustomizations to use the GHCR images.

• base yamls should reference the ghcr.io image with the latest tag.
• kustomization image patches should point to a specific tagged version.
• update cloud build pipelines, build scripts etc to parameterize passing a target version (tag).

Is master currently stable? If so, I think we should release what we currently have as v1.1.0 or v2.0.0, depending on whether it contains breaking changes from v1.0.0. My motivation is that we haven't "released" since v1.0.0 (17 months ago), and I think we should aim to be on a new-minor-version-every-month-or-two release cycle this year. Additionally, with a recent release we could re-pin our quick start to a version, instead of pointing it at live master.

I created a v1.1.0 tag & pre-release as a preview of what this release could look like, but I'm happy to delete those.

What do you think?

While setting up a new GCP project I got the following error:

Step #4 - "deploy-infra-base": Step #1 - "terraform-apply": Error: Error creating Topic: googleapi: Error 409: Resource already exists in the project (resource=gcr).
Step #4 - "deploy-infra-base": Step #1 - "terraform-apply":
Step #4 - "deploy-infra-base": Step #1 - "terraform-apply":   on gcr.tf line 17, in resource "google_pubsub_topic" "gcr":
Step #4 - "deploy-infra-base": Step #1 - "terraform-apply":   17: resource "google_pubsub_topic" "gcr" {
Step #4 - "deploy-infra-base": Step #1 - "terraform-apply":
Step #4 - "deploy-infra-base": Step #1 - "terraform-apply":
Step #4 - "deploy-infra-base": Step #1 - "terraform-apply":

Not sure why Terraform complains about the resource already existing? Shouldn't Terraform make sure the configured state is reflected in the GCP project?

After a successful deployment I try to access https://broker.endpoints.<PROJECT_ID>.cloud.goog/ and I get the following message:

Error: Bad Request
Your client has issued a malformed or illegal request.

This is error HTTP 400, so it's a client error? What am I doing wrong here?

Per the docs: