Yesterday, building for www took ~30min after Travis finished, and there was just a generic “could not connect” screen looking at the page.

It’d be nice to have a sense of “it’s building,” or “it erred,” or “awaiting build from Travis…” just some insight as to what’s happening. Bonus points if it could even display realtime build logs like Netlify.

Heighliner only listens for github releases. We should support listening for regular tags as well.

Perhaps as a config option?

I have this defined:

apiVersion: hlnr.io/v1alpha1
kind: ImagePolicy
metadata:
  name: hlnr-test-www
  namespace: websites
spec:
  image: luizbranco/hlnr-test
  versioningPolicy:
    name: release-major

but no Versioning Policy matches that name. Would it be possible for the image controller to at least warning about that? Right now it's radio silent when a releases comes in.

Instead of having to create a githubrepository by hand for each repository, we should have some api object and controller to watch image policies, and create a new githubrepository whenever one references a new repo that is under the api object's org, and share the secret configuration with it.

We made some assumptions about what your k8s cluster setup might look like when making Heighliner. We did this in order to first make it work for us, and second get it out to the world as quickly as possible. Now we want to make it work for different k8s setups. This issue is an effort to track your suggestions and commentary on why the Heighliner installation as it is might not work for you.

The issue is that you need to pull down the image first, then it needs to install the CRD, and then you can apply it. We should just add a YAML manifest to install it when installing the controllers

Rolling Updates

Currently, we're not doing rolling updates nicely. We cut off as soon as a new
release is available in the Microservice, but that doesn't mean it's actually
available yet (boot times etc.)

We should take several steps to make this a proper rolling update.

UpdateStrategy

We'll want to move the UpdateStrategy from the NetworkPolicy to the
Microservice. The main motivator for this is that we will eventually also have
non HTTP applications running through Heighliner like background workers and
temporary jobs.

From an availability point of view, the Microservice is the best point to
determine which release should actually be marked as the active release. In an
HTTP scenarion, this means that this is the release to which the traffic should
lead to.

Minimum Available

The UpdateStrategy should have a MinimumAvailable field, allowing the user to
specify how many versions they want to keep alive. This includes the active
version.

For rolling upgrades, if the version is set to 1, we will not delete the older
active version until the new version is in a state where we're happy with
availability.

The default value will be 3.

Strategy: latest

The latest strategy is fairly simple and doesn't currently take any arguments.
It will look at available images for the ReleaseGroup and deploy a new version
if one is available in the ImagePolicy. Once this new version is deployed and
available to serve requests, it will automatically be marked as active (see
more on statuses here). Other versions will be cycled accordingly.

Strategy: manual

manual takes priority over latest. Both can be set, but when manual is set
the cluster will choose this as the desired version to be marked as active
(see more on statuses here).

Microservice Changes

To reflect the UpdateStrategy, I propose we add the following to the
Microservice.Spec:

// MicroserviceSpec represents the specification for a Microservice. It houses
// all the policies which we'll use to build a VersionedMicroservice.
type MicroserviceSpec struct {
	// …

	// UpdateStrategy represents the way this Microservice will roll over from
	// one version to another.
	UpdateStrategy UpdateStrategy `json:"updateStrategy"`
}

We will keep using the existing UpdateStrategy:

type UpdateStrategy struct {
	Manual *ManualUpdateStrategy `json:"manual"`
	Latest *LatestUpdateStrategy `json:"latest"`
}

type ManualUpdateStrategy struct {
	SemVer *SemVerRelease `json:"semVer"`
}

type LatestUpdateStrategy struct{}

We also want to update the Microservice.Status data format. We want to
introduce the concept of ReleaseGroup. We internally already do this at the
NetworkPolicy level, but we should do this a level higher up so it's easily
visible to the rest of the cluster.

// MicroserviceStatus represents the status a specific Microservice is in.
type MicroserviceStatus struct {
	ReleaseGroups []ReleaseGroup `json:"releaseGroups"`
}

ReleaseGroup

A ReleaseGroup is a grouping of releases that fall within a specific release
level. For example, when a cluster is configured to monitor Preview releases,
a specific release group would be a single PR. We can have multiple PRs, each
PR would be it's own ReleaseGroup.

When we choose to use a ReleaseMinor versioning level, a release group could
be 1.2.x or it could be 1.3.x. All the versions within the x range would
then fall within that specific group.

// ReleaseGroup represents a group of releases that are tied together through
// their versioning configuration. A ReleaseGroup could be represented by a
// single Pull Request like `add-feature-x`, or it could be represented by a
// specific tag version, like `1.2.x`.
type ReleaseGroup struct {
	// Name represents the name fo the release group. For PRs this will be
	// represented by the Pull Request name. For (Pre)Releases this will be
	// represented by the filtered tag.
	Name string `json:"name"`

	// Releases represents the releases that are available for the specific
	// ReleaseGroup.
	Releases []Release `json:"release"`
}

Release

The Release itself will reflect a minor change as well. We'll add a Status
field to the release. We can have several Statuses:

• Active: the main active release, the release we actually want to use
• Canary: a temporary deployment which is active, but doesn't receive the main trafic
• Error: a temporary deployment which isn't active for some reason
• Deprecated: a release that will be removed shortly

type ReleaseStatus string

const (
	// ReleaseStatusActive refers to a release which receives the application's
	// main traffic.
	ReleaseStatusActive ReleaseStatus = "active"

	// ReleaseStatusCanary refers to a release which is healthy and receives the
	// application's test traffic.
	ReleaseStatusCanary ReleaseStatus = "canary"

	// ReleaseStatusError refers to a release which encountered an error whilst
	// trying to deploy.
	ReleaseStatusError ReleaseStatus = "error"

	// ReleaseStatusDeprecated referes to a release which will be removed
	// shortly.
	ReleaseStatusDreprecated ReleaseStatus = "deprecated"
)

// Release represents a specific release for a version of an image.
type Release struct {
	// …

	// Status represents the current status of this specific Release.
	Status ReleaseStatus `json:"status"`
}

NetworkPolicy Changes

In the NetworkPolicy.Spec, we'd remove the UpdateStrategy field and always
rely on the active release for a ReleaseGroup in the Microservice.Status.

We'll do what we do now and use the labels of Microservice Release where the
Status is set to Active. We'll then update the main Service to point to these
labels instead.

By doing this, we should achieve 0 downtime as the Microservice controller
only marks the Release Active once it's happy with it's availability.

The NetworkPolicy will now be simpler and just iterate over the ReleaseGroups.
We can now introduce a new Template Variable to the templating system for the
Domains as well, ReleaseGroupName.

We should add logs to indicate success so that it helps us troubleshoot heighliner when things are not happening as expected.

Some of these logs can be:

• succesful handling of GH webhooks n the GithubRepository controller
• sucessful syncing of DH Images and GH Tags in the ImagePolicyController
• successful deployments in the Microservice Controller/Versioned Microservice controller
• sucessful creation of ingress and services in the Networkpolicy controller