There are some issues with processes not being cleaned up if tye run doesn't gracefully shutdown. I hit this a few times when trying to close the running project, which failed to successfully start.

There is currently a PR in micronetes which does some process cleanup from a previous run: davidfowl/Micronetes#48. We can use this as a baseline.

Discovered while working on MVP. Hosting tries to calculate the name of the profile in launchSettings, so it's only going to work with the defaults. Currently it will silently fail (hosting will start, but the project won't be started).

System.Reactive isn't listed in eng/signing.props - probably because arcade auto-assigned a certain category to it. We should double check this is correct, because it's named like a BCL dll, but it's not from microsoft.

Both opulence and micronetes had great guides on getting started. These should be ported over and updated to work for all samples.

We should have up and down commands. This feels like "let me bootstrap or clean up my environment quickly" commands. We need to figure out where this fits with deploy though.

Getting validation completion for YAML in VSCode requires an extension but we could suggest using it. https://joshuaavalon.io/intellisense-json-yaml-vs-code

We could probably get pretty far by building a schema ourselves (pretty easy) and then putting a link to documentation in the generated template yaml.

I'm not sure if there's any support for intellisense for yaml in VS.

It would be nice to be able to do things like expose all services via the load balancer in a dev cluster, but use an ingress in production.

We should think more about what ideas make sense to make remote development/testing easier, but require different behavior from prod.

This is an entire feature area but right now the orchestrator only works for long running processes. Running one of these projects will start and restart it constantly.

We should have benchmarks in place to continuously improve on main scenarios we care about. Ones I can think of on top of my head include:

• tye init execution time
• tye run startup time
• tye deploy execution time, optimizing for key deployment scenarios.

• VS Code Commands to run tye commands
• Opens the tye dashboard in VSCode as part of tye run
• (?possible?) hyperlinks to attach the debugger from the dashboard when running inside VS code
• (?possible?) hyperlinks to run dotnet trace from the dashboard when running inside VS code

The following (simple) repro results in something we don't handle well.

• Create a new web project
• tye deploy -i

Because the web project has two ports in launchSettings.json we'll end up with two bindings, neither of them have a name. We have logic to autofill the binding name when adding ports to the k8s deployment, but since we have multiple unnamed ports we get a duplicate and it fails.

We have cases where we have to produce legal C# identifiers from arbitrary text, as well as producing DNS names from arbitrary text.

For instance the project file test-project.csproj can result in test_project in launchSettings.json. The reverse case would be problematics for DNS names (most k8s) stuff.

We should start thinking about cardinality as it results to kubernetes. We also want to have a concept of environments.

Is each application x environment a namespace?
Is each environment a namespace, and we try to make sure services don't collide?

We're already seeing collisions in building toy samples 😆

Tye has a bunch of things that are genuinely hard to test, or have dependencies on things that will make tests unreliable.

• Docker
• Container Registries
• Kubernetes
• Running multiple processes locally

We plan discuss and come up with a plan for what areas we're going to write real tests for and which we're going to simulate or ignore.

This would make it easier to do things like bookmark the dashboard. We can also make it configurable via command line (I think it is already) or the yaml.

Imagine:

• You are using hosted data (redis/sql/mongo/mq/whatever)
• You run locally with an instance in docker
• When you do a tye deploy you get prompted to enter the connection string (if not already saved)
• We will create the relevant kubernetes secrets for you

Tye run current hangs on shutdown. 4b6918c is the commit that regressed it. I'm currently investigating why.

The scope for the MVP is what we've outlined for public demos. Here it is in more details:

• remove opulence and micronetes branding

dotnet tool

• init (scaffold yaml config)
• run (local development)
• deploy (deploy to k8s)
• deploy supports config-based and interactive

Local development

• Multiple services
• Wiring up of bindings
• Dev-only dependencies
• Minimal dashboard so you can see what's running
• Can run .NET project, container, or arbitrary process

Publishing

• Generate Docker images based on autodetected project metadata (tfm, shared frameworks)
• Tag Docker images based on config (or interactive CLI)
• Publish images via scripting of 'docker push'
• Generate (hidden) k8s manifests based on bindings and project metadata
• Deployments/Services/Secrets (using ClusterIP)
• Deploy manifests via scripting 'kubectl apply'

Config

• Ability to autodetect config based on solution file (no config)
• Ability to define configuration with in yaml with:
  • Deployment related settings (whatever we need)
  • Services
  • Bindings

Testing

• E2E testing

Documentation

• Documentation for all of the above (on github)

As a follow up to #2, the testing strategy we come up with could be generally useful for people who want to run tye or want to test distributed applications. If we agree that wiring up distributed applications is hard, writing tests is as difficult of a problem as well.

Currently tye run relies on Bedrock.Framework and FeatherHttp. We should remove these dependencies if they aren't shipping products in the future.

Bedrock.Framework is only used for the proxying: https://github.com/dotnet/tye/blob/d0773bcd52f7ae050477714660ffce38b5850573/src/micronetes/Micronetes.Hosting/ProxyService.cs.

Feather is only used in the MicronetesHost.cs: https://github.com/dotnet/tye/blob/master/src/micronetes/Micronetes.Hosting/MicronetesHost.cs#L19

• tye init (with a few projects)
• tye run (with existing yaml, some projects and a container)
• tye deploy (with minikube)
• tye generate (as backup for tye deploy if it's too hard to test)

A few reasons why we might want to do this:

• We already have workarounds for YamlDotNet behavior (don't serialize empty collections)
• We care about generating comments for tye init
• We might want fine-grained control for error handling and versioning

The serialization support is fine for now, but if we have more stuff to fix in this area we might want to just throw it away.

We should have some.

I'm putting in a basic github actions based build so we have a minimal CI, but we should do something real.

Right now, when doing tye run, if you have multiple frontends, it won't log the frontend host and port. Also, the other frontend ports are still publicly available.

The idea behind this command is to enable sharing of tye manifests. tye pack would convert projects to container images and would put that in an archive (maybe a nuget package). tye.yml would also be extended to reference these manifests as dependencies for consumption. The idea is that I can share a set of services so that another team can depend on them.

Imagine the following:

Team B works on the backend API which relies on redis for caching.

# tye application configuration file
# read all about it at https://github.com/dotnet/tye
name: backend-services
services:
- name: backend
  project: backend\backend.csproj
- name: redis
  dockerImage: redis
  bindings:
    - port: 6379

They run the tye pack command and it produces a package that can be pushed to a feed to share the manifest. It also containerizes the backend project, pushes it to a registry (which maybe should be burned into the manifest)

tye pack
Building container image backend for backend\backend.csproj
Pushed docker image: 'davidfowl/backend:1.0.0'
Package 'backend\backend-services.1.0.0.nupkg'

Now team A (the front end team) can reference backend as a package in their tye manifest.

name: frontend
services:
- name: frontend
  project: frontend\frontend.csproj
- name: backend
  package: backend/1.0.0  # package reference

tye run will download this package from the feed (using feeds specified in nuget.config) and will grab the tye.yml file from that package and will run it as part of the services locally.

This seems to consistently repro when using ctrl+c to exit from tye run

If this is intended we should handle it, otherwise we should fix it 😆

[16:03:41 DBG] Failed to start the event pipe session
Microsoft.Diagnostics.NETCore.Client.ServerNotAvailableException: Process 1114 not running compatible .NET Core runtime.
   at Microsoft.Diagnostics.NETCore.Client.IpcClient.GetTransport(Int32 processId)
   at Microsoft.Diagnostics.NETCore.Client.IpcClient.SendMessage(Int32 processId, IpcMessage message, IpcMessage& response)
   at Microsoft.Diagnostics.NETCore.Client.EventPipeSession..ctor(Int32 processId, IEnumerable`1 providers, Boolean requestRundown, Int32 circularBufferMB)
   at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.StartEventPipeSession(IEnumerable`1 providers, Boolean requestRundown, Int32 circularBufferMB)
   at Micronetes.Hosting.Diagnostics.DiagnosticsCollector.ProcessEvents(String applicationName, String serviceName, Int32 processId, String replicaName, IDictionary`2 metrics, CancellationToken cancellationToken) in /Users/ryan/github.com/dotnet/tye/src/micronetes/Micronetes.Hosting.Diagnostics/DiagnosticsCollector.cs:line 193
[16:03:41 INF] Event pipe collection completed for test-project_1ac7e820-7 on process id 1114
[16:03:41 INF] test-project_1ac7e820-7 process exited with exit code 0

Some people have hate in their hearts for YAML. It's really cheap for us to support JSON as well as YAML, since YAML can be converted to YAML. https://github.com/aaubry/YamlDotNet/wiki/Samples.ConvertYamlToJson

Additionally, JSON has better support for schema/intellisense

We could do this by making JSON the source of truth and then converting YAML to JSON (always)

Currently you can interact with secret-based bindings via something like:

                .ConfigureAppConfiguration(config =>
                {
                    if (Directory.Exists("/var/tye/bindings/"))
                    {
                        foreach (var directory in Directory.GetDirectories("/var/tye/bindings/"))
                        {
                            Console.WriteLine($"Adding config in '{directory}'.");
                            config.AddKeyPerFile(directory, optional: true);
                        }
                    }
                })

This is ugly boilerplate that we don't want to put in every project, and it requires an extra package reference (key per file).

Using configuration via files is valuable because:

• more securities
• values can be updated

Consider making a first class custom resource for k8s with an operator to create the appropriate resources in the cluster. This would require an install (tye install) step to make the cluster type ready.

The single-phase docker build support from Opulence isn't working. This means we can't do things like p2ps 😢

I'd like to briefly discuss service discovery as well as a general design that may work. I created a PR #72 that would fix an issue with tye deploy, however the more I unwrap the issue, the more I think we need to setup a good design across the board, between tye init, tye run, and tye deploy.

Problem statement:

Today, when we setup service discovery between different services, we use environment variables in the shape of:

service:{name}:port
service:{name}:protocol
service:{name}:host

where name is the name of the service you would like to call. A URI can easily be created from this infromation, as seen here: https://github.com/dotnet/tye/blob/master/samples/multi-project/frontend/ConfigurationExtensions.cs#L9.

This all works fine and dandy when each service only has one binding, however this falls apart when multiple bindings are introduced.

For a single binding, our tye.yaml file would look like the following:

name: single-project
services:
- name: test-project
  project: test-project\test-project.csproj
  bindings:
  - port: 80

When getting each of the following values:

service:test-project:port = 80
service:test-project:protocol =  (default is http)
service:test-project:host =  (default is localhost for tye run, and the service name for tye deploy)

This will work all scenarios currently.

Now let's move to two bindings, tye.yaml could/would look like:

name: single-project
services:
- name: test-project
  project: test-project\test-project.csproj
  bindings:
  - port: 80
  - port: 443

Today, this actually works, and will create two bindings for the project. However, there are many issues with this today:

• tye run doesn't set environment variables for service discovery correctly, last one wins
• tye run doesn't know which protocol each port is using. To clarify, the protocol here would be http vs https, which we may want to discuss further in the future.
• tye bindings don't have a name here, so they can't be gotten by the convention of service:test-project:{name}:host

With 3 or more bindings, the problems get a bit more interesting, but let's focus on two for now.

Proposal

The first proposal I'd like to require is that if there are 2+ bindings, that there must be some way to reference them by name such that both are available. For now, I'd propose that we require a name for bindings if there are multiple, for example:

name: single-project
services:
- name: test-project
  project: test-project\test-project.csproj
  bindings:
  - name: http
    port: 80
  - name: https
    port: 443

However, if we have multiple bindings, suddenly doing service:test-project:port wouldn't return a value; you'd need to do service:test-project:http:port to get the http port. This is a problem as suddenly adding an https binding causes your service discovery to break.

The next proposal is to automatically fill service:test-project:port with the http port as well as equivalent variables. Right now, my proposal is to hard code looking for the name "http" when supplying values for these variables.

How this looks E2E

tye init: Instead of just putting the port in for the bindings section based on looking at the launchsettings.json, supply default names of http and https based on the protocol.

tye run: Write service:{name}:{bindingname}:port for service discovery. If name "http" is specified, write that to service:{name}:port. If no tye.yaml file is present, do the same convention in tye init.

tye deploy: From what I can tell, the same as tye run.

Open questions

• Do we really need names if we could just use protocol instead? What would that look like?
• Should we use the name "http" as the default? What about just taking the first one?
• If there is only one port (http), you can't reference service:{name}:http:port. Is that a big deal here?

Blazor supports custom UI for when your reconnection is rejected by the server (translation: when the server is gone). We should do something nicer than the default - it's meant to be replaced.

Pretty much the title. I expected tye init to noop if tye.yml is already present OR at least warn the user. However it overwrote the file instead and there is no warning.

Either we should:

1. Warn the user that they are overriding the file and prompt them for Y/N on override.
2. Do a smart merge of the yml files
3. noop

I vote for option 1 for now.

This is a bit non-source control friendly:

Maybe something like local that defaults to the local docker daemon configured settings.

We have drastically different console output depending on the command used, and we have an emoji that doesn't render on windows.

Right now, there are some hard coded places where we assume we are building and using debug assets in tye run. People may want to do tye run with release bits. This could come from a flag passed into tye run.

Microservices may consist of apps and functions (and other executables) see what it takes to launch the functions runtime as part of the microservice topology.

We should add health checking support to the host to know if a service is running properly.

There should be a way to specify that the orchestrator should launch dapr and it would appropriately wire up all of the ports and app name to make it work.

For local dev passwords (sql server etc), we need a way to store local secrets and have the ochestrator to make them available to the process/container being launched. The main scenario for this is running local databases in tye. For example:

- name: vote
  project: vote/vote.csproj
- name: postgres
  image:  postgres
  env:
    - name: POSTGRES_PASSWORD
      value: "pass@word1"
  bindings:
    - port: 5432

It would be nice if that was a secret that could be referenced as such in the manifest and made available to the application.

- name: vote
  project: vote/vote.csproj
- name: postgres
  image:  postgres
  env:
    - name: POSTGRES_PASSWORD
      source: password
  bindings:
    - port: 5432

Before

• Approval for name
• Package Publishing
• Signing
• License
• Copyright headers
• Code of conduct
• CLA
• Survey included with product (update url)
• Package prefix reservations
• Third-party notices (chill with artak)
• Plans for promotion (delay blogpost, just use twitter)

After

• Package on nuget.org
• Update getting started/samples to point to right versions
• Remove access token from redis section (raw content requires a token on private repos)

Should we do this? Should we do something else? (Prometheus)

The idea is that we can leave the host and dashboard running, and services can drop in and out.

14:01:45 INF] Executing application from  /Users/ryan/github.com/dotnet/tye/samples/single-project/tye.yaml
[14:01:45 INF] Dashboard running on http://127.0.0.1:55862
[14:01:50 INF] Building project /Users/ryan/github.com/dotnet/tye/samples/single-project/tye.yaml/test-project/test-project.csproj
[14:01:50 INF] Building /Users/ryan/github.com/dotnet/tye/samples/single-project/tye.yaml/test-project/test-project.csproj failed with exit code 1:
[14:02:11 INF] Shutting down...

I introduced a bug that lead to the project full path being computed teh wrongly. It doesn't give you a good error message.

Derp. forgot to put this in.

Repo:

• create two web projects
• put them in a solution
• tye run
• LOL

[16:05:35 INF] frontend_b85727fc-0 running on process id 21941 bound to https://localhost:5001, http://localhost:5000
[16:05:35 INF] frontend_b85727fc-0 process exited with exit code 134
[16:05:35 INF] Launching service frontend_f7112b2e-f: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:35 INF] frontend_f7112b2e-f running on process id 21942 bound to https://localhost:5001, http://localhost:5000
[16:05:36 INF] frontend_f7112b2e-f process exited with exit code 134
[16:05:36 INF] Launching service frontend_52df2c97-3: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:36 INF] frontend_52df2c97-3 running on process id 21943 bound to https://localhost:5001, http://localhost:5000
[16:05:36 INF] frontend_52df2c97-3 process exited with exit code 134
[16:05:36 INF] Launching service frontend_d0ae83f5-9: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:36 INF] frontend_d0ae83f5-9 running on process id 21944 bound to https://localhost:5001, http://localhost:5000
[16:05:37 INF] frontend_d0ae83f5-9 process exited with exit code 134
[16:05:37 INF] Launching service frontend_a823d92f-c: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:37 INF] frontend_a823d92f-c running on process id 21945 bound to https://localhost:5001, http://localhost:5000
[16:05:37 INF] frontend_a823d92f-c process exited with exit code 134
[16:05:37 INF] Launching service frontend_4167df63-f: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:37 INF] frontend_4167df63-f running on process id 21946 bound to https://localhost:5001, http://localhost:5000
[16:05:38 INF] frontend_4167df63-f process exited with exit code 134
[16:05:38 INF] Launching service frontend_952d7e3d-1: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:38 INF] frontend_952d7e3d-1 running on process id 21947 bound to https://localhost:5001, http://localhost:5000
[16:05:38 INF] frontend_952d7e3d-1 process exited with exit code 134
[16:05:38 INF] Launching service frontend_26d75dd7-9: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:38 INF] frontend_26d75dd7-9 running on process id 21948 bound to https://localhost:5001, http://localhost:5000
[16:05:39 INF] frontend_26d75dd7-9 process exited with exit code 134
[16:05:39 INF] Launching service frontend_21e66720-e: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:39 INF] frontend_21e66720-e running on process id 21949 bound to https://localhost:5001, http://localhost:5000
[16:05:39 INF] frontend_21e66720-e process exited with exit code 134
[16:05:39 INF] Launching service frontend_bf0590bf-2: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:39 INF] frontend_bf0590bf-2 running on process id 21950 bound to https://localhost:5001, http://localhost:5000
[16:05:40 INF] frontend_bf0590bf-2 process exited with exit code 134
[16:05:40 INF] Launching service frontend_798a0d69-c: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:40 INF] frontend_798a0d69-c running on process id 21951 bound to https://localhost:5001, http://localhost:5000
[16:05:40 INF] frontend_798a0d69-c process exited with exit code 134
[16:05:40 INF] Launching service frontend_e0b62b06-7: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:40 INF] frontend_e0b62b06-7 running on process id 21952 bound to https://localhost:5001, http://localhost:5000
[16:05:41 INF] frontend_e0b62b06-7 process exited with exit code 134
[16:05:41 INF] Launching service frontend_cfaffbba-e: /Users/ryan/github.com/dotnet/tye/samples/frontend-backend/frontend/bin/Debug/netcoreapp3.1/frontend
[16:05:41 INF] frontend_cfaffbba-e running on process id 21953 bound to https://localhost:5001, http://localhost:5000

We tossed around the idea of having both local and remote services running for an entire application. For example, a team's dependent services may be running externally rather than on your local dev machine. Base on some condition, a developer can switch from running an application entirely locally to running individual services and proxying other services remotely.

This work would mostly be in the ProxyService: https://github.com/dotnet/tye/blob/master/src/micronetes/Micronetes.Hosting/ProxyService.cs.