Description:
This issue is observed when using update API for updating placement intent of an application

Steps to reproduce:

1. create logcal cloud with cluster1 and cluter2.
2. specify cluster2 as the application placement intent.
3. Approve and instantiate the dig.
4. Edit the application placement intent, and specify cluster1 this time.
5. Call update API.
6. The application is terminated from cluster2 and spun on cluster1 as expected.
7. However, Status API at this point returns Deleted for cluster2, but there is no information pertaining the Deployed on cluster1 status

Attached Log:
update_err.log

Hello, I ran into two different problems with deployments with EMCO, more specifically Helm's template helpers not working as intended.

I deployed the following Bitnami chart: https://github.com/bitnami/charts/tree/master/bitnami/mongodb, which was packaged in another chart, and encountered the two following problems:

• Firstly, some functions used in bitnami .tpl aren't working with EMCO, as they leverage features of Helm 3.2+, like this function: https://helm.sh/docs/chart_template_guide/functions_and_pipelines/#using-the-lookup-function making the instantiation of the DeploymentIntent fail

I encountered this with the Chart version 10.11.0 of the Bitnami chart

It is not a problem on itself, but I haven't found the exact version of Helm currentlu supported by EMCO

• Secondly, for the same chart, I encountered another issue with template helpers, more specifically the following type of code:

{{/*
Return the appropriate apiVersion for statefulset.
*/}}
{{- define "common.capabilities.statefulset.apiVersion" -}}
{{- if semverCompare "<1.14-0" (include "common.capabilities.kubeVersion" .) -}}
{{- print "apps/v1beta1" -}}
{{- else -}}
{{- print "apps/v1" -}}
{{- end -}}
{{- end -}}

This code query the kubeVersion to render the statefulset's apiVersion
I used a cluster with Kubernetes 1.19, so normally this code should have returned the apiVersion "apps/v1"

However, with EMCO, this function returned "apps/v1beta1", making the deployment of the statefulset fail on the edge cluster, because this apiVersion is deprecated in Kubernetes 1.19

You can find this code here: https://github.com/bitnami/charts/blob/master/bitnami/common/templates/_capabilities.tpl
And it is used here: https://github.com/bitnami/charts/blob/master/bitnami/mongodb/templates/replicaset/statefulset.yaml

I encountered this with older versions of the Bitnami chart, like 10.3.3

Hello.

I'm trying to deploy an app using EMCO, but EMCO fails to render the Helm manifests when instantiating the composite app.
It fails at the rendering of the first chart of the composite app.

I'm using emcoctl and manifests to deploy everything, and I'm using images I built with the code from this repository.

I registered the clusters, created and instantiated the virtual cluster and created and approved the composite app (and its profiles and intents) with no issues.
However, when instantiating with emcoctl, I get the following error:

URL: projects/free5gc/composite-apps/free5gc-demo/v1/deployment-intent-groups/free5gc-deployment-intent/instantiate Response Code: 500 Response: Unable to get the sorted templates for app: Error while generating final k8s yaml: stat /tmp/k8s-ext-716707460/free5gc-control-plane: no such file or directory

The logs of the orchestrator give the same information:
{"level":"error","msg":"Unable to get the sorted templates for app: Error while generating final k8s yaml: stat /tmp/k8s-ext-961093037/free5gc-control-plane: no such file or directory","time":"2021-02-23T15:39:35Z"}
There are currently no other errors.

I can render the Helm archives used for the CompositeApp with helm install test --dry-run --debug <archive>.tar.gz with no issue.

If needed, here are the logs of the orchestrator, with a higher log verbosity ("info"):

{"CompositeAppHandle":"/context/3773784810858181293/","level":"info","msg":":: CreateCompositeApp ::","time":"2021-02-23T15:42:50Z"}
{"CompositeApp":"free5gc-demo","DeploymentIntentGroup":"free5gc-deployment-intent","Project":"free5gc","Release":"r1","Version":"v1","level":"info","msg":":: The meta data stored in the runtime context :: ","time":"2021-02-23T15:42:50Z"}
{"appName":"free5gc-control-plane","level":"info","msg":":: Processing App ::","time":"2021-02-23T15:42:50Z"}
{"appName":"free5gc-control-plane","level":"info","msg":":: Got the app content.. ::","time":"2021-02-23T15:42:50Z"}
{"appName":"free5gc-control-plane","level":"info","msg":":: Got the app Profile content .. ::","time":"2021-02-23T15:42:50Z"}
{"chartBasePath":"/tmp/k8s-ext-434126427","level":"info","msg":"The chartBasePath ::","time":"2021-02-23T15:42:51Z"}
{"Profile Path":"/tmp/k8s-ext-237123582","level":"info","msg":"The profile path:: ","time":"2021-02-23T15:42:51Z"}
{"level":"info","msg":"Got the profileYamlClient..","time":"2021-02-23T15:42:51Z"}
{"level":"error","msg":"Error while generating final k8s yaml","time":"2021-02-23T15:42:51Z"}
{"Dir: ":"/tmp/k8s-ext-237123582/","level":"info","msg":"Clean up k8s-ext-tmp-dir::","time":"2021-02-23T15:42:51Z"}
{"Dir: ":"/tmp/k8s-ext-434126427/","level":"info","msg":"Clean up k8s-ext-tmp-dir::","time":"2021-02-23T15:42:51Z"}
{"len(sortedTemplates):":0,"level":"info","msg":":: Total no. of sorted templates ::","time":"2021-02-23T15:42:51Z"}
10.244.0.0 - - [23/Feb/2021:15:42:50 +0000] "POST /v2/projects/free5gc/composite-apps/free5gc-demo/v1/deployment-intent-groups/free5gc-deployment-intent/instantiate HTTP/1.1" 500 160
{"level":"error","msg":"Unable to get the sorted templates for app","time":"2021-02-23T15:42:51Z"}
{"level":"error","msg":"Unable to get the sorted templates for app: Error while generating final k8s yaml: stat /tmp/k8s-ext-434126427/free5gc-control-plane: no such file or directory","time":"2021-02-23T15:42:51Z"}

Do you have an idea of how to proceed to debug ?

Thank you for your help

Hello,

I'm currently using a level-1 logical cloud to deploy Prometheus Operator.

The manifest is as follow:

version: emco/v2
resourceContext:
  anchor: projects/{{.ProjectName}}/logical-clouds
metadata:
  name: {{.PrometheusCloud}}
spec:
  namespace: prometheus
  user:
    user-name: emco
    type: certificate
    user-permissions:
    - permission-name: prometheus-permission
      apiGroups:
      - ""
      - "apps"
      - "k8splugin.io"
      - "monitoring.coreos.com"
      - "rbac.authorization.k8s.io"
      - "admissionregistration.k8s.io"
      - "batch"
      - "policy"
      - "apiextensions.k8s.io"
      resources:
      - secrets
      - pods
      - configmaps
      - services
      - deployments
      - resourcebundlestates
      - servicemonitors
      - podsecuritypolicies
      - serviceaccounts
      - rolebindings
      - roles
      - clusterroles
      - clusterrolebindings
      - validationwebhookconfigurations
      - mutatingwebhookconfigurations
      - jobs
      - prometheusrules
      - alertmanagers
      - alertmanagerconfigs
      - customresourcedefinitions
      - daemonsets
      - prometheus
      - prometheusrules
      verbs:
      - get
      - watch
      - list
      - create

Contrary to a role's definition, I cannot declare rules for a specific apiGroup/resource/verbs, like this for example:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: mynamespace-user-full-access
  namespace: mynamespace
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["batch"]
  resources:
  - jobs
  - cronjobs
  verbs: ["*"]

Furthermore, it is not clear if the logical-cloud's user permissions are cluster-wide or namespace-wide only.

I think implementing rules like a Kubernetes role, with a distinction for cluster-wide or namespace-wide permissions, could be a useful feature.

Hello,

While installing the new release with the provided Helm Charts, without providing any specific Helm parameters, the installation fails.

Etcd is stuck in a non-ready state, and EMCO's services are unable to authenticate to mongodb.
I retried after deleting everything in /dockerdata-nfs/ and deleting and recreating the namespace, but it did not fix the problem.

Some pv and pvc of etcd and mongo are still created even though the persistence is not enabled (I don't really need the persistence for my test that's why I didn't enabled it).
Perhaps the problem comes from here ?

I then installed EMCO with the parameters --set mongo.persistence.enabled=true --set etcd.persistence.enabled=true and it fixed this issue, the installation works out of the box with these parameters.

hi all:
The DTC(Distributed Traffic Controller) component really worked for NetworkPolicy which is essentially used for Access Control, not for Traffic Control. So I think the dtc may cause misunderstanding with Linux Traffic Control. Would you consider to change it's name to DAC(Distributed Access Control) which I think can show the nature of it's function. Thanks!

Hello All,

I have a question regrading EMCO level-1 logical cloud configuration/deployment. I have tried several time to deploy it with namespace "eit1" and level-1. I need it as the target namespace is always configured on my target k8s cluster.
The problem is that my configuration is not taken into account.
I know this because if I connect to mongo-db and check cloudconfig key in the DB ("key" : "{cluster,level,namespace,provider,}") it still contains level definition set to 0 and namespace definition set to default:
{
"_id" : ObjectId("60742d10499f9e246826257f"),
"cluster" : "cluster1",
"level" : "0",
"namespace" : "deafult",
"provider" : "provider1",
"config" : {
"config" : "YXBpVmV(.......)2c9PQo="
},
"key" : "{cluster,level,namespace,provider,}"
}
logicalcloudcondig.txt

However the orchestartor DB configuration contains correct data:
{
"_id" : ObjectId("60745026c1986d527c297386"),
"logical-cloud-name" : "lc1",
"project" : "proj1",
"key" : "{logical-cloud-name,project,}",
"logicalcloud" : {
"metadata" : {
"logicalcloudname" : "lc1",
"description" : "",
"userdata1" : "",
"userdata2" : ""
},
"specification" : {
"namespace" : "eit1",
"level" : "1",

Of course the instantiation will fail as there will be no reference to cluster to deploy the app on it.
But when I edit those entries in MONGO DB, all instantiation, termination operations are running fine.

Is it something that I am missing in the configuration or it's a bug?
Any help is highly appreciated. If you need any logs etc. I can send it at will.

BTW. EMCO is a nice piece of SW. I love it :)

regards
Greg

A logical cloud which is used in a deployment intent group can be terminated and then deleted by deleting cluster references.
This should not happen, otherwise logical cloud and DIG will go out of sync.
Desired behavior : if we try to terminate a logical cloud which is a part of any DIG then we should through an error like we do for other components which is "Can't terminate/delete parent without deleting child references first."

The requirement is to view the k8s resources that are deployed through the EMCO platform.
The status API today does return the list of all resources that are deployed, the requirement to view the resource yaml of a specific resorurce, something equivalent to kubect get configmap -o yaml

Hello,

I'm currently working on a process of migration of applications (containers) between multiple remote Clusters. In order to provide unninterrupted communication for the MEC customers (that might be moving - user mobility) to MEC server (K8S Cluster) EMCO should have full control of an action of migration application to the new cluster.

First of all, new instance of application should be instantiated in a new cluster, and later on, once user will communicate with new application, an old instance of app in old cluster should be removed.

After analysis of current EMCO features I can assume that migration should be composed of two steps:

1. Update Generic Placement Intent (currently only CREATE and DELETE is available, but I saw that UPDATE is ongoing, could you confirm?)
2. Update Deployment Group - that will diffrentiate current state of EMCO DB with new values overrided by step 1 and redeploy application that contains new Placemen Intent.

In my best understanding we could utilize monitor (cluster watcher) to synchro state across new and old cluster and execute method of delete old application in old cluster sequentially after the creation of new application in new cluster will be completed.

I'd like to ask - do you see right place to implement step 2? Should it be Generic Action Controller, or we should consider implementing own custom controller for migration operation?

Currently EMCO seems to be not ready to control sequence of executing isntantiatiation/delete method -> It is executing each operations at the same time, what make service (applciation) downtime in migration process -> undesirable.

Thanks in advance for response. I'm interested in both discussion and code contribution.
Greg

Description:
This issue is observed when using update API for updating network intent of an application

Steps to reproduce:

1. create logcal cloud with cluster1
2. specify cluster1 as the application placement intent.
3. specify network intents for application
4. Approve and instantiate the dig.
5. Edit the application network intent.
6. Call update API.
7. A new service instance is created for updated network intents, and the previous service instance still remains.

The requirement is to have an API to get the resource consumption in target clusters in order to make a choice on the target cluster while specifying the placement intent for an application.

It will be desirable to have,

1. GET resource consumption at cluster level.
2. GET aggregated resource consumption at logical cloud level.

-Sandeep

Hello,

I'm currently using EMCO to deploy Helm charts on a level-0 logical cloud.

From what I understood, a level-1 logical cloud has only one namespace (consistent across multiple clusters), so the apps deployed on this level-1 logical cloud will be deployed on this namespace.

However, with a level-0 logical cloud, I haven't found how to specify the deployment's namespace ? (Something similar to the --namespace flag of an Helm release).

Is it currently possible to specify a specific namespace when deploying on a level-0 logical cloud ? If so, does it need to be created beforehand (or is there an equivalent of Helm's --create-namespace)?

There should be provision to specify the cluster in the network interface intent ( under a workload intent).

It is requiered in the scenaio where the appliation has to deployed to multiple clusters in a logical cloud, and the clusters have a provider network with same subnet and VLAN id.
In the DIG, there is provision to specify multiple clusters in the generic placement intents, but there is not way to map a network interface intent to a cluster.

-Sandeep