As: App designer
I want to: support anti-affinity for Computes on openstack
So that: I can improve resilience

AC1: anti affinity on Openstack server groups

As: An Kubernetes operator

I want to: Impose that Kubernetes apps deployed by Yorc use a specific namespace

So that: I have my own reasons! I'm an OPS! (Security, auth, limits, monitoring,...)

AC1: Should be configurable

AC2: Idealy configurable in A4C to allow mapping.

AC3: If no namespace specified, then create a new one as for now

Support of applicative Jobs

As: an app manager
I want to: Access GCE computes from a public IP
So that: expose services

AC1: Could provision a new Public IP and associate it to a compute
AC2: Could reuse an existing Public IP and associate it to a compute

As: Yorc team
I want to: upgrade to a newer terraform version
So that: I can benefit from latest bugfixes and improvments

AC1: Check how to handle the plugin split from 1.10 https://www.hashicorp.com/blog/upcoming-provider-changes-in-terraform-0-10
AC2: This contains interesting info https://www.terraform.io/guides/running-terraform-in-automation.html
AC3: Take this into account in packaging (standard packages + docker)
AC4: Fix deprecations

Side note: to benefit from additional support on AWS Batch we will need to do it

As: A Yorc operator
I want: to have infrastructures configuration distributed/shared across a Yorc cluster
So that: I will not set the config on each instance

AC1: Config stored into consul
AC2: First instance that publish an infra config wins
AC3: Add CLI commands to add/update/delete/show infra config

As: an app designer
I want to: define a job that could be executed on Kubernetes
So that: I could execute some workloads on the cloud

AC1: Based on Job spec

As: an app designer
I want to: handle secrets in a more secure way by not adding them into the topology
So that: I can prevent secrets stealing

AC1: consider the use of a new get_secret TOSCA operation to be defined

Sub Tasks

• Check that Alien and our plugin send the get_secret function correctly
• implement the get_secret function in the TOSCA functions resolver
• Check if we can integrate with Ansible vault to prevent secrets from being wrote on the Orchestrator host in plain text
• refactor deployment package to expose if resolved tosca values are secret or not

Reported by @billmetangmo on an pre-3.0 version.
To be checked if still relevant

I have noticed that every time i undeploy an application that is waiting for slurm reservation resources ; as a consequence Janus is no longer able to deploy any app again

To reproduce the issue:

create a slurm partitions with 3 nodes
deploy an application with 2 nodes with ALien4Cloud
deploy the same application in another environment
undeploy it
try to deploy an app with 1 node

Regards

As: Yorc dev team
I want to: Make run operation asynchronous
So that: It won't block a worker for the duration of the Job

AC1: check if we can do it using standard TOSCA rules

As: an app manager
I want to: attach block storage on an GCE Compute
So that: I can persist informations on those persistent storages

AC1: Could provision a new volume and attach it to a compute
AC2: Could reuse an existing volume and attach it to a compute
AC3: Compute could have multiple volumes attached
AC4: All above should work with scalable computes

Generic epic for engineering user stories

As: A component designer
I want: to deal myself with TOSCA function resolution errors
So that: I can deal with the existence or not of a given attribute

AC1: See https://confluence.sdmc.ao-srv.com/display/BRBDCF/Janus+TOSCA+functions+resolver+errors+handling (should be moved to a public wiki soon)

Issue Type

Put an X between the brackets of the corresponding issue type

• Bug report
• Documentation issue report
• Improvement request

Description

Initiate a Blog for Ystia opensource project with the GCP demo

• create a site at https://ystia.github.io/
• create a blog at https://ystia.github.io/blog/
• Rework GCP sample to:
  • minimize prerequisites
  • provide scripts automating the setup
• Add a blog entry describing how to use the sample

Issue Type

Put an X between the brackets of the corresponding issue type

• Bug report
• Documentation issue report
• Improvement request

Description

See build https://travis-ci.org/ystia/yorc/builds/397266821?utm_source=github_status&utm_medium=notification

--> Running go test
# github.com/ystia/yorc/vendor/github.com/armon/go-metrics/prometheus
vendor/github.com/armon/go-metrics/prometheus/prometheus.go:1: +build comment must appear before package clause and be followed by a blank line%!(EXTRA []interface {}=[])
?   	github.com/ystia/yorc	0.058s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/bench	0.057s	coverage: 0.0% of statements [no tests to run]
ok  	github.com/ystia/yorc/commands	0.087s	coverage: 67.5% of statements
?   	github.com/ystia/yorc/commands/deployments	0.061s	coverage: 9.7% of statements [no test files]
?   	github.com/ystia/yorc/commands/deployments/tasks	0.063s	coverage: 4.6% of statements [no test files]
?   	github.com/ystia/yorc/commands/deployments/workflows	0.060s	coverage: 13.9% of statements [no test files]
?   	github.com/ystia/yorc/commands/hostspool	0.056s	coverage: 9.8% of statements [no test files]
?   	github.com/ystia/yorc/commands/httputil	0.008s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/config	0.006s	coverage: 78.4% of statements
# github.com/ystia/yorc/deployments
deployments/instances.go:141: Wrapf call needs 2 args but has 3 args
FAIL	github.com/ystia/yorc/deployments [build failed]
ok  	github.com/ystia/yorc/events	2.209s	coverage: 69.6% of statements
ok  	github.com/ystia/yorc/helper/collections	0.003s	coverage: 100.0% of statements
?   	github.com/ystia/yorc/helper/consulutil	0.005s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/helper/executil	0.003s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/helper/labelsutil	0.003s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/helper/labelsutil/internal	0.009s	coverage: 94.4% of statements
?   	github.com/ystia/yorc/helper/mathutil	0.003s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/helper/metricsutil	0.003s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/helper/provutil	0.003s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/helper/sliceutil	0.004s	coverage: 100.0% of statements
?   	github.com/ystia/yorc/helper/sshutil	0.004s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/helper/stringutil	0.005s	coverage: 100.0% of statements
?   	github.com/ystia/yorc/helper/tabutil	0.003s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/helper/ziputil	0.003s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/log	0.003s	coverage: 2.3% of statements [no test files]
ok  	github.com/ystia/yorc/plugin	0.118s	coverage: 71.4% of statements
?   	github.com/ystia/yorc/prov	0.005s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/prov/ansible	3.898s	coverage: 39.8% of statements
ok  	github.com/ystia/yorc/prov/hostspool	13.386s	coverage: 60.2% of statements
?   	github.com/ystia/yorc/prov/kubernetes	0.059s	coverage: 0.3% of statements [no test files]
# github.com/ystia/yorc/prov/monitoring
prov/monitoring/monitoring_mgr.go:248: LogEntry.Registerf call needs 2 args but has 3 args
FAIL	github.com/ystia/yorc/prov/monitoring [build failed]
?   	github.com/ystia/yorc/prov/operations	0.006s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/prov/slurm	3.423s	coverage: 18.1% of statements
?   	github.com/ystia/yorc/prov/terraform	0.006s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/prov/terraform/aws	7.697s	coverage: 60.7% of statements
?   	github.com/ystia/yorc/prov/terraform/commons	0.006s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/prov/terraform/google	3.246s	coverage: 43.4% of statements
ok  	github.com/ystia/yorc/prov/terraform/openstack	3.799s	coverage: 35.0% of statements
# github.com/ystia/yorc/prov/validation
prov/validation/compute_capability_endpoint.go:37: LogEntry.Registerf call needs 1 arg but has 2 args
FAIL	github.com/ystia/yorc/prov/validation [build failed]
?   	github.com/ystia/yorc/registry	0.005s	coverage: 1.2% of statements [no test files]
ok  	github.com/ystia/yorc/rest	2.020s	coverage: 16.2% of statements
?   	github.com/ystia/yorc/server	0.060s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/tasks	1.783s	coverage: 54.3% of statements
ok  	github.com/ystia/yorc/tasks/workflow	2.924s	coverage: 23.9% of statements
?   	github.com/ystia/yorc/testutil	0.005s	coverage: 0.0% of statements [no test files]
ok  	github.com/ystia/yorc/tosca	0.022s	coverage: 80.5% of statements
?   	github.com/ystia/yorc/vault	0.005s	coverage: 0.0% of statements [no test files]
?   	github.com/ystia/yorc/vault/hashivault	0.007s	coverage: 1.2% of statements [no test files]
make: *** [test] Error 2

As: An operator
I want to: deploy a new version of Yorc and its dependencies with minimal downtime
So that: I can deploy an new version

AC1: ...

As: an app manager
I want to: undeploy a container on top of GCP GKE
So that: I can deploy containers on GKE

AC1: All created resources should be deleted specially those that cost

As: an operator
I want to: I want to plug another Vault implementation in Janus
So that: I can use my awesome Vault instead of the default one

AC1: Based on existing vault integration

Integration with Kubernetes

Slurm support

I forget to set the environment variables about the proxy;
I try to deploy a basic application on AWS.
It's take 90 mn to give me this following error:

2018/03/29 13:21:01 [DEBUG]ESC[31mError running plan: 1 error(s) occurred:

provider.aws: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: dial tcp 54.239.21.217:443: i/o timeoutESC[0mESC[0m
2018/03/29 13:21:01 [DEBUG] [2018-03-29T13:21:01.500612304+02:00][ERROR][W-Environment][install][4928ed7f-c34e-437c-ac18-1506d484e83a][Compute][][delegate][install][
]Error running plan: 1 error(s) occurred:
provider.aws: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: dial tcp 54.239.21.217:443: i/o timeout

During 90mn I got this following debug logs:

2018/03/29 13:19:50 [DEBUG] Step "Welcome_initial" still waiting for 1 previous steps
2018/03/29 13:19:50 [DEBUG] Step "Welcome_created" still waiting for 1 previous steps
2018/03/29 13:19:50 [DEBUG] Step "Welcome_creating" still waiting for 1 previous steps
2018/03/29 13:19:50 [DEBUG] Step "Welcome_configuring" still waiting for 1 previous steps
2018/03/29 13:19:50 [DEBUG] Step "Welcome_configured" still waiting for 1 previous steps
2018/03/29 13:19:50 [DEBUG] Step "Welcome_starting" still waiting for 1 previous steps
2018/03/29 13:20:20 [DEBUG] Step "Welcome_created" still waiting for 1 previous steps
2018/03/29 13:20:20 [DEBUG] Step "Welcome_started" still waiting for 1 previous steps
2018/03/29 13:20:20 [DEBUG] Step "Welcome_initial" still waiting for 1 previous steps
2018/03/29 13:20:20 [DEBUG] Step "Welcome_starting" still waiting for 1 previous steps

As: An operator
I want to: place an deployment or a node or an instance in maintenance mode
So that: I can do some maintenance operation without being interrupted by Yorc automatic processes

AC1: Maintenance mode mean that Yorc will not try to perform any operation on element in maintenance
AC2: Available at instance, node ant deployment level
AC3: Manageable via the REST API

As: App designer
I want to: express my filtering request based on regexp or if labels not present
So that: I can have a better expressiveness

AC1: Add match (=~) and don't match (!~) a regexp
AC2: Add match if label doesn't exists (! labelname)
AC3: Check if we can add some others filters types

Issue Type

Put an X between the brackets of the corresponding issue type

• Bug report
• Documentation issue report
• Improvement request

Reported by

@stebenoist

Description

2018/07/03 14:10:36 [INFO] panic: [PANIC]Failed to store TOSCA Definition for deployment with id "a4cApp-Environment", (file path "work/deployments/a4cApp-Environment/overlay/topology.yml"): Failed to parse internal definition org.alien4cloud.alien4cloud.webapp/2.0.0-SNAPSHOT/types.yml: yaml: unmarshal errors:
 line 71: cannot unmarshal !!map into []tosca.ArtifactDefMap

Error root is raised because of the missing mandatory attirbute "file" for artifact.

Found in https://github.com/alien4cloud/csar-public-library/tree/develop/org/alien4cloud/alien4cloud/topologies/a4c_ha

In topology.yml:

topology_template:
  inputs:
    ...
  input_artifacts:
    alien_dist:
     type: tosca.artifacts.File

In the org.alien4cloud.alien4cloud.webapp:

node_types:
  org.alien4cloud.alien4cloud.webapp.nodes.Alien4Cloud:
    ...
    artifacts:
      bin:
        file: bin
        type: tosca.artifacts.File
      ssl:
        file: ssl
        type: tosca.artifacts.File
      commons:
        file: scripts/commons
        type: tosca.artifacts.File
      alien_dist:
        type: tosca.artifacts.File

For query task as infra usage collect, we need to provide at least an execution date, or an expiration date if possible. This to allow user to take in account time parameter because provided information is only valid for a specified moment.

Reported by @stebenoist

Issue Type

Put an X between the brackets of the corresponding issue type

• Bug report
• Documentation issue report
• Improvement request

Description

1/ Add the capability to start directly yorc from the bintray registry to README.md
2/ Add the link to registrib and distrib files: https://bintray.com/ystia/yorc-engine/distributions

So that we can easily download & use yorc engine.

Reported by @billmetangmo on a pre-3.0 release

---------------------------- Environment --------------
Alien4cloud: 1.3.4
Janus-slurm: build 55
Janus-alien4cloud-plugin: build 90

If you try to deploy an application with 2 nodes in a slurm partition that has 3 nodes ( 2 busy- 1 free). Janus-slurm will allocate 1 node and the request for the other node reservation would be in "PENDING" state.

From my point of view, this should not be the case as Janus manages "deployments" at functionnal level not nodes . Futhermore, the deployment of 1/2 nodes of my previous application didn't mean anyhing at functionnal level because the typical multi-node deployment is a distributed system with one or more components. This behaviour could also damage cluster usage as resource less deployments couldn't be deployed could not until reousrec-intensive apps get all requested resources ( from my previous example, an application that needs one compute should wait ...) Managing nodes should be let to Slurm.

To reproduce the issue:

create a slurm partitions with 3 nodes
deploy an application with 2 nodes with ALien4Cloud
deploy the same application in another environment
Regards

As: an app manager
I want to: scale a container at runtime on GCP GKE
So that: I can scale containers on GCP

AC1: Scale at runtime
AC2: also scale attached volumes
AC3: should work for deployments that expose a service

As: Yorc engineering team
I want to: Make the step the task unit of a workflow execution
So that: it will be the first step to distributed deployments

AC1: Different steps of a same workflow could be executed on different Yorc instances
AC2: Yorc instances synchronize themselves using consul to know which one will handle a step
AC3: Reuse as much as possible the distributed tasks framework already implemented in Yorc

Use case:

Given 2 instances of Yorc with 1 worker on each.

And a application composed of just 2 computes.

I want to see deployment of 2 computes happening in parallel

As: an app manager
I want to: attach GCE Computes to private virtual networks
So that: I can have a dedicated applicative network for my app

AC1: Provision virtual networks and attach it to GCE computes

This Epic will reference US/tasks needed to be TOSCA compliant

As: An ops
I want: to ensure that connection credentials are not left non-encrypted on disk
So that: I can relax and sleep this night

AC1: Limited to the ansible integration
AC2: reuse integration done within #90
AC3: the idea is to use ansible/ansible#22382 when implemented

Note another solution is to use ssh agent to store those keys (but it wont work with paramiko)

As: An app designer
I want to: combine policies like auto-scaling and placement for instance
So that: even with an automatic process my requirements are still met

AC1: ...

As: an operator
I want to: know how to backup and restore a Yorc Env
So that: I can recover from a disaster

AC1: Document Consul backup & restore feature
AC2: Document Yorc specific features

We could change the way the host is elect for allocation in order to select preferably an already shared host for a shareable compute.
This should optimize the usability of the hosts pool

As: an app manager
I want to: deploy a container on top of GCP GKE
So that: I can deploy containers on GCP

AC1: In A4C we could use a generic portable container node type
AC2: Exposed (mapped) ports should be retrievable

Support Singularity containers