indigo-dc / alien4cloud-deep Goto Github PK

The concat method doesn't work in outputs when combined with other methods e.g.

concat: [ "a", get_attribute: [ server, ip ], "c" ]

as listed by Github at https://github.com/indigo-dc/alien4cloud-deep/network/alerts

Allow for pull requests when updates are done.

Use Jacoco to determine the coverage

The procedure to obtain the certificate from orchestrator and INDIGO-IAM (Obtain Certificate secion) is useless if presented and explained before the the Plugin Parameters secion. Obtain Certificate secion should be a subsection of the Plugin Parameter.

Reported by @alexcos78

The imports in A4C use the format:

import_name: version
In indigo and other projects the imports are written as:

import_name: link_to_TOSCA

A4C should convert the link_to_TOSCA path to a version of the import_name that already exists in the A4C local repo.

the getEventsSince method from the IndigoDCOrchestrator doesn't update the status of a deployment.

Create an integration testing pipeline for all components in this repo.

Instead of having a version for the CSARs containing the normative types, we should store the path that is used by the topologies that make use of that CSAR.
instead of:

metadata:
  template_name: tosca-normative-types
  template_author: TOSCA TC
  template_version: 1.0.0-ALIEN20

we should have

metadata:
  template_name: tosca-normative-types
  template_author: TOSCA TC
  template_version: https://raw.githubusercontent.com/alien4cloud/tosca-normative-types/master/normative-types.yml

A4C_PEM_CA_CERT_FILE -> A4C_PEM_CERT_FILE
A4C_PEM_CA_KEY_FILE -> A4C_PEM_KEY_FILE

For now, we advise the users to set the value of the attributes that should get the values through get_attribute as strings containing the actual method call. Using the text editor is highly advisable. Example (normal TOSCA to Alien4Cloud version_): dns: { get_attribute: [ HOST, dns ] } becomes dns: "{ get_attribute: [ HOST, dns ] }" and IP_list:\n - { get_attribute: [ HOST, ip ] } becomes IP_list:\n - "{ get_attribute: [ HOST, ip ] }"

Try to get user per stored deployment instead of the logged in. Otherwise there is no way to get status of deployed apps when A4C starts up.

Some attributes are ignored by A4C after the first launch. Basically the values are already in the runtime cache like the admin username/password. Others like the certificates and the location of the runtime are taken into consideration each time A4C starts.

A4C converts the get_input function from the topologies you upload to scalar types (as defined by the node template).

`topology_template:

node_templates:
marathon:
force_pull_image: { get_input: rclone_conf }`

becomes

`topology_template:

node_templates:
marathon:
force_pull_image: true`

Attaching to a4c
a4c | 2018-10-24 11:02:29 INFO Main:53 - Creating A4C_RUNTIME_DIR dir on /mnt/a4c_runtime_data
a4c | 2018-10-24 11:02:29 INFO Main:66 - Write a4c conf at /opt/a4c/config/alien4cloud-config.yml
a4c | Exception in thread "main" java.lang.ClassCastException: org.bouncycastle.asn1.pkcs.PrivateKeyInfo cannot be cast to org.bouncycastle.openssl.PEMEncryptedKeyPair
a4c | at es.upv.alien4cloud.settingsmanager.ConfigManager.getKey(ConfigManager.java:192)
a4c | at es.upv.alien4cloud.settingsmanager.ConfigManager.createJavaKeystore(ConfigManager.java:161)
a4c | at es.upv.alien4cloud.settingsmanager.ConfigManager.enableSsl(ConfigManager.java:114)
a4c | at es.upv.alien4cloud.settingsmanager.Main.main(Main.java:72)
a4c | 2018-10-24 11:02:35 INFO Main:66 - Write a4c conf at /opt/a4c/config/alien4cloud-config.yml
a4c | Exception in thread "main" java.lang.ClassCastException: org.bouncycastle.asn1.pkcs.PrivateKeyInfo cannot be cast to org.bouncycastle.openssl.PEMEncryptedKeyPair

Right now we support one config for all orchestrator instances. For testing purposes and greater flexibility, we should add support for instances. This way each instance has its own configuration (like a different orchestrator location).

The README refers to a set of properties: tokenEndpointCert, orchestratorEndpointCert, iamHostCert that are not present when configuring the Orchestrator plugin. If these properties are no longer required to be defined by the user they should be omitted from the documentation.

• add repositories section to the custom_types.yaml file, and update each entry in those nodes having an artifacts section using indigodc-2-a4c.py
• resulting file works with a4c when uploading it (web UI), but it fails when stored in the init/archives (to be loaded at startup)

Relevant files a4c:

1. org.alien4cloud.tosca.catalog.AbstractArchivePostProcessor.java (alien4cloud-core subproject)
   private void processArtifact(ArchivePathChecker archivePathResolver, AbstractArtifact artifact, ParsingResult parsedArchive)
2. alien4cloud.repository.services.RepositoryService.java (alien4cloud-core subproject)
   public boolean canResolveArtifact(String artifactReference, String repositoryURL, String repositoryType, Map<String, Object> credentials)
3. org.alien4cloud.tosca.catalog.ArchiveUploadService.java (alien4cloud-core subproject)
   public ParsingResult upload(Path path, CSARSource csarSource, String workspace)

Reverse engineering again, now the UI

If a property of a node is not required it shouldn't appear at all, especially if it has a default value in the TOSCA types.

Internal error - 500
Unknown error
Artifact could not be accessed DeploymentArtifact{artifactName='indigo-dc.zabbix-agent'} AbstractArtifact{artifactType='tosca.artifacts.AnsibleGalaxy.role', artifactRef='indigo-dc.zabbix-agent', artifactRepository='null', archiveName='indigo-types', archiveVersion='1.0.0', repositoryURL='null', repositoryName='null', artifactPath=null}

with the tosca from a4c

tosca_definitions_version: alien_dsl_1_4_0

metadata:
template_name: TestApp
template_version: 0.1.0-SNAPSHOT
template_author: admin

description: ""

imports:

• indigo-types:1.0.0
• tosca-normative-types:1.0.0-ALIEN14

topology_template:
node_templates:
Compute:
type: tosca.nodes.indigo.Compute
properties:
zabbix_server: "orchestrator.cloud.cnaf.infn.it"
zabbix_server_port: 10051
zabbix_server_metadata: "Linux 668c875e-9a39-4dc0-a710-17c41376c1e0"
capabilities:
scalable:
properties:
min_instances: 1
max_instances: 1
count: 1
default_instances: 1
endpoint:
properties:
private_ip: true
protocol: tcp
secure: false
network_name: PRIVATE
initiator: source
workflows:
install:
steps:
Compute_install:
node: Compute
activity:
delegate: install
uninstall:
steps:
Compute_uninstall:
node: Compute
activity:
delegate: uninstall
start:
steps:
Compute_start:
node: Compute
activity:
delegate: start
stop:
steps:
Compute_stop:
node: Compute
activity:
delegate: stop

A4C doesn't generate the list of outputs when deployment has been completed.

In the Usage section please fix point 4 as follows:

cat crt_solos.tmp | tr -d '\n'

Basically renamed crt_solo.tmp in crt_solos.tmp.

Create an utility that manages the settings for the Alien4Cloud instance running in the docker container.

• it has to be executed before launching A4C
• it gets its values from the ENV
• it doesn't require additional langs support (use Java)
• it manages the files from the config folder:
  • alien4cloud-config.yml
  • elasticsearch.yml
  • log4j2.xml

methods are not supported in complex mappings. For example tosca.nodes.indigo.Container.Application.Docker.Marathon inherits properties->environment_variables. If you try to set an variable as get_input eg:
properties: environment_variables: RCLONE_CONFIG: { get_input: rclone_conf }
the parser fails because it tries to convert the method to a map.

The plugin is more or less functional now, be sure you respected the Google Java style guide at https://google.github.io/styleguide/javaguide.html

Given sometimes when trying to run the docker image.

Restart the container, and the users created before that are missing.

Right now the user integration for the plugin is non-existent. It saves the the user credentials (user name + password) in the general configuration of a plugin instance. We would like to do the following:

1. Remove the user name and password from the general settings of the plugin.
2. Get these on the fly when asking for the token using AuthorizationUtil.getCurrentUser method.
3. Maintain the clientID and clientSecret in the plugin instance's general conf. These are created for each new instance of A4C, that is each site where A4C is deployed. This way, if one site is compromised, we can safely disable those two secrets for the targeted site, without taking out other sites.

Caveats:

1. The user name and password of a4c MUST be the same with those used in the IAM. As a result, you must create an additional user.

Outputs created using the text TOSCA editor (even when saved in the text view) are deleted/replaced when switching to the GUI editor and saving from that view

EDIT 1: Iif an output is created using the GUI and then its name is modified in the text view, once a user saves the file again from the GUI the custom name is gone.

Due to the fact that we need to parse the TOSCA text obtained from the TOSCA editor, the TOSCA methods are transformed into nodes, therefore the whole syntax is changed.

We add a pre-processing step in which all TOSCA methods found in a document are transformed intro string values. At the end of the pre-processing, when the whole structure is transformed into string again, we de-comment all methods.

They have no description right now, mod the python script to insert doc

As per @mariojmdavid recommendations:

1. Add an ENV specifying the port. Will do for both HTTPS and HTTP
2. Add an ENV to set the admin username
3. Add an ENV to set the admin passw

Right now we have our own branch, switch to master and convert it.

Use the EventService class to manage status retrieval for all A4C methods querying the status.

Use a4c rest API to install a newer version of the plugin for dev purposes

Find a way to retrieve the user info during startup (password needed).

Using indigodc-2-a4c.py with the custom_types.yaml converts

 tosca.nodes.indigo.Container.Runtime.Docker:
    derived_from: tosca.nodes.SoftwareComponent
    capabilities:
      host:
        type: tosca.capabilities.indigo.Container.Docker
      scalable:
        type: tosca.capabilities.Scalable
  requirements:
    - host:
        capability: tosca.capabilities.Container
        node: tosca.nodes.Compute
        relationship: tosca.relationships.HostedOn
occurrences: [0, 1]

to

  tosca.nodes.indigo.Container.Runtime.Docker:
    derived_from: tosca.nodes.SoftwareComponent
    capabilities:
      host:
        type: tosca.capabilities.indigo.Container.Docker
      scalable:
        type: tosca.capabilities.Scalable
  requirements:
  - host:
      capability: tosca.capabilities.Container
      node: tosca.nodes.Compute
      relationship: tosca.relationships.HostedOn
      occurrences:
      - 0
      - 1

occurrences: should be [0,1]

If you try to import a template without metadata, A4C fails with Null pointer exception.

Deactivate it for release builds

A4c has its own version of the standard types, check if the standard ones work.

A4C uses different types of IDs. The DeploymentPaasId identifies the actual application and should be the same between different cycles of deployment/undeployment. The DeploymentId is unique to a specific deploy/undeploy cycle. Switch to the former to allow implementation of the history and loading when a4c instance is restarted.

Add a general, GNU style, CHANGELOG for quick summary of changes

The link to the file containing the TOSCA types used by the IndigoDC Orchestrator is included internally. It pointed towards the master branch in Github.

https://raw.githubusercontent.com/indigo-dc/tosca-types/devel-deep/custom_types.yaml

Due to the fact that the plugin may be tested with types that are not included in the master (i. temporary, certain release only, bugs etc.) we decided to expose the imports in the plugin instance configuration section.l