remove tasks associate to client/server log backups because currently they are empty and in large experiments this requires copy thousands of empty files unnecessarily.

Currently calling ./cli.py prepare from the command line applies the default configuration of the broker (qdr complete graph of size 4). We don't want to run in this situation :

./cli.py deploy rabbitmq
./cli.py destroy
# The following will install qdr instead of rabbitmq (usually we don't want to change the bus when iterating)
./cli.py prepare 

The guilty lines are here :

https://github.com/msimonin/qpid-dispatch-xp/blob/280cdb613864b131862e8da8e92528d7999baa81/synthetic/ansible/roles/ombt/tasks/deploy.yml#L2-L7

Basically we create n^2 files where n is the number of ombt-agent.
We should consider reducing this number to the bare minimum (e.g 1 file for the controller)

See http://click.pocoo.org/6/setuptools/#setuptools-integration

the iteration string used to generate dir or log names is (partially) generated twice.
the main function test_case may get this string and avoid to generate another one.

TASK [common : Mount /run] **************************************
[WARNING]: Consider using mount module rather than running mount

Support a campaign with exception by try/except/finally

Currently we send ombt_confs to ansible and it looks like

{
"controller": [list of confs for every controller],
"rpc-server": [list of confs for every controller],
"rpc-client": [list of confs for every controller]
}

We could send:

{
"inventory_hostname1": 
"inventory_hostname2":
}

and load only the part that a node is interested in. This should increase the speed of the deployment and avoid any skipped tasks.

see https://github.com/ajssmith/collectd-qdrouterd

qpidd refers to the qpid broker not qpid-dispatch-router.
In order to not confuse anyone, use qdr instead (command line/internal roles...)

Our current workflow is:

• given a set of parameters: params1, params2,...
• Deploy, bench with param_{i}, destroy and deploy, bench with params_{i+1}

In order to scale fast we probably wants to avoid destroying everything between two rounds.
I propose to start thinking/writing a campaign-like script that allows to incrementally scale a test_case.

@avankemp until now the ombt version is hard-coded. We'll need to parameterized it to allow custom version to be used before they land in ombt/master (initial use case is kgiusti/ombt#13).

• if throughput is given then pause is set to nbr_clients/throughput.
• if duration is given then nbr_calls is set to duration/pause

see https://github.com/influxdata/telegraf/blob/master/plugins/inputs/system/NETSTAT_README.md

This should be added at least for the buses (control and test bus).

External functions from cli are supported for tc1 and tc2 only. In order to be consistent all test cases should be supported otherwise the execution of campaigns takes default values.

Step to reproduce

./cli.py deploy (whatever the provider is)
./cli.py test_case_1 (any option)

ombt-controller tasks fails when getting the benchmark result.

Step to fix

The newest version of python-docker is >=3.0.0 seems to make the ansible docker_container module fails. But here we don't constraint the installation to be <3.0.0 (could use 2.7.0).

and makes this fail:

Currently we bind ombt agents in a round-robin fashion on all the bus agents.
Using several bus agents leads to have several clients and server bound to the same agent.
In order to force messages to go through the bus we'll like to bind clients to a subset of agent and servers to different subset of agents :

Example:

clients -> router1 <-> router2 -> servers

The following could go to the conf.yaml under a registry key.

https://github.com/msimonin/qpid-dispatch-xp/blob/280cdb613864b131862e8da8e92528d7999baa81/synthetic/tasks.py#L230-L235

According to :

https://github.com/msimonin/qpid-dispatch-xp/blob/master/synthetic/ansible/roles/telegraf/templates/telegraf.conf.j2#L40

we don't monitor the control-bus.
We should consider moitoring it.

E.G qdr options are hard coded in the tasks.py. This should be easily customizable.

One idea would be to put everything (the options for all supported broker) in the configuration file - the same way we are doing for the providers.

When executing the task 'mount' the following warning pops out:

[WARNING]: Consider using mount module rather than running mount

Currently telegraf database must be added to the configuration of the influxdb datasource manually.
This should be done by the script.

Not sure we need this

https://github.com/msimonin/qpid-dispatch-xp/blob/7760945810491b3bbada76be1aec447c751bb486/synthetic/ansible/roles/common/tasks/deploy.yml#L25-L26

Skip should be done in the except block, it seems that all parameters are skipped.
As a consequence, replaying an stopped campaign leads to replay all the parameters and not only the ones that haven't be done previously.

For now bus is static during a campaign. This feature would allow to run experiments with different bus during the same experimental campaign (and thus we could easily compare things during the post-mortem analysis)

Use also __future__ to ensure backward compatibility with py2

This could be declared as:

cluster:
  type: rabbitmq-cluster
  number: 3

We mainly focuses in RabbitMQ lately, it's time to reintroduce the qdr support.

#19 will track the progress and eventually be merged

Remove g5k and vagrant and provide a common allocate command on the command line

Enoslib allows to specify network_constraints (see [1]):
In the conf we could have a tc section corresponding to the object taken in input from the Enoslib:

Example:

tc: 
  enable: true
  default_delay:  20ms
  default_rate: 1gbit
  groups: [ombt-client, bus]

And a dedicated task: tc that can be called from the command line :

• cli tc enforce the constraints
• cli tc --reset reset the constraints

To ease the post-mortem analysis.

After #56 is resolved we can envision to sweep over different latencies during a campaign.

We are still running bling at some point.
Especially when something is wrong...

Currently we only test the complete_graph topology.
We'd like to test tree based deployment:

       +----r2
       |
r1 -------r3   
       | 
       +----r4

This corresponds to realistic scenarii where :

1. compute to conductor communication : calls on a single Target (report state).
   In this case the servers are on r1 and the clients are in r2 ... rn

2. neutron-server to agents communication push policies: fanout on a single Target.
   In this case the servers are on r2...rn and the client is in r1.

e.g : https://github.com/kgiusti/ombt/blob/master/ombt2#L1342-L1343

╰─$ ./cli.py test_case_1 --nbr_clients 1 --nbr_servers 1 --nbr_calls 10000 --pause 0.1 --call_type rpc-cast --timeout 7200
Usage: cli.py test_case_1 [OPTIONS]

Error: Invalid value for "--pause": 0.1 is not a valid integer

this is here :

Setting PAUSE=0.0 will fix that

Servers must have their clocks in sync (ombt requirements)

When dealing with a certain number of clients, collectd plugin throws a timeout. :

Condition('amqp:resource-limit-exceeded', 'local-idle-timeout expired').

This is probably due to the idleTimeoutSecondsto be too small (see qdrouterd.conf).
We could put idleTimeoutSeconds: 60. In both listener/connector in the qdrouterd.conf file.
It's located here :

For now we depends on https://github.com/msimonin/qpid-dispatch-generator to generate the qdr conf.
This was originally a toy project that could be now internalized alongside the code of the framework.

In other word this would make the framework depends solely on enoslib.