Initial setup:

• Machine A security group: SG1
• Machine B security group: SG2
• SG1 members: A's IP
• SG2 members: B's IP
• SG1 rule: Allow all members of SG1 to access UDP port 8000
• SG2 rule: Allow all members of SG2 to access UDP port 9000

Tests that:

• machine A can't speak with B on UDP port 9000.
• machine B can't speak with A on UDP port 8000.

Change setup:

• SG1: members: add B's IP
• SG2: members: add A's IP

Test that:

• machine A can speak with B on UDP port 9000.
• machine B can speak with A on UDP port 8000.

Change setup:

• SG1: members: remove B's IP
• SG2: members: remove A's IP

Test that:

• machine A can't speak with B on UDP port 9000.
• machine B can't speak with A on UDP port 8000.

We must dynamically tweak thread priorities to have a nice packet throughput

This should ease debug and can be used in API tests

Those tests must be added to check if changing security groups change all rules

• Test that deleting a SG block all traffic, adding it again allow traffic, etc ... x10
• Test that overwriting a whole SG will change rules (adding new rules, removing old ones and keep the same rules), again and again ... x10

We would like to check if exotic protocol filtering like sctp are managed by Butterfly.
Butterfly will also be able to manage SCTP port ranges.

Note: this issue has already been 95% done with tests/scenario_34/ except that setup/test is not done

Initial setup:

• VM1 configured on vni 42 with security group sg-1
• VM2 configured on vni 42 with security group sg-1
• sg-1 has no rules

Test that:

• Test SCTP connection VM1 -> VM2: KO
• Test SCTP connection VM2 -> VM1: KO

Change setup:

• Add rule to sg-1 allowing all protocols (-1) from all hosts

Test that:

• Test SCTP connection VM1 -> VM2: OK
• Test SCTP connection VM2 -> VM1: OK

Change setup:

• Remove rule to sg-1 allowing all protocols (-1) from all hosts

Test that:

• Test SCTP connection VM1 -> VM2: KO
• Test SCTP connection VM2 -> VM1: KO

Change setup:

• Add rule to sg-1 allowing only IP protocol 132 (sctp)

Test that:

• Test SCTP connection VM1 -> VM2: OK
• Test SCTP connection VM2 -> VM1: OK

The goal is to be able to get a server build ID when running a --help command.

This will permit to easily identify the commit from where the server where built

Just a reminder

Currently, VNI is fixed a NIC creation.
If needed, we can add a call to dynamically change VNI (NicUpdateReq)

The goal is to be able to launch all Butterfly tests using Jenkins (or other tool) at each Pull Request

typedef std::shared_ptr<struct pg_brick> Brick is not very explicit, because we don't understand we're dealing with a share_ptr, we need to find a more explicit name(BrickPtr).

BTW: in C++ typedef std::shared_ptr<struct pg_brick> Brick and
typedef std::shared_ptr<pg_brick> Brick is the same.

check that iperf/iperf3 TCP and UDP tests works (not really for benchmarking)

--dpdk-help is broken

The goal is to provide some basic tests concerning security groups. We must test that:

• TCP/UDP communication is blocked when SG does not contain any rules (by default).
• TCP/UDP communication is OK on a specific open port and in closed on other ports.
• Machine A don't have any SG open but can communicate in TCP/UDP with the other machine B who have an open port (to test stateful firewall).
• Same scenario but then we test that removing the rule don't allow a new TCP/UDP communication. We then re-open, we test that's open, we re-close, we test that's close again, etc.. 10x
• Full open SG between two machines: we start a TCP communication then we remove all rules: does it cut connection or wait end of communication ?
• Have two machines A and B with the same SG listening on ports 8000 and 9000 but SG only open 8000. Machine C can speak with A and B on 8000. When then remove the rule from SG: test that's now blocked and we add a new rule to open on 9000: test that C can access to A and B on 9000.

We currently have -r option for more details, we should have a --version to show butterfly's version

We would really need this feature from Packetgraph and integrate it in butterfly.
See vxlan-soe issue and IP frag issue

Graph::nic_add create a lot of new brick, using shared_ptr, but there is no check to know if
the creation has been successful.
we should use that: http://www.cplusplus.com/reference/memory/shared_ptr/operator%20bool/

We (at Outscale) have physical machines with Intel NICs to test DPDK capabilities.
We would like to easily run benchmarks tests without always hacking our test machines.
Script will build a packaged version of Butterfly and install it on remote machines before running benchmarks on the two machines.

If a SG contains one rule and this rule is removed. A new filter is built with this rule: ( )

This rule does not compile in NPF (but a rule containing only spaces build).

In scenarios, we should add a test to check if you can communicate in TCP and UDP between two VMs:

• On the same butterfly
• On separate butterfly

We should also make the same tests with IPv6

Instead of always installing the same software on test vm, we should:

• update all packages
• install scappy
• install nc
• install ethtools
• speedup vm boot
• ???

And then update butterfly scripts (and packetgraph)

useful for debuging

We should clean script usages with options and clear --help (like in benchmarks)
We should do this in:

• tests/scenario_*
• api/tests

• Maybe classify tests in different directories ?

This scenario fails:

• Connects two virtual machines on the same butterfly on the same network (vni)
• Make machines ping
• Kill machine 1
• Add nic 1 again
• Start machine 1
• Make machines ping

network_connect 0 1
server_start 0
client_add_nic 0 1 42
client_add_nic 0 2 42
qemu_start 1
qemu_start 2
ssh_ping 1 2
ssh_ping 2 1
qemu_stop 1
client_add_nic 0 1 42
qemu_start 1
ssh_ping 1 2
ssh_ping 2 1
qemu_stop 1
qemu_stop 2
server_stop 0
network_disconnect 0 1
return_result

Here we would like to test rules based on members of security groups.

Test setup:

• Machine A in SG1
• Machine B in SG1
• Machine C in SG2
• SG1 add a rule allowing members of SG2

Check that:

• C can communicate with A
• C can communicate with B
• A can't communicate with C
• B can't communicate with C
• A can't communicate with B
• B can't communicate with A

Change setup:

• SG1 add a new rule allowing members of SG1

Check that:

• C can communicate with A
• C can communicate with B
• A can't communicate with C
• B can't communicate with C
• A can communicate with B
• B can communicate with A

Change setup:

• Remove B from SG1

Check that:

• C can communicate with A
• C can't communicate with B
• A can't communicate with C
• B can't communicate with C
• A can't communicate with B
• B can't communicate with A

todo:

• antispoof patch
• rule building patch
• remove checks
• fix protobuf documentation
• tests with:
  • 3 ipv4
  • 3 ipv6
  • 2 ipv4 + 2 ipv6
• test single IP update through NicUpdateReq:
  • ipv4 A -> ipv4 B -> ipv4 A
  • ipv4 A -> ipv6 B -> ipv4 A
  • ipv6 A -> ipv4 B -> ipv6 A
  • ipv6 6 -> ipv4 6 -> ipv6 6
• test multiple IP update through NicUpdateReq:
  • 2 ipv4 A,B -> 2 ipv4 C,D
  • 1 ipv4 A + 1ipv6 B -> 2 ipv4 C,D
  • 2 ipv4 A,B -> 1 ipv4 C + 1ipv6 D
  • 1 ipv4 A + 1ipv6 B -> 1 ipv6 C + 1ipv4 D
  • 2 ipv6 A,B -> 2 ipv6 C,D

Option already exists in API (sniff_target_nic_id) and should be implemented.
We may use a combination of hub + diode + vhost bricks:

When exiting (normal or by signal), Butterfly should clean allocated hugepages by invoking pg_stop()

We currently write butterfly server PID but we don't really use it.
We don't want to disallow launching several butterfly (usefull for testing) but we should warn the user through logs instead of failing dpdk init.

We should add set -e in tests/function.sh and check if errors are expected or not.
We should also do this in api/test

Vhost-user performances are quite poor as it still need a packet copy between guest and host.
This ticket is a reminder to work in this issue (which is heavily related to packetgraph)

Maybe we can replace dpdk options by defaults ones

nic_id in function request is never used.

We already have some tests with tcp/udp ports (#45), here we add a few more tests:

• 2 machines A and B on different security groups. A only allow host B (test TCP/UDP). We then remove this rule (test TCP/UDP) and add a new one allowing everyone to access A (test TCP/UDP). We then remove this rule and allow every one except B to access A (test TCP/UDP).
• Same test but instead of using SG members instead of CIDR in rules

In graph.h there is struct using the linux style:
struct my_struct {...};
struct my_struct variable;

We should use google style instead:
struct MyStruct {...};
MyStruct variable;

this branch (last test) make VM machine stuck.

network_connect 0 1
server_start 0

client_add_nic 0 1 42
client_add_nic 0 2 42
qemu_start 1
qemu_start 2
ssh_ping 1 2
ssh_ping 2 1
client_del_nic 0 1
client_del_nic 0 2
ssh_no_ping 1 2
ssh_no_ping 2 1
client_add_nic 0 1 42
client_add_nic 0 2 42
ssh_no_ping 1 2
ssh_no_ping 2 1
qemu_stop 1
qemu_stop 2
qemu_start 1 <------ VM stuck here
qemu_start 2
ssh_ping 1 2
ssh_ping 2 1

server_stop 0
network_disconnect 0 1
return_result

VM don't produce any log, vnc socket is not created, we can't ssh it nor connect to console.
Maybe a particular state in message passed through socket to the new VM.

We detected a Butterfly crash located in vtep de-capsulation (see packetgraph's bug)

We must check when VM disconnect so we can remove vhost nic and it's branch

rebase NPF to the last version in packetgraph

The idea is to allocate bricks in the processing thread.
This should locally allocate memory for better performances (numa).

iperf3 by default fragments IP packets (8k) in UDP. Firewall probably don't let those packets pass.
We can test it by removing -l 1400 from iperf3 tests

Note: if we support vhost offloading, fragmentation should not occurs anymore

The goal is to run Butterfly benchmarks on Outscale's test servers for each commit on master.

• All tests results must be kept to see evolutions
• Last (or all) result(s) should be graphed (gnu plot ?)
• Generated graph should be uploaded and overwrite the last one
• README.md has this image