OVS is slow, cumbersome, and overengineered for our purposes. A nice alternative would be a VPP or XDP based switching framework which is much more lightweight and flexible. The hope is that this will result in saved CPU cycles and more accurate results.

Mininet is very heavyweight and introduces a lot of dependencies. It would be nice to move away from OVS and use a simpler topology manager and generator.

Some work has been done in the 4.19 kernel that influences our bufferbloat emulation. We need to investigate why we cannot reproduce our experiments in this environment any more.

Hi, I'm currently studying your code and I found your control pipeline working as the flow chart below. The node controller will start during the initalization of the environment and it will keep working as the pattern shown in the figure.

Please correct me if I'm wrong.

However, I'm just wondering while deploying rl algorithm like appo, ddpg, will TCP congestion control still work in the background? If so, will there be a confliction between function rtnl_qdisc_tbf_set_rate(ctrl_handle, (uint32_t) tx_rate, burst , 0); and TCP congestion control? Because according to my understanding, the essence of change the sending rate under tcp is changing the size of CWND. If both algorithms are doing the same thing, there seems to be a data race.

Thanks for your time for reading my issue. I'd appreciate it if you could give me some help.

The framework looks very interesting and can probably be very helpful. However, I am a little confused in terms of how to start with this project. Can you give some basic documentation about how to define topology (or use topology), how to implement RL algs on top of this framework, and how to collect results? I think it can be great if there can be a minimal example or such. Thank you!

Unfortunately, using the Linux tbf qdisc does not always work as expected. Depending on the rate this pushed in, and the size of the buckets, rate limiting outcomes may vary substantially. It would be nice to try out a more reliable and accurate rate limiting option for interfaces. Something like hbm for example.

prerequisite for #12

Hi, iroko team:
Thanks for your IROKO, it is powerful and easy to use. But when I try to run run_basic.py, if I choose a large number of timesteps, I encounter the following error. When I train my RL algorithm, I also encounter this problem if the number of episodes is too large. Do you know how to solve it?
Best wishes!

Investigate generating traffic by some sort of distribution (e.g. send a packet every x ~ N(5, 3) seconds. Where x is a sample from the normal distribution. Since getting actual traffic data can be challenging, previous research supposedly make assumptions of what the distribution looks like for sending patterns. Being able to define such distributions & patterns in Iroko could be quite useful for the ML part of Iroko

Currently, Goben measures the traffic stats and put them in a log file. We need to convert this file (or instead of converting, just send the statistics as a stream directly) to an output stream.

A sample pattern can be found in
https://github.com/dcgym/iroko/blob/master/dc_gym/inputs/fattree/stride4_data

This can also be done as a python wrapper of the traffic generator.

Hey, nice work!

I'm trying to play with this and reproduce results from https://arxiv.org/abs/1812.09975. I followed the installation instructions and got run_basic.py working, but sudo python run_ray.py --tune fails with the following exception:

Error creating interface pair (s1-eth2,h1-eth0): RTNETLINK answers: File exists

This happens without --tune as well. I've tried doing sudo mn -c, but it doesn't help. Here's the entire log file: https://pastebin.com/3gvZwNTT.

Any help appreciated. Thank you!

Add a baseline NUM solver and a random agent to compare their performance against the trained models.

The Mininet install is horrendously bloated and causes all sorts of dependency issues. We only need minimal features. There must be a better way to install the Python code.

Stop support for Python2. There is hardly any reason why Python 2 is still required.

Is it possible to replace mininet default switch (ovs switch) with some custom switch? For example can we, collect the metrics from the switch and feed to the AI based learner? Is there any kind of interface in this project?

Currently, the Goben can only generate traffics for a certain period of time, which could be configured by the totalDuration flag. It's probably a better idea to add an option to be able to generate the traffic for an unlimited amount of time

See title

The fattree code is very messy and relies on a lot of OVS features. It should be cleaned up.

• Dumbbell
• Nonblocking
• Fattree
• Parking Lot
• Jellyfish
• DCell
• BCube

It is important to measure the scalability limitations of the emulator. We need to test the scalability properties on a 8-core and 60-core machine by iteratively scaling up the number of hosts until throughput decline is measured.

An example of useful statistics can be found here:
https://github.com/flowgrind/flowgrind

Hi:
Iroko works well but when I try to plot RTT by

If transport == "tcp":
analyze_pcap(rl_algos, tcp_algos, plt_name, runs, data_dir)
plot_barchart(algos, plt_stats, plt_name)

in plot.py.
But it seems no tcp trace and shows 
result_folder /home/kong/iroko/results/kong-VirtualBox_4/tcp_run0/ppo
sh: 1: tcptrace: not found.
How can I solve this problem?
Best wishes.
Wu

• TIMELY
• DCTCP
• D2TCP
• D3
• PCC
• BBR
• TCP New Vegas
• Copa

Hi:
Thanks for your excellent work. I try to run run_openai_baselines.py in baselines branch. But I met some errors. I want to know if run_openai_baselines has completed and works correctly, or is it a problem in my own operation.
Best wishes!
Wu

How do I add a custom RL algorithm? is there a file I need to modify ? a class that I need to implement? Any documentation will be great! I am new to gym and ray/rllib.

The goal here is to let the traffic manager (controller) be able to adjust the traffic (bandwidth) limits based on the action (a vector with each entry as the limit to each host).

One way to do this is that: Write an independent "traffic mediator" to listen to "action packets" from traffic manager, and send a packet containing the specific traffic limit to each host (goben interface). And this is in real-time. It can be done by using netcat

• Maybe there are better designs?

• DDPG
• PPO
• REINFORCE
• Linear Policy Iteration
• A3C
• APE-X
• IMPALA
• TD3
• Soft Actor-Critic
• Curiosity-Driven Learning

Currently, the entire agent is prototyped in Python, this includes the network and data collection operations. They should be written in a more efficient language to reduce operational latency and resource usage. A nice way to compose the framework is to use Arrow to glue all modules together. Data can be collected in C and shared with the Python agent using the Arrow interface.

Checkpointing the collected bandwidth, queue, and reward stats every 1000 iterations causes a significant slowdown and affects the behavior of the agent. Checkpointing should be asynchronous from the agent's actions

Hey Fabian, I have a few questions about the internals of the framework that I'm having trouble wrapping my head around. Hoping to get some clarify on the following:

1. I'm trying to understand the interaction between the tx_rate being enforced based on the policy by BandwidthController and how cwnd gets changed by the native congestion control algorithm in the TCP setting. I see that the way we enforce the action taken by the policy is by capping the bandwidth allocated to the interface (https://github.com/dcgym/iroko/blob/master/dc_gym/control/bw_control.c#L55). This makes sense for UDP, but in the TCP scenario, isn't the underlying congestion control algorithm also doing its own thing and potentially interfering with the bandwidth control?
2. Have you been able to, or is there a way to simulate bursts of traffic to simulate a more realistic setting? Based on what I understand from iroko_traffic.py (https://github.com/dcgym/iroko/blob/master/dc_gym/iroko_traffic.py#L174), we generate a constant stream of traffic from src to dst right?

Thanks!

Instead of sampling hosts from centralized switches, hosts should notify the arbiter of intention to send a flow to a location. Path can be inferred by the arbiter. This is much more lightweight.
In addition, a queue sampling approach such as SIMON (NSDI'19) can be used.

there appears to be an extra * in line 38
matplotlib = "3.3.**"

Calling into Linux tc is a clunky method to collect data from interfaces and regulate sending rate. A better way is to directly write a C program that uses netlink.

Hi,

Your works are pretty clever! However, I met some problems when trying to observe the result folder. The host server.out&err /client.out&err /ctrl.out are all empty. And for host ctrl.err it says "no such file or directory" anyway for PPO/PG/DDPG/DCTCP agent(those agents are used so far for my current work.) The program seems to be running smoothly.

Any ideas to solve this? I tried to even commit _start_controller func but still nothing to show in those files but the .csv files have plenty of data in it, which I guess generated from goben?

Mininet, Ray, and the Iroko platform log and print to stdout completely independently, which causes a big output mess. Debugging is also complicated. It would be nice to unify all logging output under a single framework.

Trying to use DCTCP in the current emulator results in strange congestion behavior. It seems that DCTCP bypasses the qdisc which rate limits bandwidth and is pushing beyond the maximum possible amount. This could be caused by Mininet or the Linux kernel. Unclear so far.