ravinet / mahimahi Goto Github PK

HTTP Proxy should use https://github.com/keithw/ourtube/blob/master/http_parser.hh to get an ENTIRE request (request headers and complete request body) from the client.

Only when it has the WHOLE request should it then sent it along to the server (as a string).

Everything else unchanged.

Man pages currently give http://mahimahi.csail.mit.edu (should be mahimahi.mit.edu)

Separate new program

• opens two tap devices
• ferries packets from each side to the other one

When I'm connected to MIT Wi-Fi, /etc/resolv.conf contains other nameservers. These are not available inside replayshell but they're tried anyway, resulting in hostname lookup times of 5+ seconds. Therefore, page load times of URLs with hostnames are much higher than URLs with direct IP addresses. This can be manually fixed by making sure that /etc/resolv.conf is blank.

Something has changed in the kernel treatment of veth devices. (The packetshells do still work.)

It should be uid of the user running replayshell and not www-data.

DelayShell should search for a pair of IP addresses that are unused for the ingress and egress, rather than hardcoding two specific IP addresses. This will allow us to nest one program inside another (e.g. cellsh inside httpreplaysh).

The IP address search should start at 100.64.0.1 and end at 100.64.0.255 (Carrier-grade NAT range).

We should use the getifaddrs() function to list the interfaces on the machine and their addresses.

We should also pick a name for the egress interface that's not just "egress", because that might be taken (if multiple users are running delayshell at the same time). Probably just doing egress + pid is a good idea.

Steps to reproduce the problem:

1. git clone https://github.com/ravinet/mahimahi

2. cd mahimahi

3. git checkout recordshell

4. ./autogen.sh && ./configure && sudo make install

5. delayshell record

6. Run:
   wget http://web.mit.edu/anirudh/www/irtf_iab-ccirtcpaper28.pdf -O /tmp/1.pdf
   wget http://web.mit.edu/anirudh/www/irtf_iab-ccirtcpaper28.pdf -O /tmp/2.pdf

   1 and 2 have the same size (and the same size as the original file on the web) but differ in their contents.

Like FileDescriptor does

Overwriting old files is not an option.

Ensure that it compiles with Ubuntu 12.04.

It doesn't need to be a cellular link under all circumstances.

Delayshell needs to ferry DNS traffic between the container and the host. We should:

• outside the container, listen on UDP and TCP ports
• inside the container, listen for UDP and TCP traffic on port 53
• inside the container, send the UDP datagrams and TCP traffic to the egress IP address (on the ports previously opened)
• outside the container, receive traffic and resend it to 127.0.0.1, port 53 (to the local nameserver)
• outside the container, proxy the reply back to the source

If I run the following sequence of commands:
delayshell 2
screen
Ctrl-A Ctrl-D (detaches from screen)
delayshell 2
ifconfig

Now, in a new terminal:
screen -r
ifconfig

Both ifconfigs show an "ingress" tun device. Wondering if this will lead to trouble later on. Maybe not because they are in separate network namespaces?

On an additional cosmetic note, screen -r doesn't return our cool "[delay 2]" prompt, and uses the standard bash login prompt instead.

Use Selenium or equivalent.

Venet is two tun devices directly connected, where one is inside and one is outside. This way, we don't have to deal with ferrying packets from tun device (and adding delay) since delay is 0.

The Internet claims that we can't expect a 1:1 correspondence between waitable events from child processes and SIGCHLD signals.

http://stackoverflow.com/questions/8398298/handling-sigchld

http://stackoverflow.com/questions/10434218/libevent-signal-handling-sigchld-specifically

Remembering to rename everywhere it's mentioned

used fork + unshare instead

Seems like this was a workaround to silence -Weffc++. Wonder what the actual solution is.

Live graph!

HTTP Proxy should construct each HTTP response (response headers and response body) before sending to the client.

This will require parsing chunked bodies, which Ourtube hasn't handled yet. Use the HTTP specification (RFC 2616).

http://tools.ietf.org/html/rfc2616

(1) Edit Makefile.am to make the program get installed setuid root. (Add chmod u+s $(bindir)/delayshell as an install hook?)

(2) Make the child process drop privileges immediately after executing unshare.

(a) Change group first.
(b) Then change UID.
(c) The Internet will have instructions on how to do this.

Wrapping call to eventloop, parent should

(1) Create ChildProcess
(2) (Inside ChildProcess: drop privileges, then run eventloop)
(3) wait for ChildProcess to finish

umask should be user readable.

(This causes an occasional exception on quit. If we know that one of the veth devices in the pair is inside the container, we do not have to clean it up ourselves.)

I can lock it up with netperf in a matter of seconds on
3.11.0-19-generic #33-Ubuntu SMP. Which is too bad as toke is making
huge progress with netperf-wrapper and a new gui...

I setup two delay shells, then start netserver in one, and try to connect to the
other. Fire up netperf -H 10.64.0.2 and inside of a few thousand packets, it
hangs...

[delay 10 ms] d@nuc:~/git/mahimahi$ ifconfig
ingress Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:100.64.0.2 P-t-P:100.64.0.1 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
RX packets:6627 errors:0 dropped:0 overruns:0 frame:0
TX packets:4376 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:9929172 (9.9 MB) TX bytes:248836 (248.8 KB)

other window:

d@nuc:/git/netperf-wrapper$ delayshell 10
[delay 10 ms] d@nuc:/git/netperf-wrapper$ netperf -H 100.64.0.2
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
100.64.0.2 () port 0 AF_INET : demo

(hello, unsharenet, forktest, simple-ferry)

Including where they're mentioned in Makefile.am, and all their source files that aren't used elsewhere.

Same concept as delayshell but uses delay 0 (still uses tundevices and ferryqueue).

Make hello world compile nicely with autotools stealing from https://github.com/anirudhSK/explicit-vp8 if necessary.

One tap device is inside the special network namespace and one tap device is outside of it.
We are making a bridge.

Probably using NAT

Other options:

• routing
• bridging

Want to use dnat and then receive UDP datagram, obtain original destination, and handle request in new thread (create socket, send datagram, receive response, ferry back to original source). This is similar to how we handle UDP DNS.