efficient / catbench Goto Github PK

Is this a problem? Would be easy to fix now...

The Python library's JSON implementation is probably at fault—unless we're forgetting to sort somewhere—but it's kind of annoying. For instance, when writing scripts that use arrcsv, one has to use a loop to ensure the output comes back in the expected order.

So... 200 and up are not so hot. While we're at it, revert the addition of the multiplier argument to avg_data.pl.

This is necessary because some things (e.g. the base 64–encoded Kconfig) are too long to pass in as command-line arguments.

This prevents their being used by the graphing script, so the axis labels aren't human readable.

It disagrees with the comment, and only recording the first period appears nonsensical...

This would ensure we don't run into constant-factor troubles in the future.

Sanity check... or insanity check?

They're computed as contention/allocation, but since contention is the baseline in this case, we believe they should actually be 1-(allocation/contention), which is mathematically slightly different.
The existing ones are probably close enough for graphing purposes, but long-term, this needs to be fixed.

since the latter action must be done as root. This means that ^C'ing the script can leave the server running.

What a silly sanity-check!

This could be rectified by supporting a command-line switch for exponents instead of percentages

This is on 362318f atop master.

This appears to be a regression: we went from:

https://drive.google.com/a/andrew.cmu.edu/file/d/0Bx1O0RDCSN1rSDFYV2xZR09Kdmc/view and https://drive.google.com/a/andrew.cmu.edu/file/d/0Bx1O0RDCSN1rY1F3aGt1cXJUVzQ/view

to

https://drive.google.com/a/andrew.cmu.edu/file/d/0Bx1O0RDCSN1rY0NNR0xURWE0Mkk/view
https://drive.google.com/a/andrew.cmu.edu/file/d/0Bx1O0RDCSN1rNUx6RENCMF9LMUU/view

and

https://drive.google.com/a/andrew.cmu.edu/file/d/0Bx1O0RDCSN1rTDhNcVNkOTNPSFE/view
https://drive.google.com/a/andrew.cmu.edu/file/d/0Bx1O0RDCSN1rUHd1elFvWDhiRnM/view

We currently suspect:

commit 498b86b (refs/bisect/bad)
Author: Hyeontaek Lim <[email protected]>
Date:   Fri Jul 29 01:48:53 2016 -0400

    network_rtt: Control the run time of experiment stages explicitly

Otherwise, you need to tweak it when using contender_duration so that the baseline series aren't incomparable to the control and experimental ones.

Also please properly sort the legend

This makes it really hard to deal with these files because jq and Vim also tend to choke on them. Although the coreutils are mostly tolerant of these files, it's still unclear how to easily/quickly use them to extract the logs because base64 adds line breaks that jaguar saves as escapes; normally it then interprets them on the query side, but this is actually hard to do efficiently with other tools, and base64 -d doesn't handle escapes itself.

Then of course there's the obvious problem that our data extraction/processing scripts barf when jaguar does. Currently I've been working around this by replacing the network_rawnums/timescale_and_cdf superhero dream team with:
$ grep -F "Completed after:" ><temp 1>
$ cut -d" " -f4 <temp 1> ><temp 2>
$ tail -n+"$((1 + n * 30000000))" <temp 2> | head -n30000000 | tail -n20000000 | <...>

• AND -

$ ./largescale_and_cdf <...>

Its proximity to line 98 seems to mean that the lockserver may or may not be in the cgroup (albeit with the same mask) with the contenders. Breaks our control?

Repro:
$ ./square_evictions -n10000 -p1 -c100 -e100 -r -ia 16384 # runs normally
$ ./square_evictions -n10000 -p1 -c100 -e100 -r -ia 16385 # prints "Beginning {un,}saturated passes" ad infinitum

Then we wouldn't fail on e.g. 49 when e.g. you try to check it out from a different branch but forget to drag along template.json...

Currently, it just plods along working on the rest of the points for the next two hours, then calmly reports it couldn't shoehorn the data into the Jaguar file at the end.
I'm in ur results files, wastin ur time.

Might just work if you invoke via driver program, but try to figure out whether this logs enough info to be replicabile.

Also, do we want to commit usleep?

Setting the trash working set size to 102400 (1 GB) on dog results in the first trash process—the one sharing a core with the lockserver, incidentally—never printing ENDLAP002 for some data points! Does it crash? Is it just running extremely slowly, and does waitforexit fail to actually do its job? (Although in the latter case, wouldn't we see it printed later on, during some other trial?)

There doesn't seem to be enough info in the logs to discern...

It took me a while to figure out that the sole problem was a missing linux-4.7.tar.gz. :(

This will break lookups of the command-line arguments' measurement units, among other things. This may need to be done differently depending on the decision made in #2

Where's our race...?

There are a whole lot of function variables that aren't declared local, and some of our conditionals could probably still use unsets at the end.

In recent runs on the cachenice_mica2 branch, the contention case was exactly the same as the allocation one!
This bug was introduced by commit 6754023 on line 267 of network_rtt: notice how the line that launches contention is identical to the one for allocation.

To quickly tell whether a Jaguar file $file is afflicted, you can run:

$ git log --oneline "`jaguar/jaguar get "$file" meta.commit | cut -d" " -f1`" | grep 6754023 || jaguar/jaguar get "$file" meta.patch | base64 -d | grep '^+.recordtrial contention [^ ]\+ [^ ]\+ [^ ]'

No output/nonzero status means you're good; otherwise, you probably can't trust the contender numbers from that run.

This is on 362318f atop master. We're probably going to have to examine the data more closely and figure out what could be causing the variance.

We need to autodetect the number of ways on the machine.

to prevent them from causing unnecessary context switches. However, Perf is probably a much bigger concern.