usage: comparing_latencies.py [-h] [--op-count OP_COUNT] configpath outputpath
positional arguments:
configpath
outputpath
optional arguments:
-h, --help show this help message and exit
--op-count OP_COUNT
usage: ./getCost.sh <solver_replication> <solver_ecc> <solver_panda>
You should read the solution files first and discard any latency or storage information that's not relevant. That should help with loading and resource consumption.
Formulations of interest:
For fixautosplit & flexbasic, number of splits per replica DC is 1.
For the others, read out from SPLITS_dc variables.
For others, read out quorums members based on R_fe_dc or W_fe_dc variables.
It's safe to infer the formulation type based on the name of the file. It will be prefixed with that name followed by a dash.
The user shouldn't have to supply the quorum system, this should just be inferred based on the solution name. s.startswith("fixautosplit-") should use basic quorums. Will list others once we finalize the formulations and I give them names (will happen soon).
Support needs to be added for non-basic quorums. You probably want an abstract quorum system class and pass in all replica latencies and have that determine the latency for any given request.
Right now for writes you always use read quorum then write quorum for writes. This should be conditional on whether flexible paxos is enable. If we don't use flexible paxos, write quorums should be used for both.
Make this a flag. While you're at it, using argparse might be a good idea.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--quorum-system", default="basic")
parser.add_argument("--op-count", default=50000, type=int)
parser.add_argument("configpath")
parser.add_argument("outputpath")
args = parser.parse_args()
or something.
I'm seeing lot's of errors like this:
got an error with task flexbasic-any-w3x-s2.5: Command '['/w/avail-run/comparing_latencies.py', '--op-count', '100000', '/mnt/cdata0/auto-fs-avail-results/Global/results-NAS-6-FAIL-1-id172224/solutions/flexbasic-any-w3x-s2.5.final', '/mnt/cdata0/auto-fs-avail-results/Global/results-NAS-6-FAIL-1-id172224/simulations/single/flexbasic-any-w3x-s2.5']' returned non-zero exit status 1
Read quorum size 0
Write quorum size 0
Access set ['az/centralus', 'az/centralindia', 'az/eastasia', 'az/koreasouth', 'az/southindia']
List of replicas ['az/canadacentral', 'az/eastasia', 'az/southeastasia', 'az/westus2']
Flexible paxos enabled True
read quorums [['az/eastasia', 'az/southeastasia'], ['az/canadacentral', 'az/westus2'], ['az/eastasia', 'az/southeastasia'], ['az/eastasia', 'az/southeastasia'], ['az/eastasia', 'az/southeastasia']]
write quorums [['az/canadacentral', 'az/eastasia', 'az/southeastasia'], ['az/eastasia', 'az/westus2'], ['az/eastasia', 'az/southeastasia', 'az/westus2'], ['az/eastasia', 'az/southeastasia', 'az/westus2'], ['az/eastasia', 'az/westus2']]
Running simmulation for specific quorum system...
Traceback (most recent call last):
Reading network latency file...
File "/w/avail-run/comparing_latencies.py", line 536, in <module>
main()
File "/w/avail-run/comparing_latencies.py", line 520, in main
sampleGet = GetRequestGenerator(networkLatencies, storageLatencies[0], frontend, listOfReplicas, quorumSystem, numberOfSplits)
File "/w/avail-run/comparing_latencies.py", line 325, in GetRequestGenerator
storageLatency = random.choice(readStorageLatencies[dc])
KeyError: 'az/canadacentral'
Might have something to do with trimming data for DCs not in use.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.