Comments (5)
mitake
commented on July 19, 2024
2
I'd like to report my experiment of etcd 5022: etcd-io/etcd#5022
| w/ or w/o Namazu process inspector | Reproducibility |
|---|---|
| w/o | 0% |
| w/ | 2.7% |
Both of a number of test running in the above experiments is 1000.
Parameters of explorer policy:
explorePolicy = "random"
[explorePolicyParam]
procPolicy = "dirichlet"
from namazu.
AkihiroSuda
commented on July 19, 2024
Tried to reproduce ZOOKEEPER-2212 with several configs.
All the experiments are done on my local lenovo pc. (Xeon E3-1220 v3 * 4, 8 GB RAM)
| EQ Config | #CPU assigned | #Exp | Reproducibility | #Pattern@1000 exp | Notes |
|---|---|---|---|---|---|
| None | 4 | 5,000 | 0% | 156 | Data is from FOSDEM slide. |
| Ether | 4 | 1,000 | 21.8% | 573 | Ditto. With latest EQ + 1 CPU, reproducibility grew to about 50%. |
| None | 1 | 1,000 | 0% | N/A | |
| None + SCHED_BATCH | 1 | 1,000 | 0% | N/A | |
Proc(mild{UseBatch:true})(SCHED_BATCH + random nice values) |
1 | 5,000 | 0.7% | 634 | 0.08% experiments failed due to timeout |
Proc(mild{UseBatch:true}) |
4 | 5,000 | 0.32% | 548 | No experiment failed due to timeout |
Proc(mild{UseBatch:false}) |
1 | 5,000 | 0.26% | 914 | 90% experiments failed due to timeout |
mild{UseBatch:true}provides better reproducibility thanmild{UseBatch:false}, but not so good as the Ethernet inspector.mild{UseBatch:false}provides better pattern growth, but not useful for ZOOKEEPER-2212 due to too many timeouts.- Proc(
extreme) likely to cause starvation on single CPU, so I did not experimented. - Proc(
dirichlet) hits the bug mentioned in README.
from namazu.
AkihiroSuda
commented on July 19, 2024
Also tested ZOOKEEPER-2137 with the latest ZooKeeper (just 50 times on 4 CPUs):
| EQ Config | #CPU assigned | #Exp | Reproducibility | #Pattern@1000 exp | Notes |
|---|---|---|---|---|---|
| None | 4 | 50 | 2% | N/A | - |
Proc(mild{UseBatch:true})(SCHED_BATCH + random nice values) |
4 | 50 | 16% | N/A | - |
Proc(mild{UseBatch:true}) |
1 | 50 | 2% | N/A | - |
This reproducibility is useful enough (on 4 CPUs).
The process inspector works well with ZOOKEEPER-2137, although not with 2212.
I guess this is because ZOOKEEPER-2137 runs longer (> 1 min) than 2212,
i.e., much more chances to work are given to sched_setattr().
I keep this issue ticket open for discussion.
PTAL @mitake
from namazu.
AkihiroSuda
commented on July 19, 2024
Evaluated some YARN (apache/hadoop@4e4b3a8 ) tests using 13aa33b (mild{UseBatch:true), on AWS t2.large (2 CPUs assigned).
Tests are executed 100 times with/without Earthquake.
Note that this version of Earthquake does not contain an optimization (#146)
| Test | Reproducibility(without EQ) | Reproducibility(with EQ) |
|---|---|---|
| YARN-4548(RM/TestCapacityScheduler) | 11% | 82% |
| YARN-4556(RM/TestFifoScheduler | 2% | 44% |
| YARN-4168(NM/TestLogAggregationService) | 1% | 8% |
| YARN-1978(NM/TestLogAggregationService | 0% | 4% |
| YARN-4543(NM/TestNodeStatusUpdater) | 0% | 1% |
from namazu.
AkihiroSuda
commented on July 19, 2024
I found sometimes it is better to apply Namazu (formerly named Earthquake) for stress process rather than Hadoop mvn process.
Testcase: YARN-5043 (RM/TestAMRestart) (apache/hadoop@06413da
) using 8e4f268 (mild{UseBatch:true), on AWS t2.large (2 CPUs assigned). Done 100 times.
Stress: stress --cpu 2
| Running stress? | Namazu applied for | Reproducibility |
|---|---|---|
| N | None | 16% |
| Y | None | 12% |
| N | mvn | 7% |
| Y | stress | 30% |
TODO:
- reevaluate other YARN tests with
stress - scientific, and reliable analysis
from namazu.
Related Issues (20)
- Apply Namazu to logcabin
- Apply Namazu to TiKV, TiDB HOT 2
- Flaky test? (endpoint) HOT 2
- Project Rename: Earthquake --> Namazu
- Use package manager (govendor or something else) HOT 2
- LOW HANGING FRUIT: make goreportcard happy
- `nmz inspectors proc -cmd ...` lacks stdio HOT 1
- Print a warning if `sched_setattr` returns EPERM
- Flaky test: nmz/endpoint HOT 2
- Enable fs inspector in container mode HOT 3
- [doc] Mention and promote Namazu Swarm: CI Job Parallelizer built on Docker and Kubernetes
- Can not get mflag:Docker has removed pkg/mflag HOT 3
- runtime panic of process inspector HOT 2
- namazu has error when using O_DIRECT HOT 3
- scientific paper HOT 1
- restarting TiKV with file inspector hangs up forever HOT 9
- How to enable/disable inspector explictily HOT 4
- how to stop inspector HOT 2
- Some problems met when installing the namazu. HOT 6
- Does NAMAZU support CentOS HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from namazu.