Comments (12)
Bodigrim
commented on May 30, 2024
No need to escape slashes:
> MYBENCH=`cabal list-bin streamly-benchmarks:Unicode.Stream`
> $MYBENCH -p '$0 == "All.Unicode.Stream/o-1-space.ungroup-group.unlines . splitOnSuffix ([Word8]) (1/10)"'
Using small input file: benchmark-tmp/in-10MB.txt
Using big input file: benchmark-tmp/in-100MB.txt
Using output file: benchmark-tmp/out.txt
All
Unicode.Stream/o-1-space
ungroup-group
unlines . splitOnSuffix ([Word8]) (1/10): OK (2.13s)
299 ms ± 26 msIn a more complex cases a sed incantation for correct escaping could be useful:
$MYBENCH -l | sed -e 's/[\"]/\\\\\\&/g' | while read name; do $MYBENCH -p '$0 == "'$name'"'; doneParsing patterns is completely up to tasty. It would be nice to get a more specific parsing error indeed.
from tasty-bench.
harendra-kumar
commented on May 30, 2024
It works if the benchmark name is specified directly on the command line as you did. But it does not work if the benchmark name is expanded from a shell variable. For example:
$ MYBENCH=`cabal list-bin streamly-benchmarks:Unicode.Stream`
$ BENCH_NAME="All.Unicode.Stream/o-1-space.ungroup-group.unlines . splitOnSuffix ([Word8]) (1/10)"
$ $MYBENCH -p '$0 == "'$BENCH_NAME'"'
Using small input file: benchmark-tmp/in-10MB.txt
Using big input file: benchmark-tmp/in-100MB.txt
Using output file: benchmark-tmp/out.txt
option -p: Could not parse pattern
Usage: Unicode.Stream [-p|--pattern PATTERN] [-t|--timeout DURATION]
[-l|--list-tests] [-j|--num-threads NUMBER] [-q|--quiet]
[--hide-successes] [--color never|always|auto]
[--ansi-tricks ARG] [--baseline ARG] [--csv ARG]
[--svg ARG] [--stdev ARG] [--fail-if-slower ARG]
[--fail-if-faster ARG]
I tried escaping as well but it does not work. On trial and error I found that if there is a space char in the benchmark name then the "Could not parse pattern" error comes.
from tasty-bench.
harendra-kumar
commented on May 30, 2024
If I remove spaces from the benchmark names then it seems to work fine:
$ BENCH_NAME='All.Unicode.Stream/o-1-space.ungroup-group.US.unlines.S.splitOnSuffix([Word8])(1/10)'
$ $MYBENCH -p '$0 == "'$BENCH_NAME'"'
Using small input file: benchmark-tmp/in-10MB.txt
Using big input file: benchmark-tmp/in-100MB.txt
Using output file: benchmark-tmp/out.txt
All
Unicode.Stream/o-1-space
ungroup-group
US.unlines.S.splitOnSuffix([Word8])(1/10): OK (0.61s)
192 ms ± 6.5 ms
All 1 tests passed (0.61s)
But this means we will have to change names of hundreds of benchmarks. Is this a bug? Is there a workaround for this?
from tasty-bench.
harendra-kumar
commented on May 30, 2024
Ok, shell escaping is really hard to understand. This one worked even with spaces in the benchmark name:
$ $MYBENCH -p '$0 == "'"$BENCH_NAME"'"'
Using small input file: benchmark-tmp/in-10MB.txt
Using big input file: benchmark-tmp/in-100MB.txt
Using output file: benchmark-tmp/out.txt
All
Unicode.Stream/o-1-space
ungroup-group
US.unlines.S.splitOnSuffix([Word8]) (1/10): OK (0.60s)
191 ms ± 4.6 ms
All 1 tests passed (0.60s)
from tasty-bench.
harendra-kumar
commented on May 30, 2024
@Bodigrim you may want to change your sed incantation example so that it works even with spaces in the name.
from tasty-bench.
Bodigrim
commented on May 30, 2024
Ah, interesting, #18 (comment) works in my shell as is, because I happened to use zsh. If I switch to bash, it fails with a parse error. Shell escaping is hard indeed.
from tasty-bench.
Bodigrim
commented on May 30, 2024
Thanks, updated escaping in ab49d86.
from tasty-bench.
harendra-kumar
commented on May 30, 2024
Benchmark names containing double quotes are still giving me trouble. It gives the same error "Could not parse pattern". I tried escaping double quotes with a backslash, but did not work.
from tasty-bench.
harendra-kumar
commented on May 30, 2024
Ok, escaping double quotes actually worked, earlier I actually did not use the escaped shell value, my bad.
from tasty-bench.
harendra-kumar
commented on May 30, 2024
The last issue was with benchmark names using backslashes e.g. "splitOn \n". I had to use the shell read -r instead of read (otherwise it eats the backslashes) and then escape the backslashes using another backslash in the pattern.
You may want to use read -r in your example to preserve the backslashes from the input.
from tasty-bench.
Bodigrim
commented on May 30, 2024
Ha, running ShellCheck on my original snippet spots both issues:
In shell.sh line 4:
$MYBENCH -l | sed -e 's/[\"]/\\\\\\&/g' | while read name; do $MYBENCH -p '$0 == "'$name'"'; done
^--^ SC2162: read without -r will mangle backslashes.
^-------^ SC2016: Expressions don't expand in single quotes, use double quotes for that.
^---^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
$MYBENCH -l | sed -e 's/[\"]/\\\\\\&/g' | while read name; do $MYBENCH -p '$0 == "'"$name"'"'; done
from tasty-bench.
harendra-kumar
commented on May 30, 2024
Nice! I forgot about shellcheck, could have saved me some time.
from tasty-bench.
Related Issues (20)
- When running benchmarks against a baseline, it is unclear whether the baseline was compared against HOT 1
- Graceful degradation for Windows without chcp 65001
- Misleadling verbalization of speed up / slow down HOT 4
- Think about benchmarking of linear and unlifted data HOT 1
- Even less dependencies? HOT 8
- CSV reporter doesn't show baseline comparison HOT 5
- Benchmarking a memoized function HOT 2
- Estimate standard deviation for memory statistics HOT 2
- Output of the benchmark function is retained in memory HOT 11
- Custom, ad-hoc metrics? HOT 4
- Allow running benchmarks a given number of times. HOT 8
- Recommend mitigations for benchmark instability introduced by GHCs SpecConstr. HOT 5
- Benchmarks sometimes get stuck HOT 3
- Consider benchmarking until GCStats stabilise when given +RTS -T HOT 1
- Excessive inlining may optimize away the function to benchmark HOT 12
- How to remove "All" from every entry HOT 6
- Subtract benchmark baseline HOT 6
- Unhandled resource exception on pure IO action HOT 6
- Print more digits? HOT 6
- Some benchmarks written for `criterion` just complete immediately with `tasty-bench`. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tasty-bench.