Comments (7)
While JLD is valid HDF5, I've been told JLD is painfully structured for use in other languages.
Would be interesting to know more detail. Perhaps post an issue over at JLD?
from benchmarktools.jl.
Would be interesting to know more detail. Perhaps post an issue over at JLD?
I would if it was my own complaint - I've never personally attempted to use JLD outside Julia. This was an issue somebody relayed to me in a conversation about my usage of JLD for benchmark data.
from benchmarktools.jl.
I mean, JSON is probably going to be your most natural format given the nested nature of the BenchmarkTools/groups datastructures, but I'm partial to CSV since it's easier to then ingest/visualize the data.
from benchmarktools.jl.
This was an issue somebody relayed to me in a conversation about my usage of JLD for benchmark data.
Feel free to encourage them to chime in, then.
from benchmarktools.jl.
While I'm sorry about your troubles with JLD, do keep in mind that any serializer other than the one in Base (which doesn't not guarantee backwards compatibility) may encounter trouble tracking julia. For example, the recent breakage came from how SimpleVector
gets (de)serialized (due to the change in an import
in Base). With JSON, here's what you get currently:
julia> v = Core.svec(1, 2, "foo")
svec(1,2,"foo")
julia> JSON.json(v)
"{\"length\":3}"
julia> JSON.parse(JSON.json(v))
Dict{String,Any} with 1 entry:
"length" => 3
This may not be exactly what you'd like, either. In a sense, any sufficiently-capable serializer is also guaranteed to be "fragile" with respect to changes in julia, since it will probably rely on julia for some of its mechanisms.
Perhaps the best solution is to couple tests of our various serializers to changes in julia.
from benchmarktools.jl.
Now that I've looked more closely, I agree with the idea of using a different serializer at least for BaseBenchmarks
. The primary purpose is to support testing master, and master will frequently break JLD because JLD leverages a lot of low-level type machinery. If you write out the serialization manually using the exported features of Julia, it will be more robust against changes to Base.
from benchmarktools.jl.
Since this issue came up again (kind of) in JuliaIO/JLD.jl#196, it'd be a good time to reiterate how nice it would be to have a simple JSON (de)serializer in BenchmarkTools so we could get rid of the JLD dependency (which has caused pain for non-Nanosoldier users anyway, see #53).
Here's the serialization format I came up with many moons ago, should someone want to implement it: https://github.com/JuliaCI/BenchmarkTools.jl/blob/f9376e6ffafa0c491a7906ab3b3ed4656b24f92d/src/serialization.jl. I thought I had already shared this format here or in JuliaCI/Nanosoldier.jl#36, but it looks like I had only shared it with folks on Slack.
from benchmarktools.jl.
Related Issues (20)
- Automatically create keys in BenchmarkGroup
- Memory estimate?
- Re-edit docs for multiple setup
- Add logo? HOT 26
- Run docs examples during CI instead of copypasting outputs
- `tune!` on benchmarkable with evals set. HOT 4
- @benchmark creates new symbol for each interpolation of a symbol HOT 3
- Removing `leaves` export HOT 5
- Remember keyword parameters for `tune!` HOT 3
- [Feature Request] Comparing two functions HOT 1
- Memory leak when repeatedly benchmarking HOT 5
- is `@benchmarkset` usable ? HOT 1
- Tag non-breaking release of BenchmarkTools.jl? HOT 5
- Feature request: asynchronously build a benchmark group HOT 2
- Should hardcoded version number be updated?
- cannot read benchmark JSON if run & written with `seconds=Inf`
- `judge` lacks a docstring HOT 1
- `@btime` reports wrong allocation HOT 1
- timeout measures warmup (eliminating many of the reasons to use timeout) HOT 1
- @Benchmarkset fails silently, returns null HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from benchmarktools.jl.