Add a function to generate nice reports about benchmarktools.jl HOT 6 OPEN

That's probably a good idea. I think I'd want to alter the current approach to spit out HTML rather than markdown, leaving open the possibility for cool interactive JS stuff a la criterion's awesome reports.

from benchmarktools.jl.

cc @ivirshup, you're probably working on other things but just in case you have ideas

from benchmarktools.jl.

+1 for this. I'd be happy to contribute a little bit here.

In an ideal world, I would love the following to work:

versions = ["v0.0.1", "3743c273f4439833787522242efdcda87951f6d1", "v0.0.2"]
results = Pkg.benchmark("Foo",versions)

Edit: I've opened a new issue to discuss this in detail (and deleted much of this comment to reduce noise): #16

from benchmarktools.jl.

@ahwillia This issue is just for generating reports from benchmark data, which is separate from Base.Pkg support for BenchmarkTools (though both would be nice). From previous discussions on that front, it probably doesn't make sense right now for Base.Pkg to depend on BenchmarkTools. That might change in the (far) future if Base moves to a more decentralized structure.

BenchmarkTools could natively support the functionality you mentioned, instead of it being implemented in Base.Pkg. Feel free to open an issue for it.

from benchmarktools.jl.

BenchmarkTools could natively support the functionality you mentioned

Yes, this is what I was hoping for in the short term.

Feel free to open an issue for it.

Done. Apologies for commenting here first. I misinterpreted the scope of this issue too broadly.

from benchmarktools.jl.

Perhaps not too difficult would be to generate a table similar to that in export_markdown but with the signature export_markdown(::IO, ::BenchmarkGroup). Or this functionality exists already? I'm not familiar with the internals of BenchmarkTools.

from benchmarktools.jl.