Types and utility functions for summarizing Markov chain Monte Carlo simulations
Home Page: https://turinglang.org/MCMCChains.jl/
License: Other
Implementation of Julia types for summarizing MCMC simulations and utility functions for diagnostics and visualizations.
using MCMCChains
using StatsPlots
plot(chn)
See the docs for more information.
Note that this package heavily uses and adapts code from the Mamba.jl package licensed under MIT License, see License.md.
![femtocleaner[bot] avatar](https://avatars.githubusercontent.com/in/4123?v=4 "femtocleaner[bot]")
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
@JuliaRegistrator register()
It seems it is still possible to install an MCMCChain package via pkg> add MCMCChain.
This is confusing because now the package is named MCMCChains (with the s at the end), and installing MCMCChain (by accident or ignorance) gets you an older version and results in errors.
Is it possible to "unregister" MCMCChain so that only MCMCChains is listed?
Hi Cameron,
Just a quick (?) question what the long term vision is for MCMCChains.
As I pointed out elsewhere (I think it was in a PR), I noticed that many MCMCChains functions are display/show oriented, e.g. MCMCChains.hpd(). If I want to plot these bounds, e.g m2.1t.jl, I need to extract the Vector of theta values and use quantiles(). Earlier I made a similar observation for ChainSummary results.
Similarly, to clean up the NUTS chains (removing the adaptation draws), I use:
# Show corrected results (drop adaptation samples)
names = ["theta"]
a3d = chn[:theta]
chn2 = MCMCChains.Chains(a3d[1001:2000,:,:],
names,
Dict(
:parameters => names
)
)
which is a bit of extra work. Although this example at some point won't be necessary anymore, there are many other cases where further processing of the draws is needed.
I've also attempted to work directly with the AxisArrays, e.g.:
aa3d = chn.value[Axis{:var}([:theta]), Axis{:iter}(draws), Axis{:chain}(1)];
but (as I prefer not to expose this to end users) it would require encapsulating these in functions.
Thus my question is if indeed MCMCChains is primarily intended for 'display purposes' or if over time the intention is to add (or facilitate) post processing features? (I don't think there is a right or wrong answer here, at most maybe personal preferences).
It would be helpful to include an easier way to get parameter estimates out of chains.
mean(chn, :param1) for example should return the average value. Currently you'd have to do something like mean(chn[:param1].value.data, dims=1) which is pretty foolish.
I would also like this to handle cases of vector or matrix valued parameters, which should return a vector of means.
I just changed the type definition of Chains.value from
value::AxisArrayto
value::AxisArray{Union{Missing,A},3}The stored chain binary at test/sections_m10.4s.jls is now no longer of the same size -- @goedman would you mind if I wrote a slightly more robust test using random numbers?
Hi-
A few months ago, Rob and I had a brief discussion about effective sample size calculations for multiple chains. Currently, the maximum ESS is N, where N is the number of samples in a single chain. Why is this problematic? Consider the situation in which M chains consist of N samples each. The current metric does not distinguish between two extreme cases: (1) ESS for each chain is N/M and (2) ESS for each chain is N. In both cases, the combined ESS = N.
As a person with only cursory knowledge of ESS, my naive proposal is to switch to the formula used in Stan. As far as I can tell, the range of ESS is 0 to N*M, which seems desirable to me. I was wondering about the opinion of others.
In version .0.3.7, I encountered a strange error in which subsetting the chain breaks for certain sizes. Here is an example in which subsetting works:
using Turing, StatsPlots, Random
data = rand(Normal(0,1), 30)
@model model(y) = begin
μ ~ Normal(0, 10)
N = length(y)
for n in 1:N
y[n] ~ Normal(μ,1)
end
end;
# Settings of the NUTS sampler.
Nsamples = 2000
#Target acceptance rate
δ = .8
#The number of adaption or warmup samples
Nadapt = 1000
#Collects sampler configuration options
specs = NUTS(Nsamples,Nadapt,δ)
# Start sampling.
chain = sample(model(data),specs)
println("Size: ",size(chain))
subChain = chain[(Nadapt+1):Nsamples,:,:]
println("Size: ",size(subChain))
Although this works and the printed dimensions are correct, note that the size of subChain is not correct when queried by size. Now, here is an example that causes an error:
# Settings of the NUTS sampler.
Nsamples = 3000
#Target acceptance rate
δ = .8
#The number of adaption or warmup samples
Nadapt = 2000
#Collects sampler configuration options
specs = NUTS(Nsamples,Nadapt,δ)
# Start sampling.
chain = sample(model(data),specs)
println("Size: ",size(chain))
subChain = chain[(Nadapt+1):Nsamples,:,:]
Here is the error:
Object of type Chains, with data of type 1000×7×1 Array{Union{Missing, Float64},3}
Log evidence = 0.0
Iterations = 2001:3000
Thinning interval = 1
Chains = 1
Samples per chain = 1000
internals = elapsed, epsilon, eval_num, lf_eps, lf_num, lp
parameters = μ
Error showing value of type Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}:
ERROR: BoundsError: attempt to access 1000×7×1 Array{Union{Missing, Float64},3} at index [1:1500, [7], [1]]
Stacktrace:
[1] throw_boundserror(::Array{Union{Missing, Float64},3}, ::Tuple{UnitRange{Int64},Array{Int64,1},Array{Int64,1}}) at ./abstractarray.jl:484
[2] checkbounds at ./abstractarray.jl:449 [inlined]
[3] _getindex at ./multidimensional.jl:641 [inlined]
[4] getindex at ./abstractarray.jl:927 [inlined]
[5] getindex at /home/dfish/.julia/packages/AxisArrays/G6pZY/src/indexing.jl:100 [inlined]
[6] getindex(::AxisArrays.AxisArray{Union{Missing, Float64},3,Array{Union{Missing, Float64},3},Tuple{AxisArrays.Axis{:iter,StepRange{Int64,Int64}},AxisArrays.Axis{:var,Array{String,1}},AxisArrays.Axis{:chain,Array{Int64,1}}}}, ::UnitRange{Int64}, ::Array{String,1}, ::Array{Int64,1}) at /home/dfish/.julia/packages/AxisArrays/G6pZY/src/indexing.jl:117
[7] getindex(::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}, ::UnitRange{Int64}, ::String, ::Int64) at /home/dfish/.julia/packages/MCMCChains/g18Mi/src/chains.jl:191
[8] #ess#193(::Bool, ::Array{Symbol,1}, ::Int64, ::Function, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}) at /home/dfish/.julia/packages/MCMCChains/g18Mi/src/stats.jl:205
[9] #ess at ./none:0 [inlined]
[10] #summarystats#200(::Bool, ::Bool, ::Array{Symbol,1}, ::Symbol, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}) at /home/dfish/.julia/packages/MCMCChains/g18Mi/src/stats.jl:305
[11] summarystats at /home/dfish/.julia/packages/MCMCChains/g18Mi/src/stats.jl:295 [inlined]
[12] show(::IOContext{REPL.Terminals.TTYTerminal}, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}) at /home/dfish/.julia/packages/MCMCChains/g18Mi/src/chains.jl:317
[13] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}) at ./sysimg.jl:194
[14] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:131
[15] display(::REPL.REPLDisplay, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:135
[16] display(::Any) at ./multimedia.jl:287
[17] #15 at /home/dfish/.julia/packages/Media/ItEPc/src/compat.jl:28 [inlined]
[18] hookless(::getfield(Media, Symbol("##15#16")){Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}}) at /home/dfish/.julia/packages/Media/ItEPc/src/compat.jl:14
[19] render at /home/dfish/.julia/packages/Media/ItEPc/src/compat.jl:27 [inlined]
[20] render(::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}) at /home/dfish/.julia/packages/Media/ItEPc/src/system.jl:160
[21] display(::Media.DisplayHook, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(:samples, :hashedsummary),Tuple{Array{Turing.Utilities.Sample,1},Base.RefValue{Tuple{UInt64,ChainDataFrame}}}}}) at /home/dfish/.julia/packages/Media/ItEPc/src/compat.jl:9
[22] display(::Any) at ./multimedia.jl:287
[23] #invokelatest#1 at ./essentials.jl:742 [inlined]
[24] invokelatest at ./essentials.jl:741 [inlined]
[25] print_response(::IO, ::Any, ::Any, ::Bool, ::Bool, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:155
[26] print_response(::REPL.AbstractREPL, ::Any, ::Any, ::Bool, ::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:140
[27] (::getfield(REPL, Symbol("#do_respond#38")){Bool,getfield(REPL, Symbol("##48#57")){REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:714
[28] #invokelatest#1 at ./essentials.jl:742 [inlined]
[29] invokelatest at ./essentials.jl:741 [inlined]
[30] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2273
[31] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
[32] run_repl(::REPL.AbstractREPL, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
[33] (::getfield(Base, Symbol("##734#736")){Bool,Bool,Bool,Bool})(::Module) at ./client.jl:362
[34] #invokelatest#1 at ./essentials.jl:742 [inlined]
[35] invokelatest at ./essentials.jl:741 [inlined]
[36] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:346
[37] exec_options(::Base.JLOptions) at ./client.jl:284
[38] _start() at ./client.jl:436
Currently, MCMCChains does not have built in functions for sampling posterior values from chain objects. As discussed on discourse, having a sampling function or framework could be useful for generating posterior predictive distributions.
MCMCChains could use some model selection tools, like
Further recommendations are welcome, I'll add them to the list as they arrive.
Currently a Chains object stores a bunch of data in a more or less equal format. For example, describe(chain) might return something like this from a model generated with Turing:
Iterations = 1:1000
Thinning interval = 1
Chains = 1
Samples per chain = 1000
Empirical Posterior Estimates:
Mean SD Naive SE MCSE ESS
m 1.1715753065 0.98994393 0.031304776 0.082612482 143.591870
lf_num 0.0060000000 0.18973666 0.006000000 0.006000000 1000.000000
s 2.3994084547 2.79358857 0.088341027 0.233425460 143.228171
elapsed 0.0027014705 0.03728031 0.001178907 0.001703629 478.860457
epsilon 0.7113239044 0.56295227 0.017802114 0.041734566 181.949702
lp -5.9499451689 1.50461250 0.047580025 0.138894319 117.349264
eval_num 12.6320000000 11.46507846 0.362557615 0.600414301 364.629671
lf_eps 0.7113239044 0.56295227 0.017802114 0.041734566 181.949702
Quantiles:
2.5% 25.0% 50.0% 75.0% 97.5%
m -0.79725744431 0.657272530 1.148128006 1.647442905 3.351685253
lf_num 0.00000000000 0.000000000 0.000000000 0.000000000 0.000000000
s 0.57576571615 1.096100078 1.616797620 2.460184551 10.796821566
elapsed 0.00027849772 0.000659921 0.000758211 0.000969643 0.003740704
epsilon 0.17805198440 0.601473075 0.601473075 0.659085622 2.040240522
lp -10.48666390455 -6.448987120 -5.382348703 -4.953985494 -4.634529304
eval_num 4.00000000000 10.000000000 10.000000000 10.000000000 46.000000000
lf_eps 0.17805198440 0.601473075 0.601473075 0.659085622 2.040240522
Some of those values are parameters, and some are other variables generated during sampling. I would like to see a scenario where m and s are visually separated (and sorted alphabetically, see #24) from values like lp, elapsed, and epsilon.
@yebai suggested using an underscore prefix to denote "internal" values like lp, such that those values with underscores at the start of their names can be placed separately from the parameters sampled.
I'm not sure I like this idea -- I would prefer a general dictionary format (though maybe that's too slow, thoughts?) where you can specify "sections" of a ChainSummary object to report. This would work for cases where the ChainSummary object is used for other things like printing out diagnostics (heideldiag) or what have you.
As an example, imagine storing all the parameters in chain.variable_dict[:Parameters] and all the non-parameters in chain.variable_dict[:Internals]. Then, when printing a ChainSummary, we might see something more like:
Quantiles:
2.5% 25.0% 50.0% 75.0% 97.5%
Parameters:
m -0.79725744431 0.657272530 1.148128006 1.647442905 3.351685253
s 0.57576571615 1.096100078 1.616797620 2.460184551 10.796821566
Internals:
lf_num 0.00000000000 0.000000000 0.000000000 0.000000000 0.000000000
elapsed 0.00027849772 0.000659921 0.000758211 0.000969643 0.003740704
epsilon 0.17805198440 0.601473075 0.601473075 0.659085622 2.040240522
lp -10.48666390455 -6.448987120 -5.382348703 -4.953985494 -4.634529304
eval_num 4.00000000000 10.000000000 10.000000000 10.000000000 46.000000000
lf_eps 0.17805198440 0.601473075 0.601473075 0.659085622 2.040240522
I wanted to get thoughts from not only the Turing people but also anyone else who uses MCMCChains, like @goedman. It's a very rudimentary idea, but I'm open to suggestions on how to allow for the separate printing and possibly processing. An adjustment like this might make it easier for Turing to grab parameter estimates after NUTS, so that adaptation period samples aren't included in the chain.
Comments encouraged.
Chains need a proper show(IO, Chains) method.
julia> chain = sample(f(1), HMC(500,0.01,1))
[HMC] Sampling...100% Time: 0:00:03
[HMC] Finished with
Running time = 2.8699147269999976;
Accept rate = 1.0;
#lf / sample = 0.998;
#evals / sample = 2.998;
pre-cond. metric = [1.0].
Turing.Chain{AbstractRange{Int64}}(0.0, Turing.Sample[Sample(0.002, Dict{Symbol,Any}(:C=>0.844339,:lf_num=>0,:A=>1.04611,:B=>1.00852,:elapsed=>0.000361018,:epsilon=>0.01,:lp=>-5.58795,:eval_num=>2,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.848448,:lf_num=>1,:A=>1.05759,:B=>1.02014,:elapsed=>2.76864,:epsilon=>0.01,:lp=>-5.61528,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.846487,:lf_num=>1,:A=>1.064,:B=>1.03292,:elapsed=>0.000542769,:epsilon=>0.01,:lp=>-5.63354,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.842883,:lf_num=>1,:A=>1.07216,:B=>1.02505,:elapsed=>0.000310945,:epsilon=>0.01,:lp=>-5.63111,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.828432,:lf_num=>1,:A=>1.07779,:B=>1.03559,:elapsed=>0.00023048,:epsilon=>0.01,:lp=>-5.63594,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.808486,:lf_num=>1,:A=>1.07735,:B=>1.04103,:elapsed=>0.000177207,:epsilon=>0.01,:lp=>-5.62479,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.811408,:lf_num=>1,:A=>1.09045,:B=>1.04669,:elapsed=>0.000159575,:epsilon=>0.01,:lp=>-5.64726,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.814945,:lf_num=>1,:A=>1.09428,:B=>1.04883,:elapsed=>0.000145044,:epsilon=>0.01,:lp=>-5.65657,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.793754,:lf_num=>1,:A=>1.08999,:B=>1.0585,:elapsed=>0.00014354,:epsilon=>0.01,:lp=>-5.64502,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.796201,:lf_num=>1,:A=>1.09549,:B=>1.04152,:elapsed=>0.000123191,:epsilon=>0.01,:lp=>-5.63516,:eval_num=>3,:lf_eps=>0.01)) … Sample(0.002, Dict{Symbol,Any}(:C=>0.52954,:lf_num=>1,:A=>1.2853,:B=>1.03994,:elapsed=>0.000119147,:epsilon=>0.01,:lp=>-5.6827,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.532742,:lf_num=>1,:A=>1.2792,:B=>1.03433,:elapsed=>0.000119188,:epsilon=>0.01,:lp=>-5.67075,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.546827,:lf_num=>1,:A=>1.27016,:B=>1.04616,:elapsed=>0.00011863,:epsilon=>0.01,:lp=>-5.67914,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.54944,:lf_num=>1,:A=>1.26545,:B=>1.04865,:elapsed=>0.000119486,:epsilon=>0.01,:lp=>-5.67721,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.544144,:lf_num=>1,:A=>1.2696,:B=>1.04755,:elapsed=>0.000119432,:epsilon=>0.01,:lp=>-5.67843,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.539053,:lf_num=>1,:A=>1.26228,:B=>1.03966,:elapsed=>0.000119662,:epsilon=>0.01,:lp=>-5.65816,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.536991,:lf_num=>1,:A=>1.26476,:B=>1.02714,:elapsed=>0.000119664,:epsilon=>0.01,:lp=>-5.64725,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.529395,:lf_num=>1,:A=>1.26652,:B=>1.02576,:elapsed=>0.000118504,:epsilon=>0.01,:lp=>-5.64401,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.534894,:lf_num=>1,:A=>1.26016,:B=>1.03599,:elapsed=>0.000118747,:epsilon=>0.01,:lp=>-5.64945,:eval_num=>3,:lf_eps=>0.01)), Sample(0.002, Dict{Symbol,Any}(:C=>0.53443,:lf_num=>1,:A=>1.25372,:B=>1.04135,:elapsed=>0.000119365,:epsilon=>0.01,:lp=>-5.64667,:eval_num=>3,:lf_eps=>0.01))], [0.844339 0.0 … 2.0 0.01; 0.848448 1.0 … 3.0 0.01; … ; 0.534894 1.0 … 3.0 0.01; 0.53443 1.0 … 3.0 0.01], 1:1:500, ["C", "lf_num", "A", "B", "elapsed", "epsilon", "lp", "eval_num", "lf_eps"], [1], Dict{Symbol,Any}())is hilarious :-D
The current unit test for the diagnostics functions only test if the function call works. In the future, we should add proper unit tests to ensure the resulting computations are correct.
discretediaggelmandiaggewekediagheideldiagrafterydiagHi Martin,
Just saw your email to Brian and we already had been thinking to contact the Turing team on this topic.
Right now we're kind of redoing the original Stan.jl by separating it into CmdStan.jl + a series of additional packages such as StanDataFrames.jl, StanMamba.jl and we've discussed the idea for common (Plots based) plotting, diagnostics and summary stats in Julia. StanJulia is also aiming to include storage for larger models and some other extensions such as StanLOO and StanARM.
All of the Stan related packages are part of the Github organization StanJulia.
CmdStan.jl is basically the first more or less complete Julia 1.0 package in StanJulia. I would like to switch to Chain.jl (the interface could be something like StanChain.jl to create a TuringLang/Chains object). Applications would include a using CmdStan, StanChain, Chain statement to have access to all Chain capabilities.
To maintain backwards compatibility the next version of Stan.jl (v4.0) will wrap CmdStan.jl + StanMamba.jl to be feature compatible with pre-Julia 1.0 versions.
Would this be an ok approach for us to take?
Best,
Rob
The Julia naming guidelines normally is to have the type name be plural: MCMCChains. Maybe it's too late now, but this trips me every time.
I noticed two problems with ESS. First, it appears to always be ESS=Nsamples. Second, when I excluded the warmup samples, ESS=0. Here is a simple example:
using Turing, StatsPlots, Random
Random.seed!(14)
data = rand(Normal(0,1), 30)
@model model(y) = begin
μ ~ Normal(0, 10)
N = length(y)
for n in 1:N
y[n] ~ Normal(μ,1)
end
end;
# Settings of the NUTS sampler.
Nsamples = 2000
#Target acceptance rate
δ = .8
#The number of adaption or warmup samples
Nadapt = 1000
#Collects sampler configuration options
specs = NUTS(Nsamples,Nadapt,δ)
# Start sampling.
chain = sample(model(data),specs);
plot(chain,seriestype=(:autocorplot))
#Produces ESS = 0
subset = chain[(Nadapt+1):end,:,:]
Autocorrelation is good but it ESS should be less than Nsamples.
The tag name "0.1.1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
Computing quantiles by first converting the chain to an Array then using quantiles provides a narrower interval than the describe function. A plot of the chain suggests that the narrower interval is the correct one.
describe(chain[200:end,:,:])
Quantiles
─────────────────────────────────────────────────────────
parameters
2.5% 25.0% 50.0% 75.0% 97.5%
KappaMinusTwo 23.8318 116.7405 151.3484 192.5131 499.6650
theta[1] 0.0749 0.1728 0.1973 0.2262 0.4725
theta[2] 0.0557 0.1529 0.1754 0.1992 0.3252
theta[3] 0.0985 0.1782 0.2011 0.2243 0.3579
theta[4] 0.0906 0.1557 0.1722 0.1899 0.2871
theta[5] 0.0460 0.1039 0.1187 0.1344 0.2038
w 0.0926 0.1526 0.1682 0.1851 0.2980
quantile(Array(chain[:, "w", :]), [0.025, 0.5, 0.975])
= [0.124264, 0.16864, 0.223697]
quantile(Array(chain[200:end, "w", :]), [0.025, 0.5, 0.975])
= [0.124743, 0.168218, 0.221467]
E.g. as described in https://arxiv.org/abs/1903.08008.
Hi-
As discussed here, the newest release is not registered through new new system and does not show up via ]update as a result.
Currently the diagnostic functions do not segment by section, but it would nice to include this functionality.
Currently, the Chains type has a problem specific implementation which can be done using AxisArrays. Using AxisArrays as a field of Chains would simplify the code base, e.g. getindex and setindex.
The following minimal example illustrates how the Chains type could look like if we use AxisArrays.
struct Chains{T<:Real}
value::AxisArray
logevidence::T
end
function Chains(val::AbstractArray{T,3}; ...) where T<:Real
names = [:iter, :var, :chain]
axvals = [
1:size(val, 1),
map(i -> Symbol("Param$i"), 1:size(val, 2)),
map(i -> Symbol("Chain$i"), 1:size(val, 3))
]
axs = ntuple(i -> Axis{names[i]}(axvals[i]), 3)
A = AxisArray(convert(Array{Union{Missing,T},3}, val), axs...)
return Chains{T}(A, zero(T))
end
Base.getindex(c::Chains, i...) = getindex(c.values, i...)
Base.setindex!(c::Chains, v, i...) = setindex!(c.values, v, i...)This would allow us to index an instance of Chains as follows:
chn = Chains(randn(100, 5, 2));
chn[:,"Param1",:] # Sample values for Param1 at all iterations and chains.
chn[:,1,:] # Sample values for Param1 at all iterations and chains.
chn[Axis{:iter}(1 .. 50)] # Sample values of all parameters at all chains for iterations 1 to 50Add unit tests, see tests included Mamba.
It would be nice to have a quick function that accepts a chain and a variable set and returns a corner plot. We currently do something like this in the logistic regression tutorial for Turing, but it was outside the library.
In that case, we did something like this:
# The labels to use.
l = [:student, :balance, :income]
# Extract the parameters we want to plot.
w1 = chain[:student]
w2 = chain[:balance]
w3 = chain[:income]
# Show the corner plot.
cornerplot(hcat(w1, w2, w3), compact=true, labels = l)It would be nice to be able to call cornerplot(chain, [:var1, :var2, :var3]) and get a plot back. @mkborregaard would you be willing to help on this one?
Currently, the getindex and setindex! functions are very fragile and sometimes do not behave as expected. We need a thorough refactoring of these functions!
This code gives an error displaying Chains with Turing and MCMCChains master.
using Turing
@model gdemo(x, y) = begin
s ~ InverseGamma(2,3)
m ~ Normal(0,sqrt(s))
x ~ Normal(m, sqrt(s))
y ~ Normal(m, sqrt(s))
end
sample(gdemo(2.0, 1.5), SGLD(100, 0.01))A MWE is below script (the README example) where I am trying to just plot param2 from chn1:
using MCMCChain
using StatPlots
ProjDir = dirname(@__FILE__)
cd(ProjDir)
theme(:ggplot2);
# Define the experiment
n_iter = 10;
n_name = 3;
n_chain = 2;
# experiment results
val = randn(n_iter, n_name, n_chain) .+ [1, 2, 3]';
val = hcat(val, rand(1:2, n_iter, 1, n_chain));
# construct a Chains object
chn = Chains(val);
chn1 = chn[1:size(chn, 1), 2, 1:size(chn, 3)]
# visualize the MCMC simulation results
p1 = plot(chn1)
p2 = plot(chn1, colordim = :parameter)
# save to a png file
savefig(p1, "demo-plot-parameters.png")
savefig(p2, "demo-plot-chains.png")
This used (maybe a month ago?) to work. But I know you've also been updating the plotting.
Thanks,
Rob
Currently the header(chn) function prints the logevidence field always, which is quite annoying if you only ever sample with things that don't return log evidence.
header(chn) should only print logevidence if it is nonzero.
While reproducing the coin flipping example in the tuto http://turing.ml/tutorials/0-introduction/ , trying to summarize the chain leads to an error:
julia> chain
Object of type Chains, with data of type 2000×7×1 Array{Union{Missing, Float64},3}
Log evidence = 0.0
Iterations = 1:2000
Thinning interval = 1
Chains = 1
Samples per chain = 2000
internals = elapsed, epsilon, eval_num, lf_eps, lf_num, lp
parameters = p
parameters
Mean SD Naive SE MCSE ESS
p 0.8335 0.0128 0.0003 0.0018 48.5404
julia> p_summary = MCMCChain.Chains(chain[:p])
ERROR: MethodError: no method matching MCMCChain.Chains(::MCMCChains.Chains{Union{Missing,Float64},Float64,NamedTuple{(:parameters,),Tuple{Array{String,1}}},NamedTuple{(:hashedsummary,),Tuple{Base.RefValue{Tuple{UInt64,MCMCChains.ChainSummaries}}}}})
Closest candidates are:
MCMCChain.Chains(::T<:Real, ::Array{Union{Missing, T<:Real},3}, ::AbstractRange{Int64}, ::Array{T,1} where T, ::Dict{Symbol,Int64}, ::Array{Int64,1}) where T<:Real at /home/jfmoulin/.julia/packages/MCMCChain/PSRXh/src/MCMCChain.jl:35
MCMCChain.Chains(::Array{T<:Real,3}; start, thin, names, uniquenames, chains) where T<:Real at /home/jfmoulin/.julia/packages/MCMCChain/PSRXh/src/chains.jl:13
MCMCChain.Chains(::Array{Union{Missing, T<:Real},3}; start, thin, names, uniquenames, chains) where T<:Real at /home/jfmoulin/.julia/packages/MCMCChain/PSRXh/src/chains.jl:31
...
Stacktrace:
[1] top-level scope at none:0
Hello-
In many fields, highest posterior density (hpd) is used instead of quantiles. hpd defines the smallest interval that contains x% of the posterior distribution. Both statistics are similar for symmetric, unimodal distributions, but can differ quite a bit in other cases. Some users might find this feature to be useful.
Should describe() return a df or split this between show/describe or describe/summarize (as it is now). If we choose the second option I have a slight preference for keeping describe/summarize.
Right now describe(c, sections=[:internals]) returns the :parameters section, while `summarize(c, sections=[:internals]) returns the :internals.
I encountered a problem while installing Julia on a new machine. If I install Distributions followed by MCMCChains, I get the error below. Installing the packages in the reverse order works. I'm not sure if this is a problem with MCMCChains or Distributions.
Resolving package versions...
ERROR: LoadError: Unsatisfiable requirements detected for package MCMCChains [c7f686f2]:
MCMCChains [c7f686f2] log:
├─possible versions are: [0.2.4, 0.3.0-0.3.7] or uninstalled
├─restricted by compatibility requirements with Distributions [31c24e10] to versions: 0.3.7 or uninstalled
│ └─Distributions [31c24e10] log:
│ ├─possible versions are: [0.1.0-0.1.4, 0.2.0-0.2.13, 0.3.0, 0.6.4-0.6.7, 0.7.0-0.7.6, 0.8.0-0.8.10, 0.9.0, 0.10.0-0.10.2, 0.11.0-0.11.1, 0.12.0-0.12.5, 0.13.0, 0.14.0-0.14.2, 0.15.0, 0.16.0-0.16.4, 0.17.0, 0.18.0, 0.19.1] or uninstalled
│ └─restricted to versions 0.19.1 by an explicit requirement, leaving only versions 0.19.1
├─restricted by compatibility requirements with Turing [fce5fe82] to versions: [0.2.4, 0.3.0-0.3.7], leaving only versions: 0.3.7
│ └─Turing [fce5fe82] log:
│ ├─possible versions are: [0.0.1-0.0.4, 0.1.0, 0.2.0, 0.3.0, 0.4.0-0.4.1, 0.4.3-0.4.4, 0.5.0-0.5.1, 0.6.0-0.6.2, 0.6.4-0.6.9, 0.6.11-0.6.15] or uninstalled
│ ├─restricted to versions * by an explicit requirement, leaving only versions [0.0.1-0.0.4, 0.1.0, 0.2.0, 0.3.0, 0.4.0-0.4.1, 0.4.3-0.4.4, 0.5.0-0.5.1, 0.6.0-0.6.2, 0.6.4-0.6.9, 0.6.11-0.6.15]
│ ├─restricted by julia compatibility requirements to versions: [0.5.0-0.5.1, 0.6.0-0.6.2, 0.6.4-0.6.9, 0.6.11-0.6.15] or uninstalled, leaving only versions: [0.5.0-0.5.1, 0.6.0-0.6.2, 0.6.4-0.6.9, 0.6.11-0.6.15]
│ └─restricted by compatibility requirements with Distributions [31c24e10] to versions: 0.6.15 or uninstalled, leaving only versions: 0.6.15
│ └─Distributions [31c24e10] log: see above
└─restricted by compatibility requirements with KernelDensity [5ab0869b] to versions: [0.2.4, 0.3.0-0.3.4] or uninstalled — no versions left
└─KernelDensity [5ab0869b] log:
├─possible versions are: [0.0.1-0.0.2, 0.1.0-0.1.2, 0.2.0, 0.3.0-0.3.2, 0.4.0-0.4.1, 0.5.0-0.5.1] or uninstalled
├─restricted by julia compatibility requirements to versions: 0.5.0-0.5.1 or uninstalled
└─restricted by compatibility requirements with Distributions [31c24e10] to versions: uninstalled
└─Distributions [31c24e10] log: see above
Stacktrace:
[1] #propagate_constraints!#61(::Bool, ::Function, ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/GraphType.jl:1005
[2] propagate_constraints! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/GraphType.jl:946 [inlined]
[3] #simplify_graph!#121(::Bool, ::Function, ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/GraphType.jl:1460
[4] simplify_graph! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/GraphType.jl:1460 [inlined] (repeats 2 times)
[5] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Nothing) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Operations.jl:371
[6] resolve_versions! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Operations.jl:315 [inlined]
[7] #add_or_develop#63(::Array{Base.UUID,1}, ::Symbol, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Operations.jl:1171
[8] #add_or_develop at ./none:0 [inlined]
[9] #add_or_develop#15(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:54
[10] #add_or_develop at ./none:0 [inlined]
[11] #add_or_develop#14 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:31 [inlined]
[12] #add_or_develop at ./none:0 [inlined]
[13] #add_or_develop#13 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:29 [inlined]
[14] #add_or_develop at ./none:0 [inlined]
[15] #add_or_develop#12(::Base.Iterators.Pairs{Symbol,Symbol,Tuple{Symbol},NamedTuple{(:mode,),Tuple{Symbol}}}, ::Function, ::String) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:28
[16] #add_or_develop at ./none:0 [inlined]
[17] #add#20 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:59 [inlined]
[18] add at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:59 [inlined]
[19] addPackages at /Users/christopher.fisher/.julia/packages/MathPsychACTR/mGHMn/src/MathPsychACTR.jl:6 [inlined]
[20] _broadcast_getindex_evalf at ./broadcast.jl:578 [inlined]
[21] _broadcast_getindex at ./broadcast.jl:551 [inlined]
[22] getindex at ./broadcast.jl:511 [inlined]
[23] copyto_nonleaf!(::Array{Nothing,1}, ::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Tuple{Base.OneTo{Int64}},typeof(MathPsychACTR.addPackages),Tuple{Base.Broadcast.Extruded{Array{String,1},Tuple{Bool},Tuple{Int64}}}}, ::Base.OneTo{Int64}, ::Int64, ::Int64) at ./broadcast.jl:928
[24] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(MathPsychACTR.addPackages),Tuple{Array{String,1}}}) at ./broadcast.jl:791
[25] top-level scope at none:0
[26] include at ./boot.jl:326 [inlined]
[27] include_relative(::Module, ::String) at ./loading.jl:1038
[28] include(::Module, ::String) at ./sysimg.jl:29
[29] top-level scope at none:2
[30] eval at ./boot.jl:328 [inlined]
[31] eval(::Expr) at ./client.jl:404
[32] top-level scope at ./none:3
Currently, indexing a Chains object can return different object types depending on how you index.
# Returns a Chains object with P[1] as its only parameter
chn[:, "P[1]", :]
# Returns an array of the relevant parameter(s)
chn[:P]I'm not sure I like this. In Turing, chn[:P] was honestly much better. If I had a vector of parameters, chn[:P] would return a vector of vectors [ P[1], P[2], P[3] ] . In MCMCChains v0.3, you get a flat array where it can be difficult to assign columns to each individual [ P[1], P[2], P[3] ] . This behavior is kind of weird though and I think it's better served as a separate, explicit function.
I would like to change indexing so that these two indexing patterns are more consistent. The most obvious choice is that chn[:P] should do the same thing as chn[:,["P[1]", "P[2]", "P3]"],:] which would return a new Chains object containing only "P[1]", "P[2]", "P[3]".
Would people be okay with that? I would also like to provide an additional function like get_params or something, where get_params(chn, :P) which would return a tuple looking like (P[1], P[2], P[3]). This way, the parameter vectors can be packaged in a more accessible way to users who want to work that way (which I do).
The following provides a display error:
using Turing
@model gdemo(x) = begin
s ~ InverseGamma(2,3)
m ~ Normal(0, sqrt(s))
x[1] ~ Normal(m, sqrt(s))
x[2] ~ Normal(m, sqrt(s))
return s, m
end
chain = sample(gdemo([1.5, 2.0]), SGLD(200, 0.5))
chain[:lf_eps]Object of type Chains, with data of type 200×1×1 Array{Union{Missing, Float64},3}
Log evidence = 0.0
Iterations = 1:200
Thinning interval = 1
Chains = 1
Samples per chain = 200
internals = lf_eps
Error showing value of type Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals,),Tuple{Array{String,1}}},NamedTuple{(:hashedsummary,),Tuple{Base.RefValue{Tuple{UInt64,MCMCChains.ChainSummaries}}}}}:
ERROR: ArgumentError: Symbol[:parameters] not found in Chains name map.
Stacktrace:
[1] #Chains#17(::Bool, ::Type, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals,),Tuple{Array{String,1}}},NamedTuple{(:hashedsummary,),Tuple{Base.RefValue{Tuple{UInt64,MCMCChains.ChainSummaries}}}}}, ::Symbol) at /home/cameron/.julia/dev/MCMCChains/src/chains.jl:119
[2] Type at /home/cameron/.julia/dev/MCMCChains/src/chains.jl:103 [inlined]
[3] #31 at ./none:0 [inlined]
[4] iterate at ./generator.jl:47 [inlined]
[5] collect(::Base.Generator{Array{Symbol,1},getfield(MCMCChains, Symbol("##31#32")){Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals,),Tuple{Array{String,1}}},NamedTuple{(:hashedsummary,),Tuple{Base.RefValue{Tuple{UInt64,MCMCChains.ChainSummaries}}}}}}}) at ./array.jl:606
[6] get_sections at /home/cameron/.julia/dev/MCMCChains/src/chains.jl:390 [inlined]
[7] get_sections(::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals,),Tuple{Array{String,1}}},NamedTuple{(:hashedsummary,),Tuple{Base.RefValue{Tuple{UInt64,MCMCChains.ChainSummaries}}}}}, ::Symbol) at /home/cameron/.julia/dev/MCMCChains/src/chains.jl:395
[8] #summarystats#112(::Symbol, ::Bool, ::Bool, ::Symbol, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Chains{Union{Missing, Float64},Float64,NamedTuple{(:internals,),Tuple{Array{String,1}}},NamedTuple{(:hashedsummary,),Tuple{Base.RefValue{Tuple{UInt64,MCMCChains.ChainSummaries}}}}}) at /home/cameron/.julia/dev/MCMCChains/src/stats.jl:268
The issue is that the stats functions default to only showing the :parameters section. In the above case, it does not exist, and is throwing an error.
Hi all-
As far as I can tell, density and mixeddensity generate the same plot. It might be useful to change the function of mixeddensity to be a mixture of each chain. In other words, all of the samples would be pooled into one array and plotted. I think plotting individual chains is helpful for diagnostics and pooling is useful for generating uncluttered plots for publication.
The current implementation does not allow missing values. In favour of more flexible models, we should allow the Chains type to work with missing values.
Hi-
I am opening this issue based on a discussion in discourse. The current problem is that summary information is difficult to extract from the chain programmatically in large scale simulations. The current solution is to use StatsBase and extract from the chain's nested structure:
using StatsBase
function getSummary(ch)
chain = ch[1001:end,:,:]
return summarystats(chain).summaries[1].value
end
So the question is whether there is a simpler way for the user to work with summary data.
@cpfiffer In the end I did a final update. When copying the entire repo a lot of stuff comes into play which is sometimes an advantage and sometimes just bothersome. Next thing I'm planning to work is add serialization tests (in fact I used serialization of MCMCChains.Chains to complete the more-than-2-sections testing for MLMs where I created a section=:pooled.
When you have a moment, let me know if this works for you.
I think we can clean up some branches?I closed the renaming issue.
Did we move (as in a github repo move) or copy (the contents of) MCMCChain to MCMCChains?
Currently, describe and plot and probably some other functions tend to dump parameters without any ordering. It might be helpful if we could sort these parameters alphabetically (and partially numerically, for values like coefficients[1] and coefficients[10]). It might also be worth putting fields like lf_eps, elapsed, and maybe lp at the bottom so they aren't mixing with parameters we care about.
Here's an example of current output, just for everyone's reference:
Quantiles:
2.5% 25.0% 50.0% 75.0% 97.5%
lf_num 1.0000000×10⁰ 1.0000000000 1.000000000 1.0000000000 1.000000000
coefficients[7] 1.26080756×10⁰ 1.2639649619 1.268366138 1.2721493717 1.276686526
coefficients[9] 9.298949873×10⁻¹ 0.9388673926 0.949773031 0.9520454600 0.955547568
σ₂ 1.03487799×10¹ 18.1231871643 22.550735052 25.5324704767 27.824447549
coefficients[8] 1.606052281×10⁻¹ 0.1635674329 0.167162217 0.1693480690 0.172099955
coefficients[4] 2.32807381×10⁰ 2.3333278619 2.337033372 2.3427480452 2.351515896
coefficients[1] 4.039571048×10⁻¹ 0.4189601600 0.421228972 0.4307775296 0.441722272
intercept 2.46961248×10⁰ 2.4722156386 2.480326794 2.4837885820 2.487908149
elapsed 1.039896025×10⁻³ 0.0011747032 0.001225223 0.0013163918 0.005971485
epsilon 1.0000000×10⁻³ 0.0010000000 0.001000000 0.0010000000 0.001000000
eval_num 5.0000000×10⁰ 5.0000000000 5.000000000 5.0000000000 5.000000000
coefficients[2] 5.530248106×10⁻¹ 0.6180814643 0.714492110 0.8673753905 1.185196569
coefficients[10] 2.06649461×10⁰ 2.0712913162 2.076058492 2.0790757657 2.081939695
coefficients[3] 8.859855431×10⁻¹ 0.9255784371 0.976241622 1.0668492182 1.235812520
coefficients[6] 2.0030666×10⁰ 2.0128820775 2.018868543 2.0307170496 2.052840266
lp -2.99038618×10⁴ -6347.4383645068 -3224.486834173 -2157.4237399752 -1626.796760942
coefficients[5] 2.60456882×10⁰ 2.6102454792 2.613792467 2.6192909574 2.623083633
lf_eps 1.0000000×10⁻³ 0.0010000000 0.001000000 0.0010000000 0.001000000
I'm happy to work on this if you'd like.
We currently do not have a convenient show method. It would be good if we run summary stats the first time show is called, cache the computation and display the summary + further infos on the chains object each time the user calls show.
The MCMCChain.jl package adds a "plot" function to the global namespace based on Plots.jl. This creates a problem for those of us who do not use Plots.jl as our preferred plotting package. I think that an MCMC code should not be polluting the global namespace with plotting functions and causing issues with people who do not share the developers' preference of plotting package. An MCMC code is not a plotting module. It should be possible (and default) to get the MCMC features alone.
Would it be possible for MCMCChain.jl to explicitly only overwrite Plots.plot or maybe check whether Plots is even being used?
The current implementation has the following drawbacks:
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.