mcmc_recover_intervals extensions - bias + coverage about bayesplot HOT 9 OPEN

stan-dev commented on June 7, 2024

mcmc_recover_intervals extensions - bias + coverage

from bayesplot.

Comments (9)

bob-carpenter commented on June 7, 2024

If you mean the bias of an estimator, that makes sense even with "Bayesian" estimators like the posterior mean: https://en.wikipedia.org/wiki/Bias_of_an_estimator - Bob

…

On Jan 3, 2017, at 4:45 AM, wds15 ***@***.***> wrote: Hi! I just discovered the intervals function which looks great! I know that I should put all of my parameters one the unit-scale, but in practice I sometimes don't do that (even thought I should, I know). For these circumstances it would be nice to plot things as bias. So instead of showing the true values along with the intervals I would like to see an option which would allow me plot the bias. Of course, the concept of bias is shaky in a Bayesian world, but as long as I can be sure that my prior is weakly-informative, I would like to be able to do that. Another very useful extensions (I am happy to open another issue) would be a plot of the coverage when I replicate things a lot of times. BTW, these tools look awesome to me! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

from bayesplot.

betanalpha commented on June 7, 2024

Sort of — the problem is that it’s not a well-defined concept until you define a particular loss function. Which is why, for example, theta might be unbiased but theta^{2} isn’t.

…

On Jan 3, 2017, at 1:57 PM, Bob Carpenter ***@***.***> wrote: If you mean the bias of an estimator, that makes sense even with "Bayesian" estimators like the posterior mean: https://en.wikipedia.org/wiki/Bias_of_an_estimator - Bob > On Jan 3, 2017, at 4:45 AM, wds15 ***@***.***> wrote: > > Hi! > > I just discovered the intervals function which looks great! I know that I should put all of my parameters one the unit-scale, but in practice I sometimes don't do that (even thought I should, I know). For these circumstances it would be nice to plot things as bias. So instead of showing the true values along with the intervals I would like to see an option which would allow me plot the bias. > > Of course, the concept of bias is shaky in a Bayesian world, but as long as I can be sure that my prior is weakly-informative, I would like to be able to do that. > > Another very useful extensions (I am happy to open another issue) would be a plot of the coverage when I replicate things a lot of times. > > BTW, these tools look awesome to me! > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub, or mute the thread. > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

from bayesplot.

wds15 commented on June 7, 2024

Hi!

Ok, getting the bias is simple using what is there like this:

library(rstanarm)
alpha <- 1; beta <- c(-.5, .5); sigma <- 2
X <- matrix(rnorm(200), 100, 2)
y <- rnorm(100, mean = c(alpha + X %*% beta), sd = sigma)
fit <- stan_glm(y ~ X)
draws <- as.matrix(fit)
print(colnames(draws))
true <- c(alpha, beta, sigma)
mcmc_recover_intervals(draws, true)

draws_bias <- sweep(draws, 2, true)
mcmc_recover_intervals(draws_bias, rep(0, 4))

that gives me what I want for the bias. Let's see if I figure out the coverage. BTW, the package is awesome!

from bayesplot.

jgabry commented on June 7, 2024

@wds15 it would be pretty simple to add either an option or separate function to do what you did manually there. I'll try to get that into the next release, or at least up on a branch for testing soon.

…

On Tue, Jan 24, 2017 at 3:10 AM wds15 ***@***.***> wrote: Hi! Ok, getting the bias is simple using what is there like this: library(rstanarm) alpha <- 1; beta <- c(-.5, .5); sigma <- 2 X <- matrix(rnorm(200), 100, 2) y <- rnorm(100, mean = c(alpha + X %*% beta), sd = sigma) fit <- stan_glm(y ~ X) draws <- as.matrix(fit) print(colnames(draws)) true <- c(alpha, beta, sigma) mcmc_recover_intervals(draws, true) draws_bias <- sweep(draws, 2, true) mcmc_recover_intervals(draws_bias, rep(0, 4)) that gives me what I want for the bias. Let's see if I figure out the coverage. BTW, the package is awesome! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#63 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHb4Q2ovq_5fk0KXG8HYrY9-IltVBii7ks5rVbHqgaJpZM4LZdmj> .

from bayesplot.

wds15 commented on June 7, 2024

This is so simple that we could just include it as an example in the documentation maybe? What I have done is just a good use of what is there, so I am not sure if you really want to make this part of bayesplot as you need to maintain it. The gain of including it explicitly is that you encourage people to look at the their problems in this viewangle more likely as the option is more prominent. Up to you to decide.

from bayesplot.

jgabry commented on June 7, 2024

That's true. Adding an example is probably sufficient.

…

On Wed, Jan 25, 2017 at 2:42 AM wds15 ***@***.***> wrote: This is so simple that we could just include it as an example in the documentation maybe? What I have done is just a good use of what is there, so I am not sure if you really want to make this part of bayesplot as you need to maintain it. The gain of including it explicitly is that you encourage people to look at the their problems in this viewangle more likely as the option is more prominent. Up to you to decide. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#63 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHb4Q3yNPLrZ3oODQMS5cdXp0aqht6btks5rVvzpgaJpZM4LZdmj> .

from bayesplot.

wds15 commented on June 7, 2024

from bayesplot.

jgabry commented on June 7, 2024

I think the batch argument is different from what you're describing. It was intended to allow grouping parameters together that make sense to look at together or that make it easier to get the axis limits right. So for example you could put coefficients that vary by group into group-specific batches. Or put all parameters you expect to be on certain scales together in the same batch. It just affects how they are displayed in facets. Mon, Feb 6, 2017 at 3:54 PM wds15 <[email protected]> wrote:

…

So we got the bias in now which leaves the coverage. For that one would need multiple posteriors each coming from an independent fit of (usually) fake data. I am not sure of the "batch" concept is exactly that or is it? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#63 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHb4Q80797jLOSM3C-xhXBzY62xV0bFYks5rZ4hpgaJpZM4LZdmj> .

from bayesplot.

wds15 commented on June 7, 2024

I thought so. Should we then close this issue as it seems out of scope?

However, I do have to say that I have done such experiments, i.e. repeated fake-data simulations and then look at the coverage and correlation in the bias. Both of which were very interesting to learn about in the model as it did tell me a new aspect of my model how it performs under repetition. A correlated bias and the coverage is something one wants to know, I think; would be curious what others think. Possibly worth to discuss in a Stan meeting?

from bayesplot.

mcmc_recover_intervals extensions - bias + coverage about bayesplot HOT 9 OPEN

Comments (9)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent