Question on the parameters representation in the example of Bayesian NNs about zhusuan HOT 5 CLOSED

@Xihaier Hi, Thanks for the question. lb_samples represents how many samples are used to estimate the lower bound, ll_samples represents how many samples are used to estimate the test log likelihood. epochs is the number of passes through the training set, anneal_lr_freq and anneal_lr_rate is for anneal the learning rate.

from zhusuan.

Hi Jiaxin,

It's very helpful!

If I may, could you give me some suggestions on the selection of a set of appropriate setting of the aforementioned parameters? Say if my input data set X is an N by M matrix, with N denoting the number of samples and M denoting the number of features. Y is an N by 1 vector. I am wondering if there are any tricks in the initialization of these parameters in accordance with my dataset X and Y. If not, I am more than willing to go with trial and error method.

from zhusuan.

@Xihaier In fact for Bayesian NNs we often normalize the input features to be zero mean and unit standard deviation. And for lb_samples, the principle is that more samples you use, you will get a more accurate estimation of the lower_bound. For ll_samples, it relates to the evaluation for test log likelihood, you will typically need hundreds or thousands of them to obtain a reasonable estimate. The other two (epochs, anneal..) are just parameters for every machine learning program with gradient descent. You can tune it just like you do elsewhere.

from zhusuan.

Thanks a lot, I really appreciate your kind patience and helpful instruction.

from zhusuan.

You are welcome :)

from zhusuan.