thomassutter / mopoe Goto Github PK

Thanks for your great paper. I have a confusion about the PoE part and wonder if you can clarify it.

In the paper, Definition 1 shows that the posterior is made by PoE, but i don't realize how we did it.

I tried to find the answer in the code, and came down to poe
It seems to be adding mus and vars but not product it,what is the meaning of it?

I would apologize if it's a naive question, since i just study ML for a short time,and i will appreciate it if you could clarify it.
Thank a lot.

Thanks for producing this interesting paper. I have a confusion about the MoE part and wonder if you can clarify it.

In the paper, Eq (6) is the objective to be optimized, which involves KL(sum_i expert_i || prior). Suppose expert_i is Gaussian, how do you compute the KL between a mixture-of-gaussian and the prior? I don't think this has a closed form.

I tried to find the answer in the code, and came down to moe_fusion and
mixture_component_selection. It seems to be performing some sort of sampling. Is this the same as the importance sampling in MMVAE (Shi 2019)?

Any clarification would be much appreciated. Thank you.

File "MoPoE/eval_metrics/sample_quality.py", line 16, in
import prd_score.prd_score as prd
ModuleNotFoundError: No module named 'prd_score'

Hi Thomas,
I trained your code on the polymnist dataset, and got error every time in MoPoE/utils/TBLogger.py line 66 when the training epoch progress is 8.0%.

`File "/data/home/chenxiaoyu/code/VAE/MoPoE/utils/TBLogger.py", line 66, in write_lhood_logs
self.writer.add_scalars('Likelihoods/%s'%

File "/data/home/chenxiaoyu/anaconda3/envs/pytorch_1 12.1_py 9/lib/python3.9/site-packages/tensorboardX/writer.py", line 86, in add_scalars
fw = FileWriter(logdir=fw_tag)

File "/data/home/chenxiaoyu/anaconda3/envs/pytorch_1 12.1_py 9/lib/python3.9/site-packages/tensorboardX/writer.py", line 103, in init
self.event_writer = EventFileWriter(

File "/data/home/chenxiaoyu/anaconda3/envs/pytorch_1.12.1_py39/lib/python3.9/site-packages/tensorboardX/event_file_writer.py", line 105, in init
self._event_queue = multiprocessing.Queue(max_queue_size)

File "/data/ho e/chenxiaoyu/anaconda3/envs/pytorch_1.12.1_py39/lib/python3.9/multiprocessing/context.py", line 103, iQueue
return Queue(maxsize, ctx=self.get_con ext())

File "/data/home/chenxiaoyu/anaconda3/envs/pytorch_1.12.1_py39/lib/python3.9/multiprocessing/queues.py", line 42, in init
self. reader, self._writer = connection.Pipe(duplex=False)

File "/data/ho e/chenxiaoyu/anaconda3/envs/pytorch_1.12.1_py39/lib/python3.9/multiprocessing/connection.py", line 532in Pipe
fd1, fd2 = os.pipe()
OSError: [Errn 24] Too many open files`

Could you please give me some advices? @thomassutter