I might be mistaken, but the way bidirectional LSTMs are used in the encoder module does not really make sense.

In PyTorch, bidirectionality is handled by the underlying implementation of the LSTM module (e.g. by cuDNN) and is applied on the whole input sequence. As Encoder implements HR-LSTM by using a normal nn.LSTM but in a custom loop with one step at a time, the bidirectional LSTM would internally run bidirectional only on the given sequence of length one which practically would reduce it to two LSTMs being applied independently but both in forward direction.

I guess the simplest way to implement proper bidirectional handling would be to use two separate LSTMs where one processes the input in the normal and the other in the reverse direction.

Hello Sir,

From HR-VAE.py line 151, it seems that you're returning the zipped LSTM hidden states as a first output. You use it afterwards (line 309) as z. The way I understand it, this is not correct as you're supposed to sample z using the mu and logvar you produced with the linear layers in your encoder.
Is there something I'm missing here ?