Hi, this project is great. I found this code <div class="snippet-clipboard-con

And I pulled a new request for it just now, see <a class="issue-link js-issue-link" da

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

About the _fix_enc_hidden() function about opennmt-py HOT 3 CLOSED

opennmt commented on May 17, 2024

About the _fix_enc_hidden() function

from opennmt-py.

Comments (3)

voidmagic commented on May 17, 2024 2

And I pulled a new request for it just now, see #63
I rewrite it as h = torch.cat([h[0:h.size(0):2], h[1:h.size(0):2]], 2), it is 10% faster on my computer.

from opennmt-py.

voidmagic commented on May 17, 2024

Hey, I also confused with question last night, but now I have figured out how it works.

Like this example:

torch.manual_seed(1)
lstm = nn.LSTM(input_size=1,
               num_layers=1,
               bidirectional=True,
               bias=False,
               hidden_size=1)

a = Variable(torch.ones(1, 1, 1))
result, context = lstm(a)
print(context)
output: 
Variable containing:
(0 ,.,.) = 
  0.1321

(1 ,.,.) = 
 -0.0980
[torch.FloatTensor of size 2x1x1]

and then change its layer to 2:

torch.manual_seed(1)
lstm = nn.LSTM(input_size=1,
               num_layers=2,
               bidirectional=True,
               bias=False,
               hidden_size=1)

a = Variable(torch.ones(1, 1, 1))
result, context = lstm(a)
print(context[0])
output: 
Variable containing:
(0 ,.,.) = 
  0.1321

(1 ,.,.) = 
 -0.0980

(2 ,.,.) = 
  0.0037

(3 ,.,.) = 
 -0.0032
[torch.FloatTensor of size 4x1x1]

Because I manually set the random seed, so the parameter of LSTM will be same at each time, and the result shows that the order of h is :
[
layer0_forward
layer0_backward
layer1_forward
layer1_backward
layer2_forward
layer2_backward
...
]
And the same for c.

from opennmt-py.