ojus1 / time2vec-pytorch Goto Github PK

Hello,

First of all, thank you for writing and sharing this work.
I am willing to replicate the results of the paper. To make sure everything is working as it should, I have been trying to replicate the figure 4 of the paper to see if the model is indeed learning to identify the period of 7 on the toy dataset.

However, despite trying several values for the learning rate and other few modifications, I cannot obtain anything similar to the Figure 4b of the paper ; after training, all I get is a cloud of points similar to Figure 4a.

Have you been able to discover the expected frequency on the toy dataset with your code ? Am I missing something ?
Thank you very much,

Thomas

When I tried to add a new boolean feature to the toy dataset, the calculation of v1 and v2 cannot be done in the t2v() function. Then I change the definitions of self.b and self.b0 to:

    self.b = torch.randn(out_features-1)
    self.b0 = torch.randn(1)

Now everything seems to work well again.

Here is a simple version.

import torch
import torch.nn as nn

class Time2vec(nn.Module):
    def __init__(self, c_in, c_out, activation="cos"):
        super().__init__()
        self.wnbn = nn.Linear(c_in, c_out - 1, bias=True)
        self.w0b0 = nn.Linear(c_in, 1, bias=True)
        self.act = torch.cos if activation == "cos" else torch.sin

    def forward(self, x):
        part0 = self.act(self.w0b0(x))
        # print(part0.shape)
        part1 = self.act(self.wnbn(x))
        # print(part1.shape)
        return torch.cat([part0, part1], -1)


if __name__ == "__main__":
    test_x = torch.randn((1, 3, 3000))  # [N, C, L] -> batch, channel, length
    m = Time2vec(3, 10)
    out = m(test_x.permute(0,2,1))
    print(out.shape)

Can you please explain why we have got 2 dim for fc output in Model.py.
?