Some problem about testing about non-local-sparse-attention HOT 6 CLOSED

At each iteration, the random number is random. You can verify this by run two consecutive randn. Results are different.

If you start the whole program again and use the same seed, the sequence of random number is fixed. That’s how it works.

from non-local-sparse-attention.

Hi, the is because the path is not setting up correctly. After extracting the benchmark zip file, you should get a folder with name “benchmark”. Please set the —dir_data to a path pointing to the parent folder of benchmark in the demo.sh.

from non-local-sparse-attention.

Thank you. In demo.sh, I deleted '--save_result'. If I keep '--save_result', there will still be other bugs.

In addition, in line 23 of attention.py, you use the Torch. Randn function, which generates random numbers every time. Don't random numbers influence the ultimate result? How do you avoid this problem?

from non-local-sparse-attention.

Hi, it should be bug free even with the —save_results flag. What is your error message?

For the second question, the random seed is fixed in main.py to control the randomness so that results should be identical for multiple runs.

In terms of LSH, the multi-round hashing makes results highly robust.

However, the results can still be very slightly different (+-0.01) according to the pytorch doc: “Completely reproducible results are not guaranteed across PyTorch releases, individual commits, or different platforms. Furthermore, results may not be reproducible between CPU and GPU executions, even when using identical seeds.”

from non-local-sparse-attention.

Hi, I just verified that controlling the random seed 'seed=1' does limit randomness, and the output of torch. randn is the same.
However, will 'seed=1' also be fixed during training? If the output of torch. randn (shape) is the same during the training, isn't the rotation Angle in LSH fixed?

from non-local-sparse-attention.

Thank you, and I will try.

from non-local-sparse-attention.