Comments (4)
Thanks for checking this!
from sasrec.
Thanks for looking into the details of the implementation. It seems the self-attention modules in the original repo (https://github.com/Kyubyong/transformer) is updated, and the code is looking mucher simpler and cleaner right now. And I remember that I've used that for SASRec in my own experiments (e.g. replacing the code in modules.py), and it seems work as well. I'm not sure if they fixed this issue. If that's the case, we may update the github code to use the newer implementation of self-attention.
from sasrec.
Thanks for looking into the details of the implementation. It seems the self-attention modules in the original repo (https://github.com/Kyubyong/transformer) is updated, and the code is looking mucher simpler and cleaner right now. And I remember that I've used that for SASRec in my own experiments (e.g. replacing the code in modules.py), and it seems work as well. I'm not sure if they fixed this issue. If that's the case, we may update the github code to use the newer implementation of self-attention.
Thank you Wang-Cheng! It's just a small issue, and yes, they have updated the transformer repo and commented this part(as they have identified our life would be easier if masks are set as arguments and easily obtained in seq level rather than in seq embedding level by src_masks = tf.math.equal(x, 0)
lol), I'll try to get it confirmed by cc the thread to transformer repo you mentioned and keep you updated for future feedback.
The issue got created just FYI, no need to update the repo for experiment usage, but it would be better to let peers know the detail.
Have a good day :)
from sasrec.
Thanks for looking into the details of the implementation. It seems the self-attention modules in the original repo (https://github.com/Kyubyong/transformer) is updated, and the code is looking mucher simpler and cleaner right now. And I remember that I've used that for SASRec in my own experiments (e.g. replacing the code in modules.py), and it seems work as well. I'm not sure if they fixed this issue. If that's the case, we may update the github code to use the newer implementation of self-attention.
Oh, I just found they did correct it before commenting the part, pls see:
as I expected, abs should be applied before reduce_sum, no need to ping the guys in that repo now :)
from sasrec.
Related Issues (20)
- self.test_logits = self.test_logits[:, -1, :] Wrong if sequence not full? HOT 1
- help,please HOT 1
- will it work on Windows? HOT 1
- SASRec performance is inferior than BPR-MF! HOT 4
- Application for adding third-party re-implemenation link on README HOT 1
- another key_mask minor bug HOT 2
- how to implement Caser?
- Could you provide the original data of Amazon dataset HOT 1
- Why put the test[u][0] into the candidate seqences? HOT 6
- How to set the amount of negative samples? HOT 1
- WarpSampler
- cuda error
- Questions about Performance of Caser
- why the causality is True
- Why does the queries are normalized while keys not when using multihead_attention?
- 提醒大家一个坑,range里边要为整数:num_batch = math.floor(len(user_train) / args.batch_size)。
- Need help pls
- How do we calculate AUC for SASRec?
- 为什么经过过滤少于5次的用户或者项目后,beauty数据集中仍然存在少于5次的用户
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sasrec.