ziyaogeng / reclearn Goto Github PK

View Code? Open in Web Editor NEW

1.8K 35.0 492.0 111.18 MB

Recommender Learning with Tensorflow2.x

License: MIT License

Python 100.00%

recommender-system python3 ctr-prediction criteo neural-network factorization-machine tensorflow2 deepfm afm xdeepfm

reclearn's Introduction

RecLearn

简体中文 | English

RecLearn (Recommender Learning) which summarizes the contents of the master branch in Recommender System with TF2.0 is a recommended learning framework based on Python and TensorFlow2.x for students and beginners. Of course, if you are more comfortable with the master branch, you can clone the entire package, run some algorithms in example, and also update and modify the content of model and layer. The implemented recommendation algorithms are classified according to two application stages in the industry:

matching recommendation stage (Top-k Recmmendation)
ranking recommendeation stage (CTR predict model)

Update

04/23/2022: update all matching model.

Installation

Package

RecLearn is on PyPI, so you can use pip to install it.

pip install reclearn

dependent environment：

python3.8+
Tensorflow2.5-GPU+/Tensorflow2.5-CPU+
sklearn0.23+

Local

Clone Reclearn to local:

git clone -b reclearn [email protected]:ZiyaoGeng/RecLearn.git

Quick Start

In example, we have given a demo of each of the recommended models.

Matching

1. Divide the dataset.

Set the path of the raw dataset:

file_path = 'data/ml-1m/ratings.dat'

Please divide the current dataset into training dataset, validation dataset and test dataset. If you use movielens-1m, Amazon-Beauty, Amazon-Games and STEAM, you can call method data/datasets/* of RecLearn directly:

train_path, val_path, test_path, meta_path = ml.split_seq_data(file_path=file_path)

meta_path indicates the path of the metafile, which stores the maximum number of user and item indexes.

2. Load the dataset.

Complete the loading of training dataset, validation dataset and test dataset, and generate several negative samples (random sampling) for each positive sample. The format of data is dictionary:

data = {'pos_item':, 'neg_item': , ['user': , 'click_seq': ,...]}

If you're building a sequential recommendation model, you need to introduce click sequences. Reclearn provides methods for loading the data for the above four datasets:

# general recommendation model
train_data = ml.load_data(train_path, neg_num, max_item_num)
# sequence recommendation model, and use the user feature.
train_data = ml.load_seq_data(train_path, "train", seq_len, neg_num, max_item_num, contain_user=True)

3. Set hyper-parameters.

The model needs to specify the required hyperparameters. Now, we take BPR model as an example:

model_params = {
        'user_num': max_user_num + 1,
        'item_num': max_item_num + 1,
        'embed_dim': FLAGS.embed_dim,
        'use_l2norm': FLAGS.use_l2norm,
        'embed_reg': FLAGS.embed_reg
    }

4. Build and compile the model.

Select or build the model you need and compile it. Take 'BPR' as an example:

model = BPR(**model_params)
model.compile(optimizer=Adam(learning_rate=FLAGS.learning_rate))

If you have problems with the structure of the model, you can call the summary method after compilation to print it out:

model.summary()

5. Learn the model and predict test dataset.

for epoch in range(1, epochs + 1):
    t1 = time()
    model.fit(
        x=train_data,
        epochs=1,
        validation_data=val_data,
        batch_size=batch_size
    )
    t2 = time()
    eval_dict = eval_pos_neg(model, test_data, ['hr', 'mrr', 'ndcg'], k, batch_size)
    print('Iteration %d Fit [%.1f s], Evaluate [%.1f s]: HR = %.4f, MRR = %.4f, NDCG = %.4f'
          % (epoch, t2 - t1, time() - t2, eval_dict['hr'], eval_dict['mrr'], eval_dict['ndcg']))

Ranking

Waiting......

Results

The experimental environment designed by Reclearn is different from that of some papers, so there may be some deviation in the results. Please refer to Experiement for details.

Matching

Model	ml-1m			Beauty			STEAM
Model	HR@10	MRR@10	NDCG@10	HR@10	MRR@10	NDCG@10	HR@10	MRR@10	NDCG@10
BPR	0.5768	0.2392	0.3016	0.3708	0.2108	0.2485	0.7728	0.4220	0.5054
NCF	0.5834	0.2219	0.3060	0.5448	0.2831	0.3451	0.7768	0.4273	0.5103
DSSM	0.5498	0.2148	0.2929	-	-	-	-	-	-
YoutubeDNN	0.6737	0.3414	0.4201	-	-	-	-	-	-
MIND(Error)	0.6366	0.2597	0.3483	-	-	-	-	-	-
GRU4Rec	0.7969	0.4698	0.5483	0.5211	0.2724	0.3312	0.8501	0.5486	0.6209
Caser	0.7916	0.4450	0.5280	0.5487	0.2884	0.3501	0.8275	0.5064	0.5832
SASRec	0.8103	0.4812	0.5605	0.5230	0.2781	0.3355	0.8606	0.5669	0.6374
AttRec	0.7873	0.4578	0.5363	0.4995	0.2695	0.3229	-	-	-
FISSA	0.8106	0.4953	0.5713	0.5431	0.2851	0.3462	0.8635	0.5682	0.6391

Ranking

Model	500w(Criteo)		Criteo
Model	Log Loss	AUC	Log Loss	AUC
FM	0.4765	0.7783	0.4762	0.7875
FFM	-	-	-	-
WDL	0.4684	0.7822	0.4692	0.7930
Deep Crossing	0.4670	0.7826	0.4693	0.7935
PNN	-	0.7847	-	-
DCN	-	0.7823	0.4691	0.7929
NFM	0.4773	0.7762	0.4723	0.7889
AFM	0.4819	0.7808	0.4692	0.7871
DeepFM	-	0.7828	0.4650	0.8007
xDeepFM	0.4690	0.7839	0.4696	0.7919

Model List

1. Matching Stage

Paper\|Model	Published	Author
BPR: Bayesian Personalized Ranking from Implicit Feedback\|MF-BPR	UAI, 2009	Steﬀen Rendle
Neural network-based Collaborative Filtering\|NCF	WWW, 2017	Xiangnan He
Learning Deep Structured Semantic Models for Web Search using Clickthrough Data\|DSSM	CIKM, 2013	Po-Sen Huang
Deep Neural Networks for YouTube Recommendations\| YoutubeDNN	RecSys, 2016	Paul Covington
Session-based Recommendations with Recurrent Neural Networks\|GUR4Rec	ICLR, 2016	Balázs Hidasi
Self-Attentive Sequential Recommendation\|SASRec	ICDM, 2018	UCSD
Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding\|Caser	WSDM, 2018	Jiaxi Tang
Next Item Recommendation with Self-Attentive Metric Learning\|AttRec	AAAAI, 2019	Shuai Zhang
FISSA: Fusing Item Similarity Models with Self-Attention Networks for Sequential Recommendation\|FISSA	RecSys, 2020	Jing Lin

2. Ranking Stage

Paper｜Model	Published	Author
Factorization Machines\|FM	ICDM, 2010	Steffen Rendle
Field-aware Factorization Machines for CTR Prediction｜FFM	RecSys, 2016	Criteo Research
Wide & Deep Learning for Recommender Systems｜WDL	DLRS, 2016	Google Inc.
Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features\|Deep Crossing	KDD, 2016	Microsoft Research
Product-based Neural Networks for User Response Prediction\|PNN	ICDM, 2016	Shanghai Jiao Tong University
Deep & Cross Network for Ad Click Predictions｜DCN	ADKDD, 2017	Stanford University｜Google Inc.
Neural Factorization Machines for Sparse Predictive Analytics\|NFM	SIGIR, 2017	Xiangnan He
Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks\|AFM	IJCAI, 2017	Zhejiang University\|National University of Singapore
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction\|DeepFM	IJCAI, 2017	Harbin Institute of Technology\|Noah’s Ark Research Lab, Huawei
xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems\|xDeepFM	KDD, 2018	University of Science and Technology of China
Deep Interest Network for Click-Through Rate Prediction\|DIN	KDD, 2018	Alibaba Group

Discussion

If you have any suggestions or questions about the project, you can leave a comment on Issue.
wechat：

reclearn's People

Contributors

Stargazers

Watchers

Forkers

jianjunyue evergreen1992 copyhao soar200 zhangzee littleqiezi yu3401 ya7788 xiaojai kiminh zoumin6 sky-zzt smileelop hjsang test1855 lyj1998 codewaltz1994 qianrenjian softthinker xrosliang yuan1235 wangningjun tonylibing qiaojj liangsheng plddxr jerrycatleung qiuxuemao syin-debug fabian0920 flowertreeml seven-xu zc123zc qianchen94 sunyanhust siqing1996 noticeable hxz2015 lhbzx1984 hervehy qjckevin straycamel247 zhangyichi1z songiu ripingit forwardpeng nothingcpd jjon-boat nekomoon404 hetixi buptx bengshaoye defaultrobot zhouyonglong chuanguiy wangboshiwbs gongqingyi sunmingjiehub ai-hub-deep-learning-fundamental scape1989 yonghangzhou nipengmath haixiaoxuan hzj1558718 alieinapril kuailedagongzai pko89403 shansyu jasperprynne owen17311528011 jackliaoall-tensorflow-related buptygz yogixrush simmonsxia nodototaofordl lanrri chengli0327 sulinyu yunwgui shinkai125 jqsl2012 ilyi1116 zjwfno1 thu-fh yehuifzu wsliuxiao growing-luo simba2017 1051741090 littleliang laisun yxk9810 dloves1314 amychai001 ericdoug-qi kaohao westqzy ruozhenzheng bluecliff qi-weichen

reclearn's Issues

考虑到在线预测的便捷性，是否考虑将特征工程部分加入到模型。

既然主要是为了工程落地的，是否考虑增加在线预测的便捷性，将特征工程部分加入到模型。比如tf.raw_ops.Bucketize... 之类的操作实现稠密特征的分桶。

请问为什么用同样的数据集运行时会报错ValueError

代码和数据集是一样的，但是运行的时候会报错：
/keras/layers/embeddings.py:128 compute_output_shape *
raise ValueError(

ValueError: "input_length" is 1, but received input has shape (None,)

对DIN数据预处理的疑问

在最后生成train_x,test_x时，为什么要把填充两列0呢？感觉没有什么意义啊

FM的second_order那项是不是少一个reduce_sum?

关于wide&deep输入疑问

def call(self, inputs, **kwargs):
    sparse_embed = tf.concat([self.embed_layers['embed_{}'.format(i)](inputs[:, i])
                              for i in range(inputs.shape[1])], axis=-1)
    x = sparse_embed  # (batch_size, field * embed_dim)
    # Wide
    wide_inputs = inputs + tf.convert_to_tensor(self.index_mapping)
    # wide_inputs = inputs
    wide_out = self.linear(wide_inputs)
    # Deep
    deep_out = self.dnn_network(x)
    deep_out = self.final_dense(deep_out)
    # out
    outputs = tf.nn.sigmoid(0.5 * wide_out + 0.5 * deep_out)
    return outputs

您好，关于wide中的inputs有一个疑问：为什么要加上self.index_mapping？

请问criteo数据集能发一份百度网盘链接吗？谢谢！数据集下载太慢了几B每秒

關於 AttRec 在 predict 時的操作

作者您好，關於 AttRec 這份模型我有些疑問，就是在預測時會使用到 Next positive item 的 embedding，照理來說預測時是不能夠把真正的 Next positive item 輸入給模型，這樣等於是把正確答案丟給模型，請問真正在預測時，要如何得到 pos_scores 和 neg_scores 呢 ?

以下是程式碼中使用到 pos_embed 時，會讓我產生的疑問：

 # combine
 pos_scores = self.w * tf.reduce_sum(tf.multiply(short_interest, pos_embed), axis=-1, keepdims=True) \+ 
                        (1 - self.w) * tf.reduce_sum(pos_long_interest, axis=-1, keepdims=True)  # (None, 1)
 neg_scores = self.w * tf.reduce_sum(tf.multiply(short_interest, neg_embed), axis=-1, keepdims=True) \+
                       (1 - self.w) * tf.reduce_sum(neg_long_interest, axis=-1, keepdims=True)  # (None, 1)
self.add_loss(tf.reduce_mean(-tf.math.log(tf.nn.sigmoid(pos_scores - neg_scores))))

关于wide&deep中的wide输入疑问

def call(self, inputs, **kwargs):
    sparse_embed = tf.concat([self.embed_layers['embed_{}'.format(i)](inputs[:, i])
                              for i in range(inputs.shape[1])], axis=-1)
    x = sparse_embed  # (batch_size, field * embed_dim)
    # Wide
    wide_inputs = inputs + tf.convert_to_tensor(self.index_mapping)
    wide_out = self.linear(wide_inputs)
    # Deep
    deep_out = self.dnn_network(x)
    deep_out = self.final_dense(deep_out)
    # out
    outputs = tf.nn.sigmoid(0.5 * wide_out + 0.5 * deep_out)
    return outputs

GPU机器上运行报错问题

你好，我在笔记本上运行WDL代码，完全没有问题，但是把代码迁移到服务器上后，报错：ValueError: as_list() is not defined on an unknown TensorShape. 请问下有谁遇到过这个问题吗？怎么解决呢

数据集百度网盘提取码不正确

SASRec训练出错

没改动代码，训练完一个epoch就报错啊
ValueError: in user code:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1224 test_function  *
    return step_function(self, iterator)
/content/dataset/SASRec/model.py:63 call  *
    dense_inputs, sparse_inputs, seq_inputs, item_inputs = inputs

ValueError: not enough values to unpack (expected 4, got 2)

ml数据集提取label的问题

在致谢中有一段
**”
在使用movielens的utils.py文件中，trans_score并不能指定正负样本，应将

data_df.loc[data_df.label < trans_score, 'label'] = 0
data_df.loc[data_df.label >= trans_score, 'label'] = 1
更改为：

data_df = data_df[data_df.label >= trans_score]
“**

因为我也是用这个方法处理ml数据集的，不明白为什么最初的方法不可以，我测试了下是没有问题的

BN的疑问

https://github.com/ZiyaoGeng/Recommender-System-with-TF2.0/blob/2aedb66620db9a69fb0616a1832c9081835149e8/NFM/model.py#L56

bn_use这个参数看上面的说明是觉得是否使用bn层的，那么这里是不是应该写成
if self.bn_use: x = self.bn(x, training=training)
不知道我是不是理解错这里的意思的了

DIEN

您好，后面会出DIEN的tf2.x的代码复现吗

可以补充dssm吗？深度召回

SASRec模型多头注意力问题

代码参数中num_heads参数为1，当我调整为2时间，就会报错：
InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: condition [512,200,1], then [1024,200,200], and else [1024,200,200] must be broadcastable
[[node sas_rec_4/encoder_layer_8/multi_head_attention_8/SelectV2 (defined at :38) ]]
[[div_no_nan/ReadVariableOp_1/_92]]
(1) Invalid argument: condition [512,200,1], then [1024,200,200], and else [1024,200,200] must be broadcastable
[[node sas_rec_4/encoder_layer_8/multi_head_attention_8/SelectV2 (defined at :38) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_21054]

FM模型实现细节请教

你好，请问下FM模型51行
inputs = inputs + tf.convert_to_tensor(self.index_mapping)
为什么要给inputs加上 index_mapping 呢？index_mapping的作用是什么？
辛苦解答

AFM

afm代码里好像没有看到一次项w0+wx

请问数据集为什么一个都没有，都要自己下载吗

SASRec 模型疑問，Key masking 以及 Query masking

作者您好，我在閱讀您的代碼時，有一個地方怎麼想都想不通，想請教您的意見

程式碼如下，當 x 通過權重 wk 和 wk 之後會得到 key 和 query，接下來就會通過下面兩個 masking，我很好奇為何這邊要針對 key 和 query 做 masking。

# Key Masking
    key_masks = tf.sign(tf.abs(tf.reduce_sum(k, axis=-1)))  # (None, seq_len)
    key_masks = tf.tile(tf.expand_dims(key_masks, 1), [1, q.shape[1], 1])  # (None, seq_len, seq_len)

    paddings = tf.ones_like(scaled_att_logits) * (-2 ** 32 + 1)
    outputs = tf.where(tf.equal(key_masks, 0), paddings, scaled_att_logits)  # (None, seq_len, seq_len)

# Query Masking
    query_masks = tf.sign(tf.abs(tf.reduce_sum(q, axis=-1)))  # (None, seq_len)
    query_masks = tf.tile(tf.expand_dims(query_masks, -1), [1, 1, q.shape[1]])  # (None, seq_len, seq_len)
    outputs *= query_masks

依照第一行 tf.sign(tf.abs(tf.reduce_sum(k, axis=-1))) 的作用是將 key embedding 中的最後一個維度進行總和、絕對值以及 tf.sign，返回的值為 0 和 1，通常將 embedding 加總後不太可能剛好是 0 ，一定是正負數，通過 tf.abs 後只剩下正數，然後通過 tf.sign 就會形成全部都是 1 的矩陣，也就是上面的 key_masks，請問為何這裏要進行加總判斷的動作 ?

DIEN的实现

deepfm模型的fm实现和FM模型不一样

两部分的FM输入不一样：
FM模型的实现以连续特征+离散one-hot特征作为输入，再映射隐向量，计算一阶二阶，符合论文；
deepFM模型的FM实现以连续特征+离散embedding特征作为输入，再次映射隐向量计算。（并且没有和deep部分共享隐向量）
此处是否有误？

din模型训练有问题

你好，din模型和你csdn上写的不一样，csdn上输入有用户id，github上没有了，为啥？而且github上的运行会报错

请问FFM为什么在md文件里说无法实际使用呢？

BatchNormalization中的training参数

https://github.com/ZiyaoGeng/Recommender-System-with-TF2.0/blob/90c2101917f92142b05ca88732e39e542647b46b/DIN/dice.py#L20

这里需要：

    def call(self, x, training=True):
        x_normed = self.bn(x, training=training)

否则你就被TensorFlow的batch normalization实现坑啦。

请问可以上传一份最小化的数据集吗？

请问请问可以上传一份最小化的数据集吗？

MF/model.py的第102，103行，参数报错

self.mf_layer = MF_layer(num_users, num_items, latent_dim, implicit, 
                                        use_bias,user_reg, item_reg, user_bias_reg, item_bias_reg)

这句出错：提示要求4-9个参数，当前传入了10个

我把这里的implicit删掉了就能运行了，不知道对不对，求大神解答！

NFM/model/call中有一个未定义的变量

作者您好： NFM/model/call 54行左右Concat里：x = tf.concat([dense_inputs, sparse_embed], axis=-1) 这里sparse_embed前面并没有定义啊，函数参数列表也没有，是怎么跑通的？

关于DeepFM的二阶特征实现

	def call(self, inputs, **kwargs):
		sparse_inputs = inputs
		# embedding
		sparse_embed = tf.concat([self.embed_layers['embed_{}'.format(i)](sparse_inputs[:, i])
                                  for i in range(sparse_inputs.shape[1])], axis=-1)  # (batch_size, embed_dim * fields)
		# wide
		sparse_inputs = sparse_inputs + tf.convert_to_tensor(self.index_mapping)
		wide_inputs = {'sparse_inputs': sparse_inputs,
					   'embed_inputs': tf.reshape(sparse_embed, shape=(-1, sparse_inputs.shape[1], self.embed_dim))}
		wide_outputs = self.fm(wide_inputs)  # (batch_size, 1)
		# deep
		deep_outputs = self.dnn(sparse_embed)
		deep_outputs = self.dense(deep_outputs)  # (batch_size, 1)
		# outputs
		outputs = tf.nn.sigmoid(tf.add(wide_outputs, deep_outputs))
		return outputs

您好关于deepfm部分，有没有实现二阶特征，就是这个链接的中提到的：https://github.com/zxxwin/tf2_deepfm

SASRec error

你好，model.py中
/data/chengt1/SASRec_Recall/model.py:55 call *
dense_inputs, sparse_inputs, seq_inputs, item_inputs = inputs

ValueError: not enough values to unpack (expected 4, got 2)

报错，请帮忙指点下，谢谢。

ml-1m数据集的具体介绍与处理的传送门消失了

楼主能补充一下么

NFM 模型的 modules.py 中未导入 Layer

from tensorflow.keras.layers import Dense, Dropout
改为
from tensorflow.keras.layers import Dense, Dropout, Layer

NFM模型实现是不是少了pooling层

def call(self, inputs):
# Inputs layer
sparse_inputs = inputs
# Embedding layer
sparse_embed = [self.embed_layers['embed_{}'.format(i)](sparse_inputs[:, i])
for i in range(sparse_inputs.shape[1])]
sparse_embed = tf.transpose(tf.convert_to_tensor(sparse_embed), [1, 0, 2]) # (None, filed_num, embed_dim)
# Bi-Interaction Layer
sparse_embed = 0.5 * (tf.pow(tf.reduce_sum(sparse_embed, axis=1), 2) -
tf.reduce_sum(tf.pow(sparse_embed, 2), axis=1)) # (None, embed_dim)
# Concat
# 这里是不是和原论文有出入，少了pooling层
#x = tf.concat([dense_inputs, sparse_embed], axis=-1)
# BatchNormalization
x = sparse_embed
x = self.bn(x, training=self.bn_use)
# Hidden Layers
x = self.dnn_network(x)
outputs = tf.nn.sigmoid(self.dense(x))
return outputs

SASRec 模型中正负样本划分问题

作者您好：
在SASRec模型的数据处理文件util.py中，有

implicit dataset

data_df.loc[data_df.label < 2, 'label'] = 0
data_df.loc[data_df.label >= 2, 'label'] = 1
可以理解为将评分小于2的视为负样本，但是在之后构造训练集和测试集的时候，这个负样本并没有考虑到，还是以用户交互过的物品作为正样本，未交互过的视为负样本。因此这儿根据评分划分正负样本没有意义吧？
不知道我的理解对不对？

關於 DIN 的 Attention weight 的疑問

作者您好，我在比較論文與程式碼的 attention unit 區塊，關於 Out Product 這塊不是很瞭解，程式碼以及截圖如下：

# q, k, out product should concat
info = tf.concat([q, k, q - k, q * k], axis=-1)

比較完程式碼之後，我才知道 Out Product 指的是把 q (advertising embedding) 和 k (behaviors embeddings) 做相乘和相減 (原本我以為數學上的 Outer product XD)，在論文中對這個操作只有提到一句話：

a(·) adds the out product of them to feed into the subsequent network, which is an explicit knowledge to help relevance modeling.

我猜測這樣做的目的是因為神經網路只具有非線性組合的能力，像相減相乘是比較難學的，不曉得作者您的想法是如何，還是有其他的 insight 可以讓我知道的。

din中seq_inputs的形状问题？

dense_inputs, sparse_inputs, seq_inputs, item_inputs = inputs这一步，seq_inputs的shape是不是[batch_size, seq_length, field_num]格式的？我用数据测试了一下，
生成的形状好像是[batch_size, 40, 2]

Caser训练过程的疑问

您好，我想请问一下为什么我在跑Caser模型的时候，得到的HR和NDCG结果是越来越差，loss越来越高呀？

Caser等序列长度不等的模型,Embedding layer的疑问。

Embedding layer里的mask_zero设为True，那么input_dim也应该是特征值总数+1吧？毕竟index为0是mask

DIN模型README文件的标题写错了，写成xDeepFM了

SASRec error——problem solved

I tried to run SASRec project, but there was something wrong. But i don't know how to solve this problem yet. Probobaly because of the version of tensorflow. Can not use tf2.3. Tf2.0 works well.

3837/3837 [==============================] - ETA: 0s - loss: 0.5068Traceback (most recent call last):
File "train.py", line 62, in
batch_size=batch_size,
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1133, in fit
return_dict=True)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1379, in evaluate
tmp_logs = test_function(iterator)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 780, in call
result = self._call(*args, **kwds)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 823, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 697, in _initialize
*args, **kwds))
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2855, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 3213, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 3075, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py", line 986, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 600, in wrapped_fn
return weak_wrapped_fn().wrapped(*args, **kwds)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py", line 973, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1224 test_function  *
    return step_function(self, iterator)
/content/drive/My Drive/Recommender-System-with-TF2.0-master/SASRec/model.py:63 call  *
    dense_inputs, sparse_inputs, seq_inputs, item_inputs = inputs

ValueError: not enough values to unpack (expected 4, got 2)

建议数据输入方式使用tfrecords

发现代码中都是使用csv、txt、pkl等方式给模型喂数据，虽然学习起来简单，但是却没办法在工业直接复用，工业数据基本都是通过tfrecords方式进行训练。只有经得起大数据量工业验证的代码，才是真正的好代码。

fix SASRec模型的mask维度问题

    def call(self, q, k, v, mask):
        q = self.wq(q)  # (None, seq_len, d_model)
        k = self.wk(k)  # (None, seq_len, d_model)
        v = self.wv(v)  # (None, seq_len, d_model)

        # split d_model into num_heads * depth, and concatenate
        q = tf.reshape(tf.concat([tf.split(q, self.num_heads, axis=2)], axis=0),
                       (-1, q.shape[1], q.shape[2] // self.num_heads))  # (None * num_heads, seq_len, d_model // num_heads)
        k = tf.reshape(tf.concat([tf.split(k, self.num_heads, axis=2)], axis=0),
                       (-1, k.shape[1], k.shape[2] // self.num_heads))  # (None * num_heads, seq_len, d_model // num_heads)
        v = tf.reshape(tf.concat([tf.split(v, self.num_heads, axis=2)], axis=0),
                       (-1, v.shape[1], v.shape[2] // self.num_heads))  # (None * num_heads, seq_len, d_model // num_heads)

        # 改动在这儿 >>process mask for multi_head
        mask_ = tf.tile(mask, multiples=[1, 1, self.num_heads])
        mask = tf.reshape(tf.concat([tf.split(mask_, self.num_heads, axis=2)], axis=0),
                       (-1, q.shape[1], q.shape[2] // self.num_heads))  # (None * num_heads, seq_len, d_model // num_heads)
        # 改动结束
        # attention
        scaled_attention = scaled_dot_product_attention(q, k, v, mask, self.causality)  # (None * num_heads, seq_len, d_model // num_heads)

        # Reshape
        outputs = tf.concat(tf.split(scaled_attention, self.num_heads, axis=0), axis=2)  # (N, seq_len, d_model)

        return outputs

SASRec中的数据处理问题

你这样写的话，数据集里面就会出现label全是0的情况。

data_df.loc[data_df.label >= 2, 'label'] = 1
data_df.loc[data_df.label < 2, 'label'] = 0

应该将这两行数据顺序颠倒一下才对吧。

data_df.loc[data_df.label < 2, 'label'] = 0
data_df.loc[data_df.label >= 2, 'label'] = 1

AttRec 資料集中正負樣本劃分的問題

作者您好，AttRec 的資料處理程式碼中，依照您 README.md 的敘述，大於等於 trans_score 作為正樣本，小於 trans_score 作為負樣本，用戶沒看過的電影也會被當成負樣本

    # implicit dataset
    data_df.loc[data_df.label < trans_score, 'label'] = 0
    data_df.loc[data_df.label >= trans_score, 'label'] = 1

但是我在看程式碼時，發現用戶評分小於 trans_score 的電影卻會被分為正樣本。我發現問題在以下程式碼：

        pos_list = df['item_id'].tolist()
        def gen_neg():
            neg = pos_list[0]
            while neg in pos_list:
                neg = random.randint(1, item_id_max)
                return neg

使用 gen_neg 生成負樣本時，只有檢查 neg 是否在 pos_list 裡，但是 pos_list 卻是包含所有的 item_id，沒有是否為正負樣本的判斷，這樣會導致把評分小於 trans_score 的資料被分為正樣本，想請問作者這邊的問題。

FFM代码问题请教

在求教二阶交叉信息的时候，embedding_lookup之后应该是一个batch,filed_num,field_num,k的矩阵，
后面做交叉之前为什么要做这一步的reduce_sum
latent_vector = tf.reduce_sum(tf.nn.embedding_lookup(self.v, inputs), axis=1)

我理解的代码应该是这样的：

latent_vector = tf.nn.embedding_lookup(self.v, inputs) # (batch_size, field_num, field_num, k)
for i in range(self.field_num):
    for j in range(i+1, self.field_num):
        second_order += tf.reduce_sum(latent_vector[:, i, j] * latent_vector[:, j, i], axis=1, keepdims=True)

Question: Extracting the embedding layer

Hi,
Great work on this.
What is the proper way here to extract the embedding layer (with the latent features of each user) after training?

Thanks

如何使用from_generator作为deepFM的输入？

AFM模型的复现问题

试了AFM原论文提供的代码，发现AFM的结果相对FM基本没有提升(复现环节出现问题)，请问你们提供的代码对AFM相比FM的有效性进行测试了吗？AFM相比FM能否获得性能的提升？

tf版本

tensorflow2.3运行，SASRec
This message will be only logged once.
952/952 [==============================] - ETA: 0s - loss: 0.4448Traceback (most recent call last):
File "train_ifeng.py", line 59, in
batch_size=batch_size,
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
return method(self, *args, **kwargs)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1133, in fit
return_dict=True)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
return method(self, *args, **kwargs)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1379, in evaluate
tmp_logs = test_function(iterator)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 780, in call
result = self._call(*args, **kwds)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 823, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 697, in _initialize
*args, **kwds))
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2855, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3213, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3075, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 986, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 600, in wrapped_fn
return weak_wrapped_fn().wrapped(*args, **kwds)
File "/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 973, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

/data/chengt1/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:1224 test_function  *
    return step_function(self, iterator)
/data/chengt1/SASRec_Recall/model.py:55 call  *
    dense_inputs, sparse_inputs, seq_inputs, item_inputs = inputs

ValueError: not enough values to unpack (expected 4, got 2)

是不认可，def summary(self)这种输入写法吗？

ml-1m数据集链接失效了

给出的几个数据集链接中ml-1m数据集失效了，能不能补一下数据集呢，感谢

ziyaogeng / reclearn Goto Github PK

reclearn's Introduction

RecLearn

Update

Installation

Package

Local

Quick Start

Matching

Ranking

Results

Matching

Ranking

Model List

1. Matching Stage

2. Ranking Stage

Discussion

reclearn's People

Contributors

Stargazers

Watchers

Forkers

reclearn's Issues

implicit dataset

Recommend Projects

Recommend Topics

Recommend Org