Comments (3)
代码中所有new的内存都是模型参数相关的,这些内存在整个训练过程不能释放,所以偷懒没有写释放的代码,因为训练结束进程退出自然释放。
内存不断上涨是正常的,因为新的样本不断读进来,特征维度会越来越大,存储模型参数需要的内存也越来越大。样本不会常驻内存,每条过一遍就都释放了。
你的内存占了100G,一方面可能是你的特征维度过高,一方面可能是你的-dim参数的k2过大,即向量v的维度过大。如果内存不够,需要你控制一下你的特征维度或者把k2参数调小一点。
from alphafm.
感谢回复 @CastellanZhang ,今天模型跑完了(跑了70个小时),模型文件56G,参数2.3亿行,有效参数1.56亿行,删除w为0的行,去掉训练参数只保留模型参数后模型文件压缩到了14G。
dim设的1:1:8可能有点大,我把特征做了hash化,特征空间压缩在了2�^28。感觉模型的稀疏性不是很好(为0的比例大约33%)。
是不是干掉L2正则,只用L1正则稀疏性会更好一些,效果上会有多少损失,有这方面的经验可以参考吗?
from alphafm.
干掉L2没什么用。调大L1正则即可,效果损失要自己做实验,数据不同结论不同,没法互相参考。
from alphafm.
Related Issues (20)
- 在线训练时,载入模型后,卡在 start! init end!上,并不开始模型训练 HOT 3
- 如何看到训练过程中的损失函数?logloss
- 考虑支持一下group lasso?
- predict文件的label是如何确定的? HOT 1
- 预估模型报错 load model error HOT 3
- segment fault HOT 3
- 性能跑不上去 HOT 2
- Hello, why the parameters are nan HOT 2
- 同一份数据, dim 1,1,8 比 dim 1,1,0 的AUC低很多
- 我想请教一下为什么我用alphaFM跑出来的结果auc表现远不如LR HOT 2
- fm_predict, multi-core, random order HOT 1
- 请问能否支持label为小数? HOT 1
- 模型txt转bin的时候提示:read file error
- 训练完模型 进行预测得时候 ,发现,同一个样本,每次预测出来 得分,会不一样,这是为啥? HOT 3
- no member named '_Hash_node_base' in namespace 'std::__1::__detail' HOT 4
- 请教一下,多个consumer之间看起来是不会串行的,那么使用多个consumer的意义是什么? HOT 1
- 请问多值特征如何配置 HOT 1
- 得到的模型文件如何部署到线上用有案例吗 HOT 1
- 我想咨询一下您bert多GPU项目的一点小问题,您方便给个联系方式吗?
- 如何对管道数据做中间过程转换?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from alphafm.