Comments (4)
mx54039q
commented on July 20, 2024
你好, 'sos_pred_token'这个设置是和autoregressive相关的, 用来对第一个像素进行预测的, 然后将token往后移一位. 在unidirectional下这个设置是必需的, 不然会导致context model信息的泄露, 导致'当前pixel预测时, 能够看到他本身'. 在这种设置下, 你的模型是无法正常压缩和解压的. 这应该是造成你训练的模型性能变得很好的原因.
from entroformer.
xuezhongcailian
commented on July 20, 2024
谢谢。 另有一个疑惑，在代码中计算注意力时，我看到 mask = torch.tril(torch.ones((n, n))).bool()
为什么不是mask = torch.tril(torch.ones((n, n)), diagonal=-1).bool()
from entroformer.
mx54039q
commented on July 20, 2024
我已收到你的邮件。 by 钱一琛
from entroformer.
mx54039q
commented on July 20, 2024
torch.tril(torch.ones((n, n)), diagonal=-1)
如我上一个回复中所述, 已经用sos_pred_token来预测第一个像素, 所以token已经后移一位了, 所以mask的对角线上可以都为1.
from entroformer.
Related Issues (18)
- When will the code be revealed? HOT 2
- Decompressed HOT 4
- 为什么用不同的预训练模型，得到的二进制文件的大小都是一样的 HOT 3
- torchac 中的BitInputStream类 HOT 3
- 关于parallel模型的RD数据 HOT 1
- 关于bpp的计算
- 关于bpp的计算 HOT 2
- 请问您在编码部分，为什么要将图像乘2再减1呢 HOT 4
- Inquiry: Extending Model Capability for High Resolution Image Compression HOT 2
- 运行demo时出现valueerror
- I can't wait to see the code！friends, hurry up！ HOT 1
- About prob_model and training HOT 6
- About pretrain and finetune HOT 5
- The RD points on Kodak HOT 4
- Compressed binary file size HOT 7
- Checkpoint for psnr = 37.72 HOT 3
- 训练集的获取与图片选择 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from entroformer.