Comments (4)
There is no a general code that can test the FLOPs of all models, because different models are implemented differently.
If you want to get the FLOPs of a model, you should add the following code to the models' script:
flops, params = profile(model, inputs, verbose=False)
the model
is a torch instance, the inputs
is the input tensor for this model.
from fastbert.
Thanks! It worked!
from fastbert.
There is no a general code that can test the FLOPs of all models, because different models are implemented differently.
If you want to get the FLOPs of a model, you should add the following code to the models' script:
flops, params = profile(model, inputs, verbose=False)the
model
is a torch instance, theinputs
is the input tensor for this model.
Hi,
In your paper, is the total FLOPS of BERT 21785M? It looks very small.
Is thop capable of calculating FC layers?
I see that FLOPs depend on input sequence length. What is the FLOPs for one token?
Thanks,
ZK
from fastbert.
https://arxiv.org/pdf/2003.10555.pdf
It seems that this paper also represents the inference FLOPs of BERT-BASE
as 29 BFLOPs.
so 21785M looked quite affordable for me.
from fastbert.
Related Issues (20)
- 数据集报错 HOT 4
- what does fast_mode mean?
- How to load other pretrained bert model?
- 如何确定distill阶段的early stopping
- Unable to access datasets
- 请问你们有没有测试过在GLUE数据集上的效果?效果如何呢?
- 推理时间
- 关于batchsize对训练和测试的影响 HOT 1
- 请问论文中未来工作说的命名实体识别任务实现了吗?
- 关于论文中的FLOPs计算 HOT 1
- Share cloud is empty HOT 1
- The exact English pretraining data and Chinese pretraining data that are exact same to the BERT paper's pretraining data.
- 关于inference的效率问题 HOT 1
- I'm curious why you set segment_embedding's first dimension to 3. HOT 1
- I'm curious about the reason for making self-attention for each classifier layer.
- train students 的时候是否需要固定teacher的参数? HOT 1
- tensorflow版本 HOT 1
- batch预测 HOT 1
- 关于如何得出模型加速速率的问题
- 关于更换在其他数据集下预训练的BERT模型,效果达不到论文中所宣称的问题
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastbert.