sovrasov / flops-counter.pytorch Goto Github PK
View Code? Open in Web Editor NEWFlops counter for convolutional networks in pytorch framework
License: MIT License
Flops counter for convolutional networks in pytorch framework
License: MIT License
global CUSTOM_MODULES_MAPPING = custom_modules_hooks
^
SyntaxError: invalid syntax
flops_model.eval()
will return NoneType
object.
Until I decompose the line 15 into:
flops_model.eval()
flops_model.start_flops_count()
the program processes properly.
Hi @sovrasov ,
Why does the conv_flops_counter_hook
not consider the stride of the convolutions? Can you please add support for that as it significantly affects the MAC count?
Can you also support upsampling layer which is used in encoder-decoder models? Thank you.
It would be nice to have a dictionary containing each layer/operations' GMac and Params that would be useful to plot and be able to see network bottlenecks.
Hi,
This is a great repo! :-)
But can you add functionality to compute the addition count and multiplication count separately?
Thanks!
Lack of support for depthwise separable convolution
Traceback (most recent call last):
File "/home/xiaofei/my_project/SiamTrackers/compute_params/flops-counter.pytorch/sample.py", line 35, in
net = pt_modelsargs.model
File "/home/xiaofei/anaconda3/envs/siamtracker/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'x'
thank you
Hello,
Does anyone encounter this issue? A negative flops.
C:\Anaconda3\envs\pytorch\lib\site-packages\ptflops\flops_counter.py:285: RuntimeWarning: overflow encountered in long_scalars
overall_conv_flops = conv_per_position_flops * active_elements_count
- Flops: -27913.09 MMac
- Params: 556.1 k
The tested network includes Convolution, Batchnorm, and Relu only.
Thank you
Hi, first of all thank you so much, that's a really helpful repo!
I'm now working with some pretrained BERT models, so I'm wondering if this flop_counter works for all types of BERT models? I noticed here's an example using BertForSequenceClassification, with input_res=(2, 128). But when I try some other BERT models, the input_res could be the big problem. If there any baseline for BERT model like you mentioned (similar to image networks) in README? Or is there any lookup table? Thank you very much!
Hey, it's a nice tool
However, I am wondering whether the return of get_model_complexity_info is correct.
Let's assume all calculations are in floating point.
1MACs = 2OPs
MAC = Mult + Add
FLOPS = 2MACS(roughly equivalent, especially those models set up by all convolutional layers)
Thus, in your code:
The returns show flops_count, params_count
However, all printing result from the function flops_to_string are in Mac
The return type is not consistent with the example as well
The units in flops_to_string are supposed to be flops instead of mac. Otherwise, half the flops to macs. Could you please check it out again?
Hello, the tool you supply is very useful, I wanna ask is there a similar tool, which can evaluate the params and the flops of a tensorflow model, such as xx.tflite or xx.pb
今天大半天都被这个flops_counter带偏了,不管做什么尝试,我的shufflenetv2和mobilenetv2的loss 始终降不下来,而acc 就和随机猜得的一样,
Hi, I run the torch.nn.MultiheadAttention model and find it is 0 MACs. The simplified code is shown as follows. Could anyone give me a hand? The pytorch version is 1.8.1.
import torch
import torch.nn as nn
from ptflops import get_model_complexity_info
class NN(nn.Module):
def __init__(self, in_size, nhead):
super().__init__()
self.net = nn.MultiheadAttention(in_size, nhead)
def forward(self, x):
x = x.transpose(0, 1)
x = self.net(x, x, x)[0]
x = x.transpose(0, 1)
return x
model = NN(512, 4)
model.eval()
with torch.no_grad():
macs, params = get_model_complexity_info(
model, (580, 512), as_strings=True, print_per_layer_stat=True, verbose=True
)
print('{:<30} {:<8}'.format('Computational complexity: ', macs))
print('{:<30} {:<8}'.format('Number of parameters: ', params))
Error message:
~/anaconda3/envs/torch1.8/lib/python3.7/site-packages/ptflops/flops_counter.py in flops_repr(self)
122 flops_to_string(accumulated_flops_cost,
123 units=units, precision=precision),
--> 124 '{:.3%} MACs'.format(accumulated_flops_cost / total_flops),
125 self.original_extra_repr()])
126
ZeroDivisionError: float division by zero
Can GMac to be negative
I build custome model with almost 22 layers and I get -14.23 GMac
MACs: Multiply-Add cumulation?
FLOPs: Floating Point Operations?
Are they same? What is their relationship?
Hi, I have a training script with structure as :
model=Net()
flops, params = get_model_complexity_info(model, (3, 256, 256), as_strings=True, print_per_layer_stat=True)
print('Flops: ' + flops)
print('Params: ' + params)
training
torch.save(model)
Then when I tried to load from above saved checkpoint using torch.load('checkpoint.pth'), it occurs into an error:
AttributeError: 'Net' object has no attribute 'start_flops_count'.
Do you have any idea why I get this error and how can i solve it?
I really appreciate this flops counter. Would you guys plan to support the calculation with FP16?
please could you answer my questions
Q1- can we computing flops for a model without training the model? is there any relation between flops and training? can training affect flops? when flops can be computed?
I am asking this question because I defined a model and then I computed the flops and here is the results.
Computational complexity: 0.03 GMac
Number of parameters: 2.24 M
Q2- if we want to have flops in million, we should multiply 0.03 * 1000 ? if yes then for this case the computational complexity is 30.0 million.
Q3- what I understand from your code, Mac is flops, am I right ?
Thank you
Can anyone help me regarding how to use this code for 10 class (other than Imagenet) classification with our custom architechture ?
Why use the GMACs?GMACs is different with GFLOPs.
Hi, thanks for your tools. However, why did the result of GMac is negativae number when I calculate my own model? This is my code and output.
model =ResUNet(layers=[2,3,4,6,3])
macs, params = get_model_complexity_info(model,(1,256,256), as_strings=True,
print_per_layer_stat=True,verbose=True)
print('{:<30} {:<8}'.format('Computational complexity: ', macs))
print('{:<30} {:<8}'.format('Number of parameters: ', params))
Computational complexity: -52.1 GMac
Number of parameters: 97.23 M
How to compute FLOPS for LSTM model for skeleton data (NTU-RGBD)? Error is due to input_res parameter as for skeleton data what should be this value.
If a module contain two types of operations:
The results output by ptflops will be misleading. Since in the calculation of full module's flops:
It just counts all flops of modules() output.
But to output per layer flops:
it ignores flops which a module's parent is in supported modules.
This mismatch cause the final results pretty confusing.
I think you should at least make these two types of calculation consistent.
When I am running code, I encounter the following warnings:
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
How to avoid the warnings?
Does this already defined warning affect my code?
To get the number of FLOPs and params, use "get_model_complexity_info". But I get an Error, Illegal instruction (core dumped).
Seems like you add bias addition ops to the total FLOPs count for convolutional and RNN modules, but it looks like it is not the case for linear layers. Is that something to be addressed?
print_per_layer_stat=True, ignore_modules=[torch.nn.BatchNorm1d,torch.nn.BatchNorm2d,torch.nn.ReLU])
File "/usr/local/lib/python3.7/dist-packages/ptflops-0.6.2-py3.7.egg/ptflops/flops_counter.py", line 46, in get_model_complexity_info
File "/usr/local/lib/python3.7/dist-packages/ptflops-0.6.2-py3.7.egg/ptflops/flops_counter.py", line 141, in print_model_with_flops
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1134, in repr
extra_repr = self.extra_repr()
File "/usr/local/lib/python3.7/dist-packages/ptflops-0.6.2-py3.7.egg/ptflops/flops_counter.py", line 121, in flops_repr
ZeroDivisionError: float division by zero
from the last table only saw MACs and params, is that FLOPs doesn't needed a record? While some paper still report their FLOPs/G for comparasion.
I have two networks as following, I thought they should result in the same MACC and parameters but got different results. Any suggestion?
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cudnn as cudnn
import torchvision
import torchvision.transforms as transforms
import os
import re
from models import *
from ptflops import get_model_complexity_info
import torchvision.models as models
from tensorboardX import SummaryWriter
class NetOne(nn.Module):
def __init__(self):
super(NetOne, self).__init__()
self.conv1 = nn.Conv2d(3, 24, kernel_size=3, bias=False, groups= 1)
self.conv2 = nn.Conv2d(24, 24, kernel_size=3, bias=False, groups= 24)
def forward(self, x):
out = self.conv2((self.conv1(x)))
return out
class NetTwo(nn.Module):
def __init__(self):
super(NetTwo, self).__init__()
self.conv1 = nn.Conv2d(3, 24, kernel_size=3, bias=False, groups= 1)
self.conv2 = nn.Conv2d(24, 12, kernel_size=3, bias=False, groups=12)
self.conv3 = nn.Conv2d(24, 12, kernel_size=3, bias=False, groups=12)
def forward(self, x):
out = self.conv1(x)
out1 = self.conv2(out)
out2 = self.conv3(out)
out = torch.cat([out1,out2], 1)
return out
if __name__ == "__main__":
# Check the first network
net = NetOne()
with torch.cuda.device(0):
#net = models.densenet161()
flops, params = get_model_complexity_info(net, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
print('Flops: ' + flops)
print('Params: ' + params)
# Check the second network
net = NetTwo()
with torch.cuda.device(0):
#net = models.densenet161()
flops, params = get_model_complexity_info(net, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
print('Flops: ' + flops)
print('Params: ' + params)
And the results that I got is
NetOne(
0.001 GMac, 8e+05 MACs,
(conv1): Conv2d(0.001 GMac, 6e+05 MACs, 3, 24, kernel_size=(3, 3), stride=(1, 1), bias=False)
(conv2): Conv2d(0.0 GMac, 2e+05 MACs, 24, 24, kernel_size=(3, 3), stride=(1, 1), groups=24, bias=False)
)
Flops: 0.75 MMac
Params: 864
NetTwo(
0.001 GMac, 9e+05 MACs,
(conv1): Conv2d(0.001 GMac, 6e+05 MACs, 3, 24, kernel_size=(3, 3), stride=(1, 1), bias=False)
(conv2): Conv2d(0.0 GMac, 2e+05 MACs, 24, 12, kernel_size=(3, 3), stride=(1, 1), groups=12, bias=False)
(conv3): Conv2d(0.0 GMac, 2e+05 MACs, 24, 12, kernel_size=(3, 3), stride=(1, 1), groups=12, bias=False)
)
Flops: 0.92 MMac
Params: 1.08 k `
Thank you
Hey, Thanks for your package it is quite useful!
I have one small issue and that is that the models I use have a __str__
method defined on them, which prints a shorter version of the modules and thus the __repr__
is not used. This causes your print_per_layer_stat
argument not to work correctly.
In order to fix this, you could replace Line 144:
print(repr(model), file=ost)
This will force to use the __repr__
of the module, independent of whether a __str__
has been defined or not.
The efficientnet paper insists efficientnet-b0 is 0.39 Gflops, but the measurement is 00.2Gflops = (2*0.01 GMac)
% python
Python 3.6.5 (default, Nov 11 2019, 18:04:50)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ptflops import get_model_complexity_info
>>> from efficientnet_pytorch import EfficientNet
>>> network = EfficientNet.from_pretrained("efficientnet-b0")
Loaded pretrained weights for efficientnet-b0
>>> get_model_complexity_info(network, (3, 224, 224), print_per_layer_stat=False)
('0.01 GMac', '5.29 M')
The flops of ConvTranspose2d operation maybe not correct?
It should be calculated as same as Conv2d: K_w*K_h*C_in*C_out*O_w*O_h
when group = 1.
However your implement is K_w*K_h*C_in*C_out*I_w*I_h
as follows:
For now, I see that the GMac of each block is displayed, but for the number of parameters, only the total number is shown in the end. How can I know the exact number of parameters of each block? Would you please give me some advice? Thank you!
As I understand it, in PyTorch, Linear
is the name for a fully-connected layer. Its forward pass is a matrix multiplication of an input tensor of shape [B, M, K] by a weight tensor of shape [B, K, N], with an output with shape of [B, M, K]. The number of MACs (not counting any bias terms or other things) is B*M*N*K
. However, when I was using flops-counter.pytorch
in my work, I noticed that flops-counter.pytorch
's MAC estimates for Linear layers seemed really low relative to my hand-calculations. I think I found the issue. The current code (below) only does B*K*N
, and it does not account for the M dimension.
def linear_flops_counter_hook(module, input, output):
input = input[0]
batch_size = input.shape[0]
module.__flops__ += int(batch_size * input.shape[1] * output.shape[1])
The following is my proposed fix. At least on my particular models, it produces results that match both my intuition and my calculations. If it would be of use, I would be happy to submit the following as a pull request.
def linear_flops_counter_hook(module, input, output):
input = input[0]
M = input.shape[-2]
K = input.shape[-1]
N = output.shape[-1]
batch_size = input.shape[0]
module.__flops__ += int(batch_size * M*N*K)
I have fit a model with pixelshuffle operation to this flops calculation.
However, it cannot calculate the flops in pixelshuffle layer.
PixelShuffle(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, upscale_factor=2)
Can anyone help?
This tool is very useful, but there is still a lack of functionality in the benchmark.
so, could u add benchmark mode support?
Just like:
code
flops, params = get_model_complexity_info(net, (batch, 3, 224, 224), mode='layer', loop=1000, as_strings=True, print_per_layer_stat=True)
output
(conv3d): Conv3d(x GMac, x% average MACs, x average TFLOPS, x average fps, x, x, kernel_size=[x, x, x], stride=(x, x, x), bias=True)
or
(conv3d): Conv3d(x GMac, x% average MACs, x average TFLOPS, x average ms, x, x, kernel_size=[x, x, x], stride=(x, x, x), bias=True)
Sorry, I found another problem. When I calculate the 3D model's params, the answer is different from my own results. My own code is
def count_param(model):
param_count = 0
for param in model.parameters():
param_count += param.view(-1).size()[0]
return param_count
A 3D model's param calculated by my code is 166.21M and by your code is 160.64 M
Hello, thank you for sharing your code. Recently I'm trying to design new architectures based on Mobilenet v2, but I found the Macc calculated by your project is different from it in the paper.
In the paper:
It should be 585 MMac and 6.9M paremeters for MobilNet V2(1.4x).
However,
I get :
By the way, I get the different result by Lyken17's code
code I used:
from torchvision.models import mobilenet_v2
model = mobilenet_v2(width_mult=1.4)
from ptflops import get_model_complexity_info
flops, params = get_model_complexity_info(model, (3,224,224,), as_strings=True, print_per_layer_stat=True,)
print('{:<30} {:<8}'.format('Computational complexity: ', flops))
print('{:<30} {:<8}'.format('Number of parameters: ', params))
Could you tell me the reason for the difference? Thanks.
When installing using pip with python 3.7, the following error occurs:
ERROR: Package 'dataclasses' requires a different Python: 3.7.9 not in '>=3.6, <3.7'
Then I try to run the source code directly, and it works well with python 3.7 and python 3.6.9.
It seems that it's ok to remove the dataclasses dependency to better support python 3.7+
flops, params = get_model_complexity_info(net, input_res=(3, 256, 512), input_constructor=prepare_input, as_strings=True, print_per_layer_stat=True)
input_res的格式是什么?channel,row,col?还是channel,col,row?
After using this package, my model output is changed a little bit!
My model is 3D U-Net, and the input is a 3D image patch, the output is a 3d patch.
It took me some time to debug this and finally found that the problem is from using this package!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.