skip connection structure about pytorch-opcounter HOT 6 CLOSED

Resnet flops calculated by the OpCOunter is smaller than the value in paper.

from pytorch-opcounter.

Yes, as pytorch is a dynamic graph library, it is impossible to determine skip-connection structure at nn.Module level.

One quick fix is to write a special judge for inverted residual block. I'll try to provide a more precise one via tracing computational graph.

from pytorch-opcounter.

@Lyken17 In torch1.1, a new placeholder module is provided for this kind of awkward situation.

See torch.nn.Idnetity

As I can see, you should add this Module into count_hooks.py and profile.py, counting the extra ops for skip connection.

And update your readme for special notice is important as well that one must use the standard module provided by PyTorch for skip connection to get an accurate FLOPs.

from pytorch-opcounter.

not really. Most existing codebase still defines the identity connection in forward. The real problem, as I mentioned before, is pytorch's define-by-run philosophy. The only way to record all operations is to trace the DAG.

In this case, please have a look at my labmate's project zhijian-liu/onnx-profiler

from pytorch-opcounter.

What you mentioned is quite right. I didn't say it would fix this problem without changing the codes. If someone would like to call your hook based op-counter while considering skip connection, he can represent skip connection with nn.Identity. And by adding this module to your profile we can fix the problem.

It is kind of a patch, not a global solution.

By unify the model to onnx mode is a very interesting idea, I may look deeper into it some time.

from pytorch-opcounter.

It is kind of like a user-defined patch, so I don't think you should update your project.

from pytorch-opcounter.